4. Server

4.1 Introduction to servers

Serveez is a kind of server server. It allows different protocol servers to listen on various TCP/UDP ports, on ICMP sockets and on named pipes. Servers are instantiated with a certain configuration. It is possible to run multiple different servers on the same port.

This chapter covers all questions about how to write your own Internet protocol server with this package. Most of the common tasks of such a server have got a generic solution (default routines) which could be "overridden" by your own routines. There are some examples within this package. They are showing the possibilities with this package and how to implement servers.

The ‘foo’ server does not do anything at all and is of no actual use but could be a basis for a new protocol server. We are now going to describe how this specific server works. Eventually the reader might get an impression of what is going on.

For better understanding the text below we will use the following terminology:

server definition

A server definition is a svz_servertype_t structure which contains server specific members like its name, different callbacks, a single default configuration and a list of configuration items which determines what can be configured.

server configuration

A server configuration can be any kind of structure. The default server configuration must be specified within the server definition (see above). When instantiating a server (which is done via the configuration file) the configuration items specified in the server definition get processed and are put into a copy of the default configuration. Thus we get an instance.

server instance

A server instance is a copy of the server definition including the modified server configuration. A server gets instantiated by the configuration file parser. The concept of server instances has been introduced because we wanted Serveez to have the following features. A single server can have multiple instances with different behaviour due to different server configurations. A server instance can be bound to multiple port configurations. Different server instances (of the same and/or different server type) can share the same port configuration.

port configuration

A port configuration in Serveez is represented by the svz_portcfg_t structure. Depending on a shared flag it contains different type of information specifying a transport endpoint. See section Define ports, for more on this topic. It also can be a special configuration item within a server configuration. This is necessary if a server needs for some reason a remote transport endpoint. A server instance does not usually need to know about the port configuration it is bound to.

4.2 Writing servers

Serveez is meant to be a server framework. That is why it supports various ways to implement Internet servers. First of all there are some servers already included in the main serveez package. (See section Existing servers.) These are called ‘Builtin servers’. Another possibility to add a new server are ‘Embedded servers’ which are shared libraries (or DLL's) containing the server implementation. These can be dynamically loaded by Serveez at runtime. This kind of server is also called ‘Server modules’. A third possibility are the ‘Guile servers’ which allow even unexperienced schemers to write an Internet server.

This section describes each of the above possibilities in detail.

4.2.1 Embedded servers

The core library of Serveez can be used to write standalone server modules. Serveez defines a certain interface for shared libraries which contain such server modules. If Serveez detects an unknown server type (server type which is not builtin) in its configuration file it tries to load a shared library containing this server type during startup.

4.2.1.1 Prerequisites

In order to implement a server module you need an existing installation of Serveez. This can be achieved issuing the following commands:

$ ./configure --enable-shared --prefix=/usr/local
$ make
$ make install

After successful installation you are able to compile and link against the Serveez core API. The headers should be available in ‘/usr/local/include’ and the library itself (‘libserveez.so’ or ‘libserveez.dll’) is located in ‘/usr/local/lib’ if you passed the configure script ‘--prefix=/usr/local’.

4.2.1.2 Server definition

The interface mentioned in the introduction is defined via the extern declaration of the server type in the shared library of the server module. Imagine you want to implement a server type called ‘foo’. This requires the external symbol foo_server_definition in the shared library. You can achieve this inserting the following lines into your header file:

/* Either Unices. */
extern svz_servertype_t foo_server_definition;

/* Or Windows. */
__declspec (dllexport) extern svz_servertype_t foo_server_definition;

The data symbol foo_server_definition must be statically filled with the proper content (See section Builtin servers.)

4.2.2 Guile servers

This section describes the Guile interface to Serveez which provides the ability to write servers with Guile. Of course, you could do this without any help from Serveez, but it makes the task a lot easier. This interface reduces the Guile implementation of an Internet server to a simple data processor.

4.2.2.1 Special Data Types

Serveez extends Guile by various new data types which represent internal data structures of Serveez's core API.

• #<svz-servertype> represents a server type.
• #<svz-server> represents a server (an instance of a server type).
• #<svz-socket> represents a socket structure.

4.2.2.2 Passing Binary Data

The new binary data type (#<svz-binary>) provides access to any kind of unstructured data. It manages the data exchange between Guile and Serveez. There are some conversion functions for strings and lists which help to process this binary data in a more complex (guile'ish) way.

Function: binary->string binar

Converts the given binary smob binary into a string. Returns the string cell itself.

Function: string->binary strin

Converts the given string cell string into a binary smob. The data pointer of the binary smob is marked as garbage which must be free()'d in the sweep phase of the garbage collector.

Function: binary? binar

Smob test function: Returns #t if the given cell binary is an instance of the binary smob type.

Function: list->binary lis

Convert the scheme list list into a binary smob. Each of the elements of list is checked for validity. The elements can be either exact numbers in a byte's range or characters.

Function: binary->list binar

Convert the given binary smob binary into a scheme list. The list is empty if the size of binary is zero.

Function: binary-search binary needl

This routine searches through the binary smob binary for the cell needle. The latter argument can be either an exact number, character, string or another binary smob. It returns #f if the needle could not be found and a positive number indicates the position of the first occurrence of needle in the binary smob binary.

Function: binary-set! binary index valu

Set the byte at position index of the binary smob binary to the value given in value which can be either a character or an exact number.

Function: binary-ref binary inde

Obtain the byte at position index of the binary smob binary.

Function: binary-length binar

Return the size in bytes of the binary smob binary.

Function: binary-concat! binary appen

Append either the binary smob or string append onto the binary smob binary. If binary has been a simple data pointer reference it is then a standalone binary smob as returned by string->binary.

Function: binary-subset binary start en

Create a subset binary smob from the given binary smob binary. The range of this subset is specified by start and end both inclusive (thus the resulting size is = end - start + 1). With a single exception: If end is not given or specified with -1 the routine returns all data until the end of binary.

Function: binary-reverse binar

Returns a new binary smob with the reverse byte order of the given binary smob binary.

Function: binary-reverse! binar

Performs an in place reversal of the given binary smob binary and returns it.

Function: binary-long-ref binary index

Returns the long value of the binary smob binary at the array index index.

Function: binary-long-set! binary index value

Sets the long value of the binary smob binary at the array index index to the given value value. The procedure returns the previous (overridden) value.

Function: binary-int-ref binary index

Returns the int value of the binary smob binary at the array index index.

Function: binary-int-set! binary index value

Sets the int value of the binary smob binary at the array index index to the given value value. The procedure returns the previous (overridden) value.

Function: binary-short-ref binary index

Returns the short value of the binary smob binary at the array index index.

Function: binary-short-set! binary index value

Sets the short value of the binary smob binary at the array index index to the given value value. The procedure returns the previous (overridden) value.

Function: binary-char-ref binary index

Returns the char value of the binary smob binary at the array index index.

Function: binary-char-set! binary index value

Sets the char value of the binary smob binary at the array index index to the given value value. The procedure returns the previous (overridden) value.

4.2.2.3 Server Definition

In order to set up a new server type, you use the procedure (define-servertype!). This procedure takes one argument which must be an associative list specifying the server type in detail. There are optional and mandatory elements you can set up in this alist.

The following example shows the overall syntax of this procedure:

(define-servertype! '(

  ;; Mandatory: server type prefix for later use in (define-server!)
  (prefix          . "foo")

  ;; Mandatory: server type description
  (description     . "guile foo server")

  ;; Mandatory for TCP and PIPE servers: protocol detection
  (detect-proto    . foo-detect-proto)

  ;; Optional: global server type initialisation
  (global-init     . foo-global-init)

  ;; Optional: server instance initialisation
  (init            . foo-init)

  ;; Optional: server instance finalisation
  (finalize        . foo-finalize)

  ;; Optional: global server type finalisation
  (global-finalize . foo-global-finalize)

  ;; Mandatory for TCP and PIPE servers: socket connection
  (connect-socket  . foo-connect-socket)

  ;; Optional: server instance info
  (info-server     . foo-info-server)

  ;; Optional: client info
  (info-client     . foo-info-client)

  ;; Optional: server instance reset callback
  (reset           . foo-reset)
  
  ;; Optional: server instance notifier
  (notify          . foo-notify)
  
  ;; Mandatory for UDP and ICMP servers: packet handler
  (handle-request  . foo-handle-request)

  ;; Mandatory: server type configuration (may be an empty list)
  (configuration   . (

    ;; The server configuration is an alist (associative list) again.
    ;; Each item consists of an item name and a list describing the 
    ;; item itself.
    ;; Syntax: (key . (type defaultable default))
    (foo-integer       . (integer  #t 0))
    (foo-integer-array . (intarray #t (1 2 3 4 5)))
    (foo-string        . (string   #t "default-foo-string"))
    (foo-string-array  . (strarray #t ("guile" "foo" "server")))
    (foo-hash          . (hash     #t (("foo" . "bar"))))
    (foo-port          . (portcfg  #t foo-port))
    (foo-boolean       . (boolean  #t #t))
  ))))

Function: define-servertype! arg

Guile server definition: This procedure takes one argument containing the information about a new server type. If everything works fine you have a freshly registered server type afterwards. Return #t on success.

4.2.2.4 Predefined Procedures

The following subset of procedures may be used to implement a Guile server. They should be used within the callbacks defined in the (define-servertype!) procedure. Each of these callbacks gets passed the appropriate arguments needed to stuff into the following procedures. Please have a look at the example Guile servers for the details.

Function: svz:sock? soc

Returns #t if the given cell sock is an instance of a valid #<svz-socket>, otherwise #f.

Function: svz:sock:check-request sock proc

Set the check-request member of the socket structure sock to the Guile procedure proc. Returns the previously handler if there is any.

Function: svz:sock:check-oob-request sock proc

With this procedure you can setup the check-oob-request callback of the given socket structure sock. The previous callback is replaced by the proc procedure and will be returned if there was set any before. The callback is run whenever urgent data (out-of-band) has been detected on the socket.

Function: svz:sock:send-oob sock oo

This procedure expects a TCP #<svz-socket> in sock and an exact number or single character in oob. The byte in oob is sent as urgent (out-of-band) data through the underlying TCP stream. The procedure returns #t on successful completion and otherwise (either it failed to send the byte or the passed socket is not a TCP socket) #f.

Function: svz:sock:handle-request sock proc

Set the handle-request member of the socket structure sock to the Guile procedure proc. The procedure returns the previously set handler if there is any.

Function: svz:sock:boundary sock boundar

Setup the packet boundary of the socket sock. The given string value boundary can contain any kind of data. If you pass an exact number value the socket is setup to parse fixed sized packets. In fact this procedure sets the check-request callback of the given socket structure sock to a predefined routine which runs the handle-request callback of the same socket when it detected a complete packet specified by boundary. For instance you can setup Serveez to pass your handle-request procedure text lines by calling (svz:sock:boundary sock "\n").

Function: svz:sock:floodprotect sock fla

Set or unset the flood protection bit of the given socket sock. Returns the previous value of this bit (#t or #f). The flag argument must be either boolean or an exact number and is optional.

Function: svz:sock:print sock buffe

Write the string buffer buffer to the socket sock. The procedure accepts binary smobs too. Return #t on success and #f on failure.

Function: svz:sock:final-print soc

This procedure schedules the socket sock for shutdown after all data within the send buffer queue has been sent. The user should issue this procedure call right *before* the last call to (svz:sock:print).

Function: svz:sock:no-delay sock enabl

Turns the Nagle algorithm for the TCP socket sock on or off depending on the optional enable argument. Returns the previous state of this flag (#f if Nagle is active, #t otherwise). By default this flag is switched off. This socket option is useful when dealing with small packet transfer in order to disable unnecessary delays.

Function: svz:sock:send-buffer soc

Return the send buffer of the socket sock as a binary smob.

Function: svz:sock:send-buffer-size sock siz

This procedure returns the current send buffer size and fill status in bytes of the socket sock as a pair of exact numbers. If the optional argument size is given the send buffer will be set to the specified size in bytes.

Function: svz:sock:receive-buffer soc

Return the receive buffer of the socket sock as a binary smob.

Function: svz:sock:receive-buffer-size sock siz

Returns the current receive buffers size and fill status in bytes of the socket sock as a pair of exact numbers. If the optional argument size is given the receive buffer will be set to the specified size in bytes.

Function: svz:sock:receive-buffer-reduce sock lengt

Dequeue length bytes from the receive buffer of the socket sock which must be a valid #<svz-socket>. If the user omits the optional length argument, all of the data in the receive buffer gets dequeued. Returns the number of bytes actually shuffled away.

Function: svz:sock:connect host proto por

Establishes a network connection to the given host [ :port ]. If proto equals PROTO_ICMP the port argument is ignored. Valid identifiers for proto are PROTO_TCP, PROTO_UDP and PROTO_ICMP. The host argument must be either a string in dotted decimal form, a valid hostname or an exact number in host byte order. When giving a hostname this operation might be blocking. The port argument must be an exact number in the range from 0 to 65535, also in host byte order. Returns a valid #<svz-socket> or #f on failure.

Function: svz:sock:disconnected sock proc

Set the disconnected-socket member of the socket structure sock to the Guile procedure proc. The given callback runs whenever the socket is lost for some external reason. The procedure returns the previously set handler if there is one.

Function: svz:sock:kicked sock proc

Sets the kicked-socket callback of the given socket structure sock to the Guile procedure proc and returns any previously set procedure. This callback gets called whenever the socket gets closed by Serveez intentionally.

Function: svz:sock:trigger sock proc

Sets the trigger callback of the socket structure sock to the Guile procedure proc and returns any previously set procedure. The callback is run when the trigger-condition callback returned #t.

Function: svz:sock:trigger-condition sock proc

This procedure sets the trigger-condition callback for the socket structure sock to the Guile procedure proc. It returns the previously set procedure if available. The callback is run once every server loop indicating whether the trigger callback should be run or not.

Function: svz:sock:idle sock proc

This procedure sets the idle callback of the socket structure sock to the Guile procedure proc. It returns any previously set procedure. The callback is run by the periodic task scheduler when the idle-counter of the socket structure drops to zero. If this counter is not zero it gets decremented once a second. The idle callback can reset idle-counter to some value and thus can re-schedule itself for a later task.

Function: svz:sock:idle-counter sock counte

This functions returns the socket structure sock's current idle-counter value. If the optional argument counter is given, the function sets the idle-counter. Please have a look at the (svz:sock:idle) procedure for the exact meaning of this value.

Function: svz:sock:data sock dat

Associate any kind of data (any Guile object) given in the argument data with the socket sock. The data argument is optional. The procedure always returns a previously stored value or an empty list.

Function: svz:sock:parent sock paren

Return the given socket's sock parent and optionally set it to the socket parent. The procedure returns either a valid #<svz-socket> object or an empty list.

Function: svz:sock:referrer sock referre

Return the given socket's sock referrer and optionally set it to the socket referrer. The procedure returns either a valid #<svz-socket> or an empty list.

Function: svz:sock:server sock serve

This procedure returns the #<svz-server> object associated with the given argument sock. The optional argument server can be used to redefine this association and must be a valid #<svz-server> object. For a usual socket callback like connect-socket or handle-request, the association is already in place. But for sockets created by (svz:sock:connect), you can use it in order to make the returned socket object part of a server.

Function: svz:sock:local-address sock addres

This procedure returns the current local address as a pair like (host . port) with both entries in network byte order. If you pass the optional argument address, you can set the local address of the socket sock.

Function: svz:sock:remote-address sock addres

This procedure returns the current remote address as a pair like (host . port) with both entries in network byte order. If you pass the optional argument address, you can set the remote address of the socket sock.

Function: svz:sock:find iden

The given argument ident must be a pair of numbers where the car is a #<svz-socket>'s identification number and the cdr the version number. The procedure returns either the identified #<svz-socket> or #f if the given combination is not valid anymore.

Function: svz:sock:ident soc

This procedure returns a pair of numbers identifying the given #<svz-socket> sock which can be passed to (svz:sock:find). This may be necessary when you are passing a #<svz-socket> through coserver callback arguments in order to verify that the passed #<svz-socket> is still valid when the coserver callback runs.

Function: svz:sock:protocol soc

Returns one of the PROTO_TCP, PROTO_UDP, PROTO_ICMP, PROTO_RAW or PROTO_PIPE constants indicating the type of the socket structure sock. If there is no protocol information available the procedure returns #f.

Function: svz:read-file port siz

This procedure returns either a binary smob containing a data block read from the open input port port with a maximum number of size bytes or the end-of-file object if the underlying ports end has been reached. The size of the returned binary smob may be less than the requested size size if it exceed the current size of the given port port. The procedure throws an exception if an error occurred while reading from the port.

Function: svz:htons hostshor

The (svz:htons) function converts the 16 bit short integer hostshort from host byte order to network byte order.

Function: svz:ntohs netshor

The (svz:ntohs) function converts the 16 bit short integer netshort from network byte order to host byte order.

Function: svz:htonl hostlon

The (svz:htonl) function converts the 32 bit long integer hostlong from host byte order to network byte order.

Function: svz:ntohl netlon

The (svz:ntohl) function converts the 32 bit long integer netlong from network byte order to host byte order.

Function: svz:inet-aton addres

Converts the Internet host address address from the standard numbers-and-dots notation into binary data in network byte order. The (svz:inet-aton) function returns #f if the address is invalid.

Function: svz:inet-ntoa addres

The (svz:inet-ntoa) function converts the Internet host address address given in network byte order to a string in standard numbers-and-dots notation.

Function: svz:server? serve

Returns #t if the given cell server is an instance of a valid #<svz-server>, otherwise #f.

Function: svz:server:listeners serve

Returns a list of listening #<svz-socket> smobs to which the given server instance server is currently bound, or an empty list if there is no such binding yet.

Function: svz:server:clients serve

Returns a list of #<svz-socket> client smobs associated with the given server instance server in arbitrary order, or an empty list if there is no such client.

Function: svz:server:config-ref server ke

This procedure returns the configuration item specified by key of the given server instance server. You can pass this function a socket too. In this case the procedure will lookup the appropriate server instance itself. If the given string key is invalid (not defined in the configuration alist in (define-servertype!)) then it returns an empty list.

Function: svz:server:state-set! server key valu

Associates the Guile object value with the string key. The given server argument can be both, a #<svz-server> or a #<svz-socket>. Returns the previously associated object or an empty list if there was no such association. This procedure is useful for server instance state savings.

Function: svz:server:state-ref server ke

Returns the Guile object associated with the string value key which needs to be set via (svz:server:state-set!) previously. Otherwise the return value is an empty list. The given server argument must be either a valid #<svz-server> object or a #<svz-socket>.

Function: svz:server:state->hash serve

Converts the server instance's state into a Guile hash. Returns an empty list if there is no such state yet.

Function: serveez-port? name

Returns #t if the given string name corresponds with a registered port configuration, otherwise the procedure returns #f.

Function: serveez-server? name

Checks whether the given string name corresponds with an instantiated server name and returns #t if so.

Function: serveez-servertype? name

This procedure checks whether the given string name is a valid server type prefix known in Serveez and returns #t if so. Otherwise it returns #f.

Function: serveez-exceptions enabl

Controls the use of exceptions handlers for the Guile procedure calls of Guile server callbacks. If the optional argument enable set to #t exception handling is enabled and if set to #f exception handling is disabled. The procedure always returns the current value of this behaviour.

Function: serveez-nuke voi

This procedure can be used to schedule Serveez for shutdown within Guile. Serveez will shutdown all network connections and terminate after the next event loop. You should use this instead of issuing (quit).

Function: serveez-load fil

This procedure can be used as a replacement for (primitive-load) in serveez configuration files. It tries to locate the given filename file in the paths returned by (serveez-loadpath). If file cannot be loaded the procedure returns #f.

Function: serveez-loadpath arg

Make the search path for the Serveez core library accessible for Guile. Returns a list a each path as previously defined. Can override the current definition of this load path. The load path is used to tell Serveez where it can find additional server modules.

Function: serveez-interfaces arg

Make the list of local interfaces accessible for Guile. Returns the local interfaces as a list of ip addresses in dotted decimal form. If another list is given in args it should contain the new list of local interfaces.

Function: getrpc ar

Function: getrpcent

Function: getrpcbyname name

Function: getrpcbynumber number

Lookup a network rpc service by name or by service number, and return a network rpc service object. The (getrpc) procedure will take either a rpc service name or number as its first argument; if given no arguments, it behaves like (getrpcent).

Function: setrpc stayope

Function: setrpcent stayopen

Function: endrpcent

The (setrpc) procedure opens and rewinds the file ‘/etc/rpc’. If the stayopen flag is non-zero, the net data base will not be closed after each call to (getrpc). If stayopen is omitted, this is equivalent to (endrpcent). Otherwise it is equivalent to (setrpcent stayopen).

Function: portmap prognum versnum protocol por

A user interface to the portmap service, which establishes a mapping between the triple [prognum,versnum,protocol] and port on the machine's portmap service. The value of protocol is most likely IPPROTO_UDP or IPPROTO_TCP. If the user omits protocol and port, the procedure destroys all mapping between the triple [prognum,versnum,*] and ports on the machine's portmap service.

Function: portmap-list addres

This procedure returns a list of the current RPC program-to-port mappings on the host located at IP address address. When you leave this argument it defaults to the local machine's IP address. This routine can return an empty list indicating either there is no such list available or an error occurred while fetching the list.

Function: svz:coserver:dns host callback ar

This procedure enqueues the host string argument into the internal DNS coserver queue. When the coserver responds, the Guile procedure callback is run as (callback addr arg). The addr argument passed to the callback is a string representing the appropriate IP address for the given hostname host. If you omit the optional argument arg it is run as (callback addr) only. The arg argument may be necessary if you need to have the callback procedure in a certain context.

Function: svz:coserver:reverse-dns addr callback ar

This Guile procedure enqueues the given addr argument which must be an IP address in network byte order into the internal reverse DNS coserver queue. When the coserver responds, the Guile procedure callback is run as (callback host arg) where host is the hostname of the requested IP address addr. The last argument arg is optional.

Function: svz:coserver:ident sock callback ar

This procedure enqueues the given #<svz-socket> sock into the internal ident coserver queue. When the coserver responds, it runs the Guile procedure callback as (callback user arg) where user is the corresponding username for the client connection sock. The arg argument is optional.

4.2.2.5 Callback Prototypes

The Guile interface of Serveez is completely callback driven. Callbacks can be set up in the associative list passed to (define-servertype!), or by using the predefined procedures described in the previous section. Each of the callbacks is passed certain arguments and is meant to return specific values to indicate success or failure. This section describes each of these callbacks.

(global-init #<svz-servertype>)

This callback is invoked once for every type of server right after the (define-servertype!) statement. Here you can initialise resources shared between all instances of your server type. The callback is optional and can be set up in (define-servertype!). It should return zero to indicate success and non-zero to indicate failure. If the global initialiser fails, Serveez will refuse to register the server type.

(global-finalize #<svz-servertype>)

If you want to free shared resources, which were possibly allocated within the global initialiser, you can do so here. The callback is invoked when Serveez shuts down (issued by (serveez-nuke)) or the server type gets unregistered for some reason. It should return zero to signal success. The callback can be set up in (define-servertype!) and is optional.

(init #<svz-server>)

Within this callback you can initialise everything you might need for a single instance of your server. The callback is invoked for each server instance which has been created by (define-server!) and should return zero to indicate success, otherwise Serveez rejects the server instance. The callback can be set up in (define-servertype!) and is optional.

(finalize #<svz-server>)

The server instance finaliser gets its instance representation passed as argument. You need to free all resources used by this server instance which might have been allocated within the server instance initialiser or consumed while running. You can set this callback in the (define-servertype!) statement. The callback is optional and should return zero to indicate success.

(detect-proto #<svz-server> #<svz-socket>)

Connection oriented protocols like TCP and PIPE allow more than one server to be listening on the same network port. Therefore, it is necessary to be able to detect the type of client connecting to a port.

This callback takes two arguments; the first is the server instance and the second is the client socket object containing the client connection information. You can set up this callback in the (define-servertype!) statement.

Serveez may invoke this callback several times as data becomes available from the client until one of the servers recognises it. The servers can retrieve the data received so far using the (svz:sock:receive-buffer) call.

To indicate successful client detection, you need to return a non-zero value. (Note that for historical reasons, this is inconsistent with other functions which return zero on successful completion.) Once the server has indicated success, Serveez invokes any further callbacks for the connection only on that server.

If no server has recognised the client after the first 16 bytes, Serveez will close the connection. The connection will also be closed if the client has not sent enough data for a server to recognise it within 30 seconds of connecting.

If multiple servers are listening on the same network port, Serveez invokes this callback for each of them in an arbitrary order. Only one server at most should indicate successful detection.

This callback is mandatory for servers which get bound to connection oriented protocol (TCP and PIPE) port configurations by (bind-server!).

(connect-socket #<svz-server> #<svz-socket>)

If the client detection signalled success, this callback is invoked to assign the client connection to a server instance. The arguments are the same as the detection callback. In this callback you can assign all the connection specific callbacks for your server and perform some initial tasks. Basically you should specify the handle-request and/or check-request callback. This can be achieved by calling (svz:sock:handle-request) and (svz:sock:check-request). The connect-socket callback is also mandatory for connection oriented protocols and must be defined in (define-servertype!). On success you should return zero, otherwise the connection will be shutdown by Serveez.

(info-server #<svz-server>)

This callback gets invoked when requested by the builtin Control Protocol Server. The callback is optional and can be set up in (define-servertype!). The returned character string can be multiple lines separated by \r\n (but without a trailing separator). Usually you will return information about the server instance configuration and/or state.

(info-client #<svz-server> #<svz-socket>)

This callback is optional. You can set it up in the (define-servertype!) procedure. It is meant to provide socket structure specific information. (The socket structure is a client/child of the given server instance.) You need to return a single line character string without trailing newlines. The information provided can be requested by the builtin Control Protocol Server.

(notify #<svz-server>)

The server instance notifier callback will be invoked whenever there is idle time available. In practice, it is run approximately once a second. A server instance can use it to perform periodic tasks. The callback is optional and can be set up in (define-servertype!).

(reset #<svz-server>)

This callback is invoked when the Serveez process receives a SIGHUP signal which can be issued via ‘killall -HUP serveez’ from user land. If the underlying operating system does not provide SIGHUP there is no use for this callback. It provides the possibility to perform asynchronous tasks scheduled from outside Serveez. You can optionally set it up in the (define-servertype!) procedure.

(handle-request #<svz-socket> #<svz-binary> size)

This callback is invoked whenever a complete packet has been detected in the receive buffer. The packet data is passed to the callback as a #<svz-binary>. The size argument is passed for convenience and specifies the length of the packet in bytes.

The detection, and therefore the invocation, can be made in one of two ways. When Serveez can determine itself when a packet is complete, the callback will be invoked directly. Serveez can make this determination for connections with packet oriented protocols such as UDP and ICMP, or if you tell Serveez how to parse the packet using (svz:sock:boundary sock delimiter) or (svz:sock:boundary sock size) and do not specify a check-request callback.

Whenever you specify a check-request callback to determine when a packet is complete, it becomes the responsiblity of that callback to invoke handle-request itself.

Serveez recognises two different return value meanings. For connection oriented protocols (TCP and PIPE), zero indicates success and non-zero failure; on failure, Serveez will shutdown the connection. For packet oriented protocols (UDP and ICMP), a non-zero return value indicates that your server was able to process the passed packet data, otherwise (zero return value) the packet can be passed to other servers listening on the same port configuration.

This callback must be specified in (define-servertype!) for packet oriented protocols (UDP and ICMP) but is optional otherwise. You can modify the callback by calling (svz:sock:handle-request).

(check-request #<svz-socket>)

This callback is invoked whenever new data has arrived in the receive buffer. The receive buffer of the given #<svz-socket> can be obtained using (svz:sock:receive-buffer). The callback is initially not set and can be set up with (svz:sock:check-request). Its purpose is to check whether a complete request was received. If so, it should be handled (by running the handle-request callback) and removed from the receive buffer (using (svz:sock:receive-buffer-reduce)). The callback is for connection oriented protocols (TCP and PIPE) only. You should return zero to indicate success and non-zero to indicate failure. On failure Serveez shuts the connection down.

(disconnected #<svz-socket>)

The disconnected callback gets invoked whenever the socket is lost for some external reason and is going to be shutdown by Serveez. It can be set up with (svz:sock:disconnected).

(kicked #<svz-socket> reason)

This callback gets invoked whenever the socket gets closed by Serveez intentionally. It can be set up with (svz:sock:kicked). The reason argument can be either KICK_FLOOD, indicating the socket is a victim of the builtin flood protection, or KICK_QUEUE which indicates a send buffer overflow.

(idle #<svz-socket>)

The idle callback gets invoked from the periodic task scheduler, which maintains a idle-counter for each socket structure. This counter is decremented whenever Serveez becomes idle and the callback is invoked when it drops to zero. The idle callback can set its socket's idle-counter to some value with the procedure (svz:sock:idle-counter) and thus re-schedule itself for a later task. You can set up this callback with (svz:sock:idle).

(trigger-condition #<svz-socket>)

This callback is invoked once every server loop for the socket structure. If you return #f nothing else is happening. Otherwise the trigger callback will be invoked immediately. You can set up the callback using the procedure (svz:sock:trigger-condition).

(trigger #<svz-socket>)

The trigger callback is invoked when the trigger-condition returns #t. The callback can be set up with the procedure (svz:sock:trigger). Returning a non-zero value shuts the connection down. A zero return value indicates success. This callback can be used to perform connection related updates, e.g. you can ensure a certain send buffer fill.

(check-oob-request #<svz-socket> oob-byte)

This callback is invoked whenever urgent data (out-of-band) has been detected on a socket. Initially this event is ignored and the callback can be set up with the procedure (svz:sock:check-oob-request). The oob-byte argument is a number containing the received out-of-band data byte ranging from 0 to 255. If the callback returns non-zero the connection will be shutdown. A zero return value indicates success. You can use (svz:sock:send-oob) to send a single out-of-band data byte.

Please note: The urgent data is not supported by all operating systems. Also it does not work for all types of network protocols. We verified it to be working for TCP streams on GNU/Linux 2.x.x and Windows 95; let us know if/how it works on other platforms.

4.2.3 Builtin servers

All of the servers listed in Existing servers are builtin servers. The following sections describe in detail how to setup a new builtin server type. This kind of server will be part of the Serveez executable. That is why you should make it configurable in the configure script via a ‘--enable-xxxserver’ argument.

4.2.3.1 Making and configuring preparations

Serveez is configured and built via automake and autoconf. That is why you are not supposed to write your own Makefiles but simplified ‘Makefile.am’s. Automake will automatically generate dependencies and compiler/linker command lines. Here are the steps you basically need to follow:

• Change to the ‘src/’ directory in the source tree.
• Edit the ‘Makefile.am’. Add your sub directory name and library name which you are going to create.
• Now create the sub directory and change into it.
• You need to create a new ‘Makefile.am’ therein. If you want to have this file ‘configure’d you need to add a further line to the AC_OUTPUT() statement in ‘configure.in’ which is in the top level directory. You have to put at least the following into the newly created ‘Makefile.am’:

  noinst_LIBRARIES = libfoo.a libfoo_a_SOURCES = foo-proto.h foo-proto.c INCLUDES = $(SERVEEZ_CFLAGS) -I$(top_srcdir)/src CLEANFILES = *~ MAINTAINERCLEANFILES = Makefile.in

• Just have a look at all the other server directories. For more information about automake read the info pages.

4.2.3.2 Server header file ‘foo-proto.h’

This file contains at least your server's extern declaration of your server definition which must be available from the outside. The foo server implements all kinds of configuration items which can be integers, integer arrays, strings, string arrays, port configurations, booleans and hash maps. Every item of the server configuration can later be manipulated from the configuration file.

4.2.3.3 Server implementation file ‘foo-proto.c’

If you want to define default values for your configuration you have to define them somewhere and put them into the default configuration structure. This structure will be used to instantiate your server. For this example we simply called it simply foo_config.

In order to associate the configuration items in a server configuration to keywords within the configuration file you have to define an array of key-value-pairs. This is done in the foo_config_prototype field. There are several macros which make different associations. These are the SVZ_REGISTER_* macros which take three arguments. The first argument is the keyword which will occur in the configuration file, the second is the associated item in your default configuration structure and the last argument specifies if this item is defaultable or not.

4.2.3.4 Server definition

The server definition is in a way the ‘class’ of your server. Together with the default values (foo_config_prototype) it serves as a template for newly instantiated servers. The structure contains a long and a short description of your server. The short name is used as the prefix for all server instances of this specific type. The long description is used in the control protocol (See section Control Protocol Server.). The server definition also contains the callbacks your server (mandatorily) provides.

4.2.3.5 Server callbacks

There are several callback routines, which get called in order to instantiate the server and for describing the actual behaviour of your server. Here are the description of all of these callbacks. Some of them have to be implemented. Others have reasonable default values.

global initializer (optional)

This callback is executed once for every type of server. Here you can initialize data or whatever is shared between all instances of your server. For instance the HTTP server initializes its file cache here.

global finalizer (optional)

If you want to free shared resources which were possibly allocated within the global initializer you can do so here. The foo server frees its default hash previously allocated in the global initializer.

instance initializer (mandatory)

Within this routine you can initialize everything you might need for one instance of your server. The foo server does not do anything in this callback.

instance finalizer (optional)

The server instance finalizer gets its instance representation as argument. You have to free all resources used by this server instance.

protocol detection (mandatory)

Because it is possible to have more than one server listening on one network port we need to detect the type of client which is connecting to this port. The foo server checks if the first five bytes the client was sending is identifying it as a foo client. This routine is getting two arguments where the first one is a pointer to this servers instance and the second is the client socket object containing all information of the client connection. This structure is described a bit later. Be patient. For successful client detection return non-zero value.

socket connection (mandatory)

If the client detection signaled success this routine is called to assign the client connection to the server instance. The arguments are just the same as in the detection routine. In this callback you can assign all the connection specific callbacks for your server and do some initial things. The foo server sets the check_request callback to the default svz_sock_check_request which is using the packet delimiter information to find whole packets. When a client sent such a packet the handle_request callback is executed. That is why the foo server assigns the handle_request method.

client info (optional)

If this callback is given the control protocol (See section Control Protocol Server.) can give information about a specific client if requested with ‘stat id NUM’. The first argument given is the server instance and the second one the client's socket structure. You have to return a static single line character string.

server info (optional)

This function is called when listing the server instances via ‘stat all’ from the control protocol (See section Control Protocol Server.). The returned character string might be multilined separated by \r\n (no trailing separator). Usually you will return all the server configuration information.

notifier (optional)

If this callback is not NULL it is called whenever there is some time left. It gets the server instance itself as argument. Actually it gets called every second.

handle request (mandatory for UDP and ICMP servers)

The arguments to this callback are the client's socket structure, the address of the packet data and its length. When implementing a UDP or ICMP server you need to return non-zero if your server could process the packet. Thus it is possible that there are multiple UDP servers on a single port.

4.2.3.6 Make your server available

You distribute your server by editing the ‘cfgfile.c’ file in the ‘src/’ directory. There you have to include the servers header file and add the server definition by calling svz_servertype_add()

4.2.3.7 More detailed description of the callback system and structures

The client connection information is stored within the svz_socket_t object. All of the client connection specific callbacks get this object as first argument. Following is a description of the most important elements of this object.

int id

The socket id is a unique id for a client connection.

int version

This item validates this socket structure. If you pass the id and version to a coserver you can check if the delivered socket structure is the original or not within the coserver callback.

int proto

The proto flag determines a server sockets protocol type which can be PROTO_PIPE, PROTO_TCP, PROTO_UDP, PROTO_ICMP or PROTO_RAW.

int flags

The flag field of the client connection contains informations about the state of this connection. See ‘socket.h’ in the ‘src/libserveez/’ directory for more information. Basically this bitfield specifies how this object is handled by the main server loop.

int userflags

This bitfield could be used for protocol specific information. You can use it for any information.

char *boundary, int boundary_size

If you are going to write a packet oriented protocol server you can use the svz_sock_check_request method to parse packets. These two properties describe the packet delimiter.

char *send_buffer, int send_buffer_size, int send_buffer_fill

This is the outgoing data for a client connection object.

char *recv_buffer, int recv_buffer_size, int recv_buffer_fill

Within the receive buffer all incoming data for a connection object is stored. This buffer is at least used for the client detection callback.

int read_socket (svz_socket_t)

This callback gets called whenever data is available on the socket. Normally, this is set to a default function which reads all available data from the socket and feeds it to check_request, but specific sockets may need other policies.

int write_socket (svz_socket_t)

This routine is called when data is is valid in the output buffer and the socket gets available for writing. You normally leave this callback untouched. It simply writes as much data as possible to the socket and removes the data from the send buffer.

int disconnected_socket (svz_socket_t)

This gets called whenever the socket is lost for some external reason.

int connected_socket (svz_socket_t)

If some piece of code tries to connect to another host via svz_tcp_connect this connection might be established some time later. This callback gets called when the socket is finally connected.

int kicked_socket (svz_socket_t, int)

We call this whenever the socket gets closed by us. The second argument specifies a reason.

int check_request (svz_socket_t)

This gets called whenever data was read from the socket. Its purpose is to check whether a complete request was read, and if it was, it should be handled and removed from the input buffer.

int handle_request (svz_socket_t, char *, int)

This gets called when the check_request got a valid packet. The request arguments contains the actual packet and the second argument is the length of this packet including the packet delimiter.

int idle_func (svz_socket_t)

This callback gets called from the periodic task scheduler. Whenever idle_counter (see below) is non-zero, it is decremented and idle_func gets called when it drops to zero. idle_func can reset idle_counter to some value and thus can re-schedule itself for a later task.

int idle_counter

Counter for calls to idle_func.

void *data

Miscellaneous field. Listener keeps array of server instances here. This array is NULL terminated. Some servers store server specific information here.

void *cfg

When the final protocol detection in svz_sock_detect_proto has been done cfg should contain a pointer to the actual configuration hash map taken from the server instance object.

4.2.3.8 Using coservers

Coservers are designed to complete blocking tasks. Each coserver runs in its own thread/process. There are several coservers implemented: the dns, reverse dns and ident coserver. You need to implement the callback which gets called when a coserver completed its task. This routine must be a svz_coserver_handle_result_t. The first argument is the actual coserver result which might be NULL if the request could not be fulfilled and the latter two arguments are the arguments you specified yourself when issuing the request. To invoke a coserver you use one of the svz_coserver_* macros. The foo server uses the reverse dns coserver to identify the host name of the remote client.

4.3 Some words about server configuration

If you define a server you basically pass an instance name and a list of items to the (define-server!) function. Each item has a name and a value. A value has a type. We provide several types: integers (numbers), integer arrays, strings, string arrays, booleans (yes/no-values), hashes (associations) and port configurations.

The following table shows how each kind of value is set up in the configuration file. item is the name of the item to be configured.

Integer

Example: (item . 42)

Integer array

Example: (item . (0 1 2 3))

String

Example: (item . "a character string")

String array

Example: (item . ("abc" "cba" "bca" "acb"))

Boolean

A normal boolean in guile is represented by #t or #f. But the configuration file parser additional understand some bare words and numbers.
Example: (item . #f)

Hash

Hash maps associate keys with values. Both must be character strings.
Example: (item . (key1 . "value1") (key2 . "value2"))

Port configuration

See section Define ports, for more information on this. When configuring a port configuration you need to define it via (define-port!) previously and put its symbolic name into the configuration.
Example: (item . foo-tcp-port)

The next chapter describes the servers currently implemented using Serveez. The configuration items used by each of them are described in the following format:

NameOfTheItem (Type, default: DefaultValue, Comments)

Description of the configuration item named ‘NameOfTheItem’ (case sensitive). ‘Type’ can be either ‘integer’, ‘integer array’, ‘string’, ‘string array’, ‘boolean’, ‘hash’ or ‘port configuration’. The ‘Comments’ is an optional text.

The example configuration file ‘data/serveez.cfg’ contains an example definition of each server already implemented. You can copy and modify the example for an easy start.

4.4 Existing servers

4.4.1 HTTP Server

4.4.1.1 General description

The integrated HTTP server was originally meant to be a simple but fast document server. But now it can even execute CGI scripts. The GET, HEAD and POST methods are fully functional. Additionally Serveez produces directory listings when no standard document file (e.g. ‘index.html’) has been found at the requested document node (directory). Furthermore it implements a file cache for speeding up repetitive HTTP request.

In comparison to other web server projects like Apache and Roxen this web server is really fast. Comparative benchmarks will follow. The benchmark system is a 233 MHz Mobile Pentium MMX. Both the server and the client (http_load - multiprocessing http test client) ran on the same computer.

Small files

The small-file test load consists of 1000 files, each 1KB long, requested randomly.

concurrent fetches 1 10 50 100 200 500 1000 hits/second 501 520 481 475 420 390 295

CGI

The CGI test load consists of a trivial "hello world" C program. I noticed GNU/Linux (2.2.17 in this case, probably others too) to throw "Resource temporarily unavailable" errors when fork()ing very fast. This limits the test to about 200 concurrent fetches on the test system.

Large files

The large-file test load consists of 100 files, each 1MB long, requested randomly. Also, each connection is throttled to simulate a 33.6Kbps modem. Note that 1000 33.6Kbps connections is 3/4 of a T3. There was no problem to get 1000+ concurrent fetches.

4.4.1.2 Configuration

The following options can be set from the configuration file.

indexfile (string, default: index.html)

The indexfile parameter is the default file served by the HTTP server when the user does not specify a file but a document node (e.g. http://www.textsure.net/).

docs (string, default: ../show)

The docs parameter is the document root where the server finds its web documents.

userdir (string, default: public_html)

Each ‘~user’ request gets converted into the given users home directory. The string will be appended to this directory. Its default value is ‘public_html’.

cgi-url (string, default: /cgi-bin)

This parameter is the first part of the URL the HTTP server identifies a CGI request. For instance if you specify here ‘/cgi-bin’ and the user requests http://www.textsure.net/cgi-bin/test.pl then the HTTP server tries to execute the program ‘test.pl’ within the cgi-dir (see below) and pipes its output to the user.

cgi-dir (string, default: ./cgibin)

The cgi-dir is the CGI document root (on the server).

cgi-application (hash, default: empty)

Within the MinGW32 and native port you can use this hash to associate certain file suffices with applications on your computer (e.g. "pl" with "perl"). This is necessary because there is no possibility to check whether a file is executable on Win32.

cache-size (integer, default: 200 kb)

This specifies the size of the document cache in bytes for each cache entry.

cache-entries (integer, default: 64)

This parameter specifies the maximum number of HTTP file cache entries (files). When you instantiate more than one HTTP server the biggest value wins. The HTTP file cache is shared by all HTTP servers.
Please note: If your harddrive/filesystem combination proves to be faster than the HTTP file cache you should disable it by setting both cache-size and cache-entries to zero.

timeout (integer, default: 15)

The timeout value is the amount of time in seconds after which a keep-alive connection (this is a HTTP/1.1 feature) will be closed when it has been idle.

keepalive (integer, default: 10)

On one keep-alive connection can be served the number of keepalive documents at all. Then the connection will be closed. Both this and the timeout value are just to be on the safe side. They protect against idle and high traffic connections.

default-type (string, default: text/plain)

The default-type is the default content type the HTTP server assumes if it can not identify a served file by the types hash and the type-file (see below).

type-file (string, default: /etc/mime.types)

This should be a file like the ‘/etc/mime.types’ on Unix systems. It associates file suffices with MIME types.

types (hash, default: empty)

If you want to specify special content types do it here. This parameter is a hash map associating file suffices with HTTP content types (MIME types).

admin (string, default: root@localhost)

Your address, where problems with the server should be e-mailed. This address appears on some server-generated pages, such as error documents.

host (string, default: localhost)

This is the native host name of your web server. Sometimes the server has to send back its own name to the client. It will use this value. Be aware that you cannot invent such a name.

nslookup (boolean, default: false)

If this is true the HTTP server invokes a reverse DNS lookup for each client connection in order to replace the remote ip address with the remote host name in the access logfile.

ident (boolean, default: false)

If this is true the HTTP server processes identd requests for each client connection for logging purposes.

logfile (string, default: http-access.log)

The location of the access logfile. For each HTTP request a line gets appended to this file.

logformat (string, default: CLF)

The format of the access logfile. There are special placeholders for different kinds of logging information. The default log format is the Common Log Format (CLF). It contains a separate line for each request. A line is composed of several tokens separated by spaces.

CLF = host ident authuser date request status bytes

If a token does not have a value then it is represented by a hyphen (-). The meanings and values of these tokens are as follows:

%h (host)

The fully-qualified domain name of the client, or its IP number if the name is not available.

%i (ident)

This is the identity information reported by the client. Not active, so we will see a hyphen (-).

%u (authuser)

If the request was for an password protected document, then this is the userid used in the request.

%t (date)

The date and time of the request, in the following format:

date = [day/month/year:hour:minute:second zone] day = 2*digit month = 3*letter year = 4*digit hour = 2*digit minute = 2*digit second = 2*digit zone = (`+' | `-') 4*digit

%R (request)

The request line from the client, enclosed in double quotes (").

%r (referrer)

Which document referred to this document.

%a (agent)

What kind of web browser did the remote client use.

%c (status)

The three digit status code returned to the client.

%l (bytes)

The number of bytes in the object returned to the client, not including any headers.

4.4.2 IRC Server

4.4.2.1 General description

Internet Relay Chat. The mother of all chat systems. The integrated IRC server is intended to be compatible with the EFNet. There are no good possibilities to test this in real life, so it is still under heavy construction. But it can be used as a standalone server anyway.

IRC itself is a teleconferencing system, which (through the use of the client-server model) is well-suited for running on many machines in a distributed fashion. A typical setup involves a single process (the server) forming a central point for clients (or other servers) to connect to, performing the required message delivery/multiplexing and other functions.

The server forms the backbone of IRC, providing a point for clients and servers to connect to. Clients connect to talk to each other. Servers connect to build up a network of servers. IRC server connections have to build up a spanning tree. Loops are not allowed. Each server acts as a central node for the rest of the network it sees.

4.4.2.2 Configuration

The following table shows the configuration keys provided. Most of the configuration items are similar to those of an Hybrid IRC server. They seem archaic at first sight but IRC operators are used to it. Refer to the Hybrid documentation for further information. It can be found on the EFNet web page.

MOTD-file (string, default: ../data/irc-MOTD.txt)

When a user initially joins it will get this file's content as the message of the day comment. When changing on disk the server will notice that and reload the file automatically.

INFO-file (string, default: no file)

The INFO-files content gets displayed when the user issues the /INFO command.

tsdelta (integer, default: 0)

This value is the timestamp delta value to UTC (Coordinated Universal Time) in seconds.

channels-per-user (integer, default: 10)

Configures the maximum number of channels a single local user can join.

admininfo (string, no default)

Some administrative information delivered on the /ADMIN command.

M-line (string, no default, mandatory)

The TCP level configuration of this IRC server. The server info field is sometimes given to the client for informational use. The server will croak about if the settings do not correspond with the actual bindings. The format of this line is:

":" virtual hostname ":" optional bind address (real hostname) ":" server info: "World's best IRC server" ":" port

A-line (string, no default, mandatory)

The administrative info, printed by the /ADMIN command.

":" administrative info (department, university) ":" the server's geographical location ":" email address for a person responsible for the IRC server

Y-lines (string array, no default, suggested)

The connection classes. They are used in other parameters (e.g. I-lines). A Y-line describes a group of connections. You usually have at least two Y-lines: One for server connections and one for client connections. Format of each line is:

":" class number (higher numbers refer to a higher priority) ":" ping frequency (in seconds) ":" connect frequency in seconds for servers, 0 for client classes ":" maximum number of links in this class ":" send queue size

I-lines (string array, no default, mandatory)

Authorization of clients, wildcards permitted, a valid client is matched user@ip OR user@host.

":" user@ip, you can specify ‘NOMATCH’ here to force matching user@host ":" password (optional) ":" user@host ":" password (optional) ":" connection class number (YLine)

O-lines (string array, no default, optional)

Authorize operator, wildcards allowed.

":" user@host, user@ forces checking ident ":" password ":" nick

o-lines (string array, no default, optional)

Authorize local operator.

":" user@host, user@ forces checking ident ":" password ":" nick

C-lines (string array, no default, networked)

List of servers to connect to. Note: C and N lines can also use the user@ combination in order to check specific users (ident) starting servers. C and N lines are usually given in pairs.

":" host name ":" password ":" server name (virtual) ":" port (if not given we will not connect) ":" connection class number (YLine)

N-lines (string array, no default, networked)

Servers which may connect.

":" host name ":" password ":" server name (virtual host name) ":" password ":" how many components of your own server's name to strip off the front and be replaced with a ‘*’. ":" connection class number (YLine)

K-lines (string array, no default, optional)

Kill user, wildcards allowed.

":" host ":" time of day ":" user

4.4.3 Control Protocol Server

4.4.3.1 General description

Serveez implements something like a telnet protocol for administrative purposes. You just need to start a telnet session like:

$ telnet www.textsure.net 42420

After pressing <RET> you will be asked for a password which you might setup passing Serveez the -P argument. See section Using Serveez. The next section describes the interactive commands available.

4.4.3.2 Using the Control Protocol

‘help’

This command will give you a very short help screen of all available commands.

‘quit’

This command closes the connection to Serveez.

‘restart ident’

Restarts the internal ident coserver. This is useful if you just want to start a new one if the old one died or is otherwise unusable.

‘restart dns’

Restarts the internal dns lookup server.

‘restart reverse dns’

Restarts the internal reverse dns lookup server.

‘killall’

This might be useful if Serveez seems to be unstable but you do not want to restart it. With ‘killall’ you disconnect all client network connections except the control protocol connections.

‘kill id NUM’

Disconnects a specific connection identified by its ID. These IDs will be stated when you type ‘stat con’ (see below).

‘stat’

General statistics about Serveez. This will show you some useful information about the computer Serveez is running on and about the state of Serveez in general.

‘stat coserver’

Statistics about all running coserver instances.

‘stat SERVER’

This command is for selecting certain server instances to be listed. SERVER is one of server names you specified in the configuration file.

‘stat id NUM’

Show statistics about a specific connection. This will give you all available information about every connection you specified. See section Writing servers, for more information about how to provide these information.

‘stat con’

Connection statistics. This will give a list of all socket structures within Serveez. If you want more detailed information about specific connections, coservers or servers you need to request these information with ‘stat id NUM’ or ‘stat all’.

‘stat all’

Server and coserver instance statistics. This command lists all the information about instantiated servers and coservers. See section Writing servers, for more information about how to provide these information.

‘stat cache’

HTTP cache statistics. This command produces an output something like the following where ‘File’ is the short name of the cache entry, ‘Size’ the cache size, ‘Usage’ the amount of connections currently using this entry, ‘Hits’ the amount of cache hits, ‘Recent’ the cache strategy flag (newer entries have larger numbers) and ‘Ready’ is the current state of the cache entry.

File Size Usage Hits Recent Ready zlib-1.1.3-20000531.zip 45393 0 0 1 Yes texinfo.tex 200531 0 0 2 Yes shayne.txt 2534 0 1 1 Yes Total : 248458 byte in 3 cache entries

‘kill cache’

Reinitialize the HTTP file cache. Flushes all files from the cache.

4.4.3.3 Configuration

There is nothing to be configured yet.

4.4.4 Foo Server

4.4.4.1 General description

The Foo Server is a simple example on how to write Internet protocol servers with Serveez. See section Writing servers.

4.4.4.2 Configuration

There are all kinds of configuration items. They are used to explain the implementation of servers. A complete list will follow.

port (port configuration, default: tcp, 42421, *)

Sets up the TCP port and local address.

bar (integer, no default)

Some integer value. Printed as server information.

reply (string, default: Default reply)

Some string. Printed as server information.

messages (string array, default: ...)

Some string array which is actually a list of strings. Also printed as server information.

ports (integer array, default: 1, 2, 3, 4)

Some array of integer numbers. Printed as server information.

assoc (hash, default, default: ...)

An hash map associating keys with values. Printed as server information.

truth (boolean, default: true)

Some boolean value. Printed as server information.

4.4.5 SNTP Server

4.4.5.1 General

The SNTP server can be queried with the ‘netdate’ command. It is used to synchronize time and dates between Internet hosts. The protocol is described in the ARPA Internet RFC 868. Thus it is not really an SNTP server as described by RFC 2030 (Simple Network Time Protocol (SNTP) Version 4 for IPv4, IPv6 and OSI). It is rather an excellent example on how to implement a UDP server in Serveez.

This protocol provides a site-independent, machine readable date and time. The Time service sends back time in seconds since midnight on January first 1900.

One motivation arises from the fact that not all systems have a date/time clock, and all are subject to occasional human or machine error. The use of time-servers makes it possible to quickly confirm or correct a system's idea of the time, by making a brief poll of several independent sites on the network.

4.4.5.2 Configuration

The configuration of this server does not require any item.

4.4.6 Gnutella Spider

4.4.6.1 What is it ?

The Gnutella net is a peer-to-peer network which is based on client programs only. There are no servers. The network itself is formed by client connections only. Generally the Gnutella network is for sharing files of any kind.

This Gnutella spider is for seeking the needle in the haystack. Once connected to the network it regularly tries to find certain files in there. It keeps track of all connected clients and tries to reconnect them if the current connections are lost.

Gnutella, however has nothing to do with the GNU project. The original client is just a free (as in free beer) piece of software. With Serveez you have a free (as in freedom) way to use it. Have a look at the Gnutella page for further information.

4.4.6.2 Configuration

The Gnutella spider knows the following configurations items.

net-url (string, default: gnutella-net)

If you want to see the host catcher list of this Gnutella spider you can connect to this port with any WWW browser at http://host:port/net-url. The host:port combinations depend on the bindings.

hosts (string array, no default)

This is the start of the haystack, the initial host list of the clients the spider tries to connect to. Each list item should be of the format ip:port (e.g. ‘146.145.85.34:6346’). You can also pass Internet host names. If the port information is left blank it defaults to 6346. If you need some entry point for the Gnutella network have a look at http://www.gnutellahosts.com/ or http://www.gnutellanet.com/.

search (string array, default: Puppe3000, Meret Becker)

This is the needle. Each search line is either a set of space delimited tokens where every token must match. Or a kind of wildcard expression including ‘?’ and ‘*’. Search lines are always matched case insensitive.

search-limit (integer, default: 30)

This limits how many results the Gnutella spider returns to other people searching your files. This is for protection against "*" search requests.

max-ttl (integer, default: 5)

Every Gnutella packet has got a TTL. This is the maximum TTL allowed for outgoing packets. When a packet comes in it gets its TTL value decremented and is forwarded to it destination. If however an incoming packet has a TTL larger than max-ttl the ttl value is set to max-ttl. This is necessary since most people use far too large TTL values.

ttl (integer, default: 5)

When creating a new Gnutella packet we use this as TTL. Please use a sane value. This ttl needs not to be as large as it is for IP packets. A value below 10 is more than enough. Have a look at the Gnutella page for a calculation of a 'sane value'.

download-path (string, default: /tmp)

This is where the spider saves needles in.

share-path (string, default: /tmp)

Here are all the files we share with others. The Gnutella spider will recurse into directories. So be careful with this option.

max-downloads (integer, default: 4)

Maximum number of concurrent downloads from the network.

max-uploads (integer, default: 4)

Maximum number of concurrent uploads to the network.

connection-speed (integer, default: 28)

This is what we send as our connection speed in KBit/s. We also use this value to throttle down the network transfer rate for file uploads.

min-speed (integer, default: 28)

Search for needles on hosts with a minimum speed. Set it to 0 if you do not care about that. This value is in KBit/s, too.

file-extensions (string array, default: empty list)

If we get replies on search queries we check if the file extension of this reply matches one of these extensions. Useful extensions are ‘mp3’ and ‘mpg’.

connections (integer, default: 4)

This is the number of concurrent connections the Gnutella spider tries to keep up to the network. The IP addresses and the port information is taken from the host catcher hash.

force-ip (string, default: not set)

You can force the Gnutella spider to send outgoing replies with this IP as host information. Must be in dotted decimals notation. This is useful if you are behind a masquerading gateway. You need to install some kind of port forwarder on the gateway so other people can reach you from the outside. Serveez is a good port forwarder.

force-port (integer, default: not set)

Force the Gnutella spider to send outgoing replies with the force-port as port information. See above for more information.

disable (boolean, default: false)

With this configuration option you can disable the bindings for a Gnutella server instance. This means that no remote client can connect without being told so (e.g. by push requests).

4.4.7 Tunnel Server

4.4.7.1 General description

The Tunnel server is for mapping one port configuration to another. So we should rather speak of a port forwarder. Two port forwarders can form a tunnel. Generally this means that you can setup Serveez to accept network or pipe connections in order to pass all transfer data on this line to another port configuration. This can be useful to work around gateways and firewalls. When instantiating an ICMP source or destination you must ensure root privileges for the application. On Windows NT and Windows 2000 you need to be either logged in as Administrator or have set the registry key HKLM\System\CurrentControlSet\Services\Afd\Parameters\DisableRawSecurity to 1 (DWORD). One of the given examples in ‘serveez.cfg’ shows how you can setup a tunnel server for forwarding a pipe connection. Please keep in mind when forwarding a TCP or pipe connection over ICMP or UDP you loose reliability since the latter two are packet oriented rather than connection oriented. We are not willing to implement our own TCP stack to work on ICMP/UDP directly.

Forwarding between the same types of connection is always possible. When forwarding to an ICMP tunnel we use a special protocol which we will outline in the following section.

4.4.7.2 Extended ICMP protocol specification

Since ICMP (Internet Control Message Protocol) does have a fixed packet format we had to extend it in order to use it for our own purposes. The protocol field of the IP header contains a binary ‘1’ which is the identifier for ICMP (e.g. ‘6’ identifies TCP). When creating an ICMP socket the IP header is always generated by the kernel. This is the main difference to raw sockets where the IP header must be generated at userspace level.

When receiving an ICMP packet it also contains the IP header. When sending an ICMP packet you must not prepend this IP header. The kernel will do this itself. The IP header always follows the actual ICMP header followed by the ICMP packet data. Since this section does not cover raw sockets we leave the IP header structure out here.

The modified ICMP message format is as:

Each of these fields can be modified and processed by Serveez and do not get touched by the kernel at all. The ICMP socket implementation of Serveez differentiates two types of sockets: listening and connected ICMP sockets. This is non-standard because it actually makes no sense since there is no difference for the kernel. The introduction of these semantics allow Serveez to forward data between connection-oriented (TCP and named pipes) and packet-oriented (UDP and ICMP) protocols.

Message type

Valid message types are for instance ‘8’ for an echo message and ‘0’ for its echo reply. These two messages are used for the systems builtin ping services. Serveez uses its own message type identifier which is ‘42’ (ICMP_SERVEEZ) by default.

Message type sub code

Serveez also defines its own message type sub codes described in the following table.

Sub code	Constant identifier	Description
0	ICMP_SERVEEZ_DATA	packet contains data
1	ICMP_SERVEEZ_REQ	unused
2	ICMP_SERVEEZ_ACK	unused
3	ICMP_SERVEEZ_CLOSE	disconnection message
4	ICMP_SERVEEZ_CONNECT	connect message

Checksum

The checksum field of the ICMP header is used to check the ICMP headers and the payloads (packet data) validity. We are using the standard Internet Checksum algorithm described in RFC 1071. If the check failed we drop the packet.

Senders unique identifier

The senders identifier field is used to determine if a received packet has been sent by the sender itself and should therefore be dropped. This happens because each ICMP socket setup for receiving gets all sent ICMP packets system wide. Thus Serveez will even be notified if the kernel creates some echo reply or destination unreachable message due to a request completely outside the scope of Serveez.

Sequence number

Each connected ICMP socket increments its sequence number when sending a packet. Thus a connection message type packet of such a socket always has a zero sequence number. This field could (but is not yet) also be used to reorder ICMP packets or to detect missing packets.

Port number

The port field of the modified packet format helps Serveez to establish connected ICMP sockets. A simple packet filter detects if a received packet is kind of reply to a sockets sent packets by comparing this port number. The packet is dropped if the comparison fails and it is not a listening socket.

Except the data message type subcode all ICMP packets created and sent by Serveez have a zero payload. The connect message subcode identifies a new connection and the disconnection message subcode its shutdown without actually transmitting data. These two subcodes emulate a TCP connections connect(), accept() and shutdown() system call.

4.4.7.3 Configuration

This might be the most easiest configuration to setup. You essentially need to define the source port configuration and the target port configuration. The ‘serveez.cfg’ in the ‘data/’ directory shows two example configurations how to tunnel TCP connections over UDP and ICMP. The UDP tunnel accesses the standard HTTP port 80 and the ICMP tunnel accesses the standard Telnet port 23.

source (port configuration, no default)

The source port configuration. This is usually the same you bind the server to.

target (port configuration, no default)

The target port configuration.

4.4.8 Chat Server

4.4.8.1 General description

The whole Serveez package was originally written by Martin Grabmueller as a fast C backend for a chat system (now called textSure). This can be found at http://www.textsure.net/. The server implements two kinds of protocols. One is for the master server which does all the administrative stuff. The second is for the chat clients (applets or bots). Basically the chat server is a simple multicast unit.

4.4.8.2 Specification of the Slave/Master server protocol of aWCS

Master server

The first client that connects and sends an aWCS initialize message (see below) becomes the master server. When the master server disconnects, all clients get kicked.

Other clients

All other clients have to send the aWCS client initialize message, which is the string "aWCS\0" (note the final NUL character, which is part of the message). Messages received from those clients are simply sent to the master server with the client number of the sender prefixed (see `Messages from clients').

Message format

A request consists of a sequence of characters terminated with the NUL character '\0'.

Messages from clients

All messages from clients are routed to the master server, but modified in such a way that some digits and one space precede the original message. These digits are the (decimal) client's id.

Messages from the master server

Messages that come from the master server must have the following format:

The first character is one of the following and has a special meaning:

Character	Meaning
0	Send message to all clients.
1	Send message to the clients denoted in the message.
2	Report status to the master server.
3	Kick the clients denoted in the message.
4	Turn off flood protection for the clients in the message.
5	Turn on flood protection for the clients in the message.
6	Initialization message.

Those special characters are followed by a single space.

Messages which require additional parameters (multicast for example) carry these parameters after the space in a comma separated list of client ids in the format mentioned above. The client id list is terminated by the \0 character terminating the request (if no parameters follow) or by a single space (if parameters follow). Note that no spaces are allowed in comma separated lists.

Examples: 0 Hallo Welt\0 - Send "Hallo Welt\0" to all clients except the master server. 1 1,4,6 Gruezi!\0 - Send "Gruezi\0" to the clients 1, 4 and 6. 2 \0 - Send status data back to the master server. 3 3,4\0 - Kicks clients 3 and 4. 4 4,16\0 - Turn off flood protection for clients 4 and 16. 5 1,5\0 - Turn on flood protection for clients 1 and 5. 6 \0 - Initialize data structures and accept the connection as the master server.

Status messages for the master server

The low level server posts messages to the master server on several occasions, for example when clients connect, disconnect etc.

Status messages have the same format as other client messages for the master server. The client number in the message is the id of the master server itself, followed by a space, the status message type and additional parameters.

Type	Description	Parameter
0	Client has connected.	Client id, client IP number:port
1	Client has disconnected.	Client id, error code
2	Client got kicked.	Client id, kick reason
3	Low level server is alive.	42
4	Notify time elapsed.	42

Error code	Description
0	Success
1	Client closed connection

Kick reason	Description
0	Flooding
1	Output buffer overflow

4.4.8.3 Configuration

There is nothing to be configured yet. You should notice that it is possible to bind this server to both named pipes and TCP ports.

4.4.9 Fake Ident Server

4.4.9.1 General description

Most systems run the 'ident protocol' on port 113. Internet hosts can connect to that port and find out what user is having a connection to the host. For example a webserver can query your username when you fetch a file (e.g. Serveez' internal ident-coserver can do that). Most IRC servers protect themselves by allowing only users that have a valid ident response. Therefore mIRC (for windoze) has a built in ident server. This fake ident server can be used to 'fake' a response. This is useful when you connect through a masquerading gateway and the gateway cannot handle ident requests correctly. (Or, of course, you are using windoze, need an ident response and do not have mIRC at hand.)

This server has two modes of operation. In one mode all requests get "ERROR : NO-USER" as response. This is a valid but not very helpful response. The other mode makes the server send a valid and useful response. It contains a system type and a username. The system type is usually 'UNIX'. Others are valid but never used (at least i have never seen something else).

4.4.9.2 Configuration

This server is easy to configure.

systemtype (string, default: UNIX)

The system type to respond. The username field of the response has other meanings depending on this field, so do not make things up here. Read the RFC to learn more.

username (string, default: <NULL>)

If no username is set (which means this field does not appear in the configuration file) the server runs in the flag-all-requests-as-error mode. Use your favourite nickname here.

4.4.10 Passthrough Server

4.4.10.1 General description

The program passthrough server provides basic inetd functionality. Basically it can accept connections and pass this connection to the standard input (stdin) and standard output (stdout) handles of programs. Depending on the platform (operating system) the user is able to configure different methods how this can be achieved.

4.4.10.2 Configuration

This server has different types of configuration options specifying its behaviour. Some of them are mandatory and some are optional. The very least to configure is the program to be started when a new connection is made.

binary (string, no default)

This parameter specifies the program to execute when a new connection has been accepted. The parameter is mandatory and must be a fully qualified file name (including path).

directory (string, no default)

This will be the working directory of the executed program. If you omit this parameter the server uses the current directory (the directory is not changed).

user (string, no default)

If you omit this parameter no user or group will be set for the started program. Otherwise you need to specify this information in the format ‘user[.group]’. If the group is omitted the user's primary group will be used.

argv (string array, no default)

This list of character strings is going to be the program's argument list (command line). If the first list item (which is argv[0] and the program's name) is left blank it defaults to the name specified in the binary parameter.

do-fork (boolean, default: true)

This flag specifies the method used to pass the connection to the program. If it is true the server uses the Unix'ish fork() and exec() method. Otherwise it will pass the data through a unnamed pair of sockets [ or two pairs of anonymous pipes ].

single-threaded (boolean, default: true)

This parameter applies to servers bound to UDP and ICMP port configurations only. For programs which process all incoming packets and eventually time out, the program is said to be ‘single-threaded’ and should use a true value here. If a program gets a packet and can receive further packets, it is said to be a ‘multi-threaded’ program, and should use a false value.

thread-frequency (integer, default: 40)

The optional thread-frequency parameter specifies the maximum number of program instances that may be spawned from the server within an interval of 60 seconds.

4.4.11 Mandel Server

[ Example Mandelbrot picture. ]

4.4.11.1 General description

The distributed Mandelbrot server is an Internet server completely written in Guile with the help of the API provided by the underlying Serveez application. The reader will not see any occurrence of the networking API of Guile.

It implements a protocol called ‘dnc’. ‘dnc’ - short for "Distributed Number Cruncher". The Mandelbrot server manages the computation of a graphic visualization of the Mandelbrot set fractal. Each client can connect to the server and ask for something to calculate and is meant to send its result back to the server. Finally the server produces a bitmap in the XPM format and uses a specified viewer application to bring it onto your screen.

4.4.11.2 Configuration

The server can be setup to manage the calculation of the Mandelbrot set at various locations (rectangular region in the complex plane), in a specific pixel resolution and colour depth. Moreover you can define the name of the final output file and the viewer application the output file is displayed with.

start (string, default: -2.0-1.5i)

Specifies the upper left corner of the final bitmap in the complex plane.

end (string, default: +1.1+1.5i)

Specifies the lower right corner of the final bitmap in the complex plane.

x-res (integer, default: 320)

The real part pixel resolution.

y-res (integer, default: 240)

The imaginary part pixel resolution.

colors (integer, default: 256)

Number of maximum colours used in the bitmap. Also determines the maximum iteration depth.

outfile (string, default: mandel.xpm)

When the Mandel server has managed to calculate the whole bitmap it produces an output file in the XPM format. You can specify the name and location of this output file.

viewer (string, default: xv)

Here you can setup your favourite bitmap viewer application. It should be able to parse and display the XPM format.

This document was generated on September, 7 2009 using texi2html 1.78.