14. THOR and Merline Toolkit: Servers

Back to Table of Contents

14.1 Introduction: THOR and Merlin Objects

The THOR and Merlin Toolkits provide access to the THOR and Merlin "views" of databases, respectively. Although Merlin and THOR present very different views of the data (THOR's retrieval/storage versus Merlin's exploratory data analysis), the two systems share many features, and operate on the same databases. Because of this, many features of the THOR and Merlin Toolkit are presented together.

An old adage might be paraphrased here: "An example is worth a thousand words." Many tutorial examples of Thor and Merlin Toolkit usage can be found in the "contrib" directory: 

     $DY_ROOT/contrib/src/thor/
     $DY_ROOT/contrib/src/merlin/
We strongly encourage you to study these examples before attempting to write Thor and/or Merlin Toolkit programs.

THOR and Merlin objects have the same basic characteristics as the objects introduced in earlier chapters. They are opaque, respond to most of the standard polymorphic functions, are subject to revocation, and so forth. However, THOR and Merlin objects have a more concrete existence since they represent real data in a database. Thus, unlike a molecule object which disappears forever when deallocated, some THOR and Merlin objects represent things that "live on" even after the Toolkit deallocates them.

The following illustrates the object hierarchies of THOR and Merlin, showing which objects are common to both:

Thor and Merlin Objects

Although this drawing shows the Server and Database objects as common to both Toolkits, there are actually different object types for each; THOR has objects of TYP_SERVER and TYP_DATABASE, while Merlin has objects of TYP_MERSERVER and TYP_POOL. Since polymorphism makes these objects behave nearly identically, we present them as a single concept. As you get into the details of their use, the differences will become apparent.

Note: By historical accident THOR server objects get the simple designation TYP_SERVER, while Merlin server objects are the more specific TYP_MERSERVER.

Below is a brief outline of each object type.

THOR and Merlin Toolkits:

  • A server object represents a connection to a server. The server object "knows" about the server's network address, the interprocess- communication mechanisms, which databases the are open on the server, and so forth. Server objects are discussed in detail in this chapter.

  • A database object represents an open database. In THOR this is the open database files on disk; in Merlin it is the in-memory image of the database's data.

  • A datatype object represents the definition of a datatype. Datatypes define the meaning of data in a database; each dataitem in THOR is associated with its datatype (shown with a dotted line, above).

  • A fieldtype object is a constituent part of a datatype object, and defines the meaning of one datafield within a dataitem; each datafield object in THOR and each column object in Merlin is associated with its fieldtype (shown with dotted lines, above).

THOR Toolkit:

  • A datatree object contains a THOR Datatree (TDT) from the database.

  • A dataitem object represents one dataitem from a TDT. Each dataitem object has a datatype object associated with it (shown with a dotted line, above) that defines its meaning, and has one or more child fieldtype objects for its datafields.

  • A datafield object contains one datum: a string of characters. It represents the basic unit of information in the THOR system. This string of characters might be interpreted by an application as a number, text, or some other way (including binary data).

Merlin Toolkit:

  • A hitlist object represents an ordered subset of the datatrees in a THOR database.

  • A column object represents a "vertical slice" through the database: one datum of a particular fieldtype from each TDT in the database. For example, a column of $NAM would contain one name for each database entry.

The above brief descriptions are just a quick sketch of the object types available in the THOR and Merlin Toolkits. A detailed description of each is given in this and subsequent chapters.

14.2 Connecting to a Server

Most of the material in this chapter requires an acquaintance with the material in the Daylight Theory Manual and Daylight System Administrator's Guide. You should read these for an introduction to the concepts of server/client systems, security, search paths, file ownership, and network configuration.

THOR and Merlin are both client/server systems. The Merlin and THOR Toolkits provide a programmer's library for the client side of the client/server system -- that is, the Toolkits provide the mechanisms you need to connect to and communicate with the server side of the system. A server object represents a connection to a THOR or Merlin server. The object "knows" about the server's address, the interprocess-communication mechanisms, which databases are open on the server, and so forth.

When a client connects to a THOR or Merlin server, a new object of type TYP_SERVER2 or TYP_MERSERVER is created. The server-object is used in subsequent calls to create, open, and close databases, and other database operations, and in calls to modify security, search paths, and related administrative tasks.

A server object becomes the parent object of any databases ("pools" in Merlin) that are opened via the server (see the chapter on POLYMORPHIC FUNCTIONS for a discussion of parent objects and base objects). This implies that all such database objects can only exist while the server object exists (the actual databases, of course, exist until you erase them; here we are talking about the Toolkit objects that represent databases). If the connection to the server is broken (via dt_dealloc(server)), all databases on that server will be closed, and their child objects will be deallocated. In the case of accidental disconnection, as when a THOR or Merlin client program "crashes", the server itself closes the databases to insure no data are lost and to free allocated resources (memory, open files); if a server crashes, the client program will report "lost connection" for any subsequent database transactions.

The following functions connect to Merlin and Thor servers, respectively: 

dt_mer_server(string host      /* server's hostname    */
              string service,   /* service name */
              string userid,    /* user's login name    */
              string passwd,    /* user's password      */
              integer isnew)    /* existing or new?     */
                  ==> Handle server

dt_thor_server(string host    /* server's hostname    */
               string service,  /* service name */
               string userid,   /* user's login name    */
               string passwd,   /* user's password      */
               integer isnew)   /* existing or new?     */
                  ==> Handle server
Connects to a server on the specified host using the specified service name; returns an object of type TYP_SERVER (THOR) or TYP_MERSERVER (Merlin). If the connection already exists, returns the handle that was originally allocated for the server (see the parameter isnew, below).

The parameters host and service specify the machine on which the Thor or Merlin server is running and the TCP/IP port to be used for establishing communications.

The parameters userid and passwd are used by the server to verify that you have permission to connect and "log you in".

There is no built-in limit to the number of servers to which a single client can connect simultaneously. However, the operating system on which the client program is running may restrict the number of open files or otherwise limit resources in a way that limits the number of servers that can be accessed; similarly the number of clients one server can support may be restricted by the operating system of the server's machine.

Note that there is no "disconnect" function; instead, you deallocate the server object (see dt_dealloc()) to break the connection and free the resources (databases, datatrees, ...) associated with the server.

Most of the polymorphic functions behave as expected with server objects. There are several functions which are worth further discussion:

dt_set_server_timeout(Integer value) ==> Integer old
Sets a connection timeout value for the toolkit. In general, it is not possible to predict whether or not a server connection will finish. It is possible for the connect functions to go away forever. In 4.61, it is possible to set a timeout, after which time the connect function will return failure. The default is not to use a timeout (timeout is zero).

dt_getusers(Handle server) ==> Handle sos
Returns a sequence of strings (SOS); each string object contains the name of a currently connected user and the number of connections that user has to the server.

dt_ping(Handle server, string text) ==> boolean ok
"Pings" the server: sends the text on a round-trip to the server and back to verify that the connection is working correctly.

dt_server(Handle ob) ==> Handle server
This polymorphic "convenience" function works on any descendent object of a server (e.g. databases, hitlists, TDTs, etc.). It returns the server object that is the ultimate "ancestor" of the specified object. For example: 
     dt_server(database)  == dt_parent(database)
     dt_server(hitlist)	  == dt_parent(dt_parent(hitlist))
     dt_server(fieldtype) == dt_parent(dt_parent(dt_parent(fieldtype)))

14.3 Security

The THOR and Merlin servers are entirely responsible for security in the Daylight system. The basic idea is that clients aren't trusted; if the servers trusted the client programs, a devious user could easily write a program to masquerade as a "proper" client, and could thereby gain access to databases. If you are concerned with security, you should never assume that client programs are legitimate; only trust the servers' security mechanisms. Client security is no security.

The THOR and Merlin servers share a single security mechanism, and usually share a single passwords file. (See the Daylight THOR-Merlin Administration Manual for how servers are started with different passwords files.) This implies that all changes that affect a THOR server (such as adding a user or changing a password) will also affect a Merlin server running on the same machine. This is also true for multiple THOR or Merlin servers running on a single machine: unless they were started with options to give each a separate passwords file, all will share changes made to any.

14.3.1 Restricted user DX_INFO_USER

A special user, DX_INFO_USER (defined in the Merlin and THOR "include" files), is restricted by the servers to a subset of the capabilities normally available. Specifically, DX_INFO_USER can't open databases or ask about security information; this user can only connect to the servers and ask what databases are available.

The purpose of DX_INFO_USER is to allow client programs to query all available servers and assemble a complete list of databases. A user can thus avoid providing the user/password login for every server, and instead only has to provide the user/password for those servers that have databases of interest.

Note that the restricted user DX_INFO_USER appears in the passwords file like any other user; if it is removed, or if a password is added for DX_INFO_USER, the "no-login" query capabilities will not be available. This allows administrators at particularly security- conscious sites to disable this feature. Thus, programmers who take advantage of the DX_INFO_USER capabilities must be prepared for it to be unavailable.

The specific Toolkit functions available to DX_INFO_USER are: 

     dt_mer_server()
     dt_ping()
     dt_getdatabases()
     dt_exists()
     dt_info()
     dt_ispublic()
     dt_isopen()

14.3.2 Adding and Changing Users and Passwords

dt_setpassword(Handle sh, string who, string authorizing_pw, string value) ==> boolean ok
Changes entries in the server's passwords file. This function is used to add or delete users, change their passwords, and to add "equivalent hosts" or "restricted hosts". See the Daylight THOR-Merlin Administration Manual for more information about server security.

dt_getpasswords(Handle server) ==> Handle sos
Returns a sequence of strings (SOS), each string-object of which contains one line from the server's passwords file. There are two types of entries in the passwords file: user/password entries, and host entries.

Back to Table of Contents
Go to previous chapter Program Objects Toolkit
Go to next chapter THOR and Merlin: Databases/a>.