Thor

• Data quoting is uniformly enforced.

  • All data containing characters used for datatree syntax  $ < ; > |  must be quoted in lexical datatrees.

  • Same as previous quoting convention but now enforced uniformly.

  • We needed to do this at revision-time for reactions. (which use >)

  • Note that datatype definitions now require quotes, e.g. $D<"$SMI">, or _P<"*;*">

  • All databases must be reloaded (thorfix451 is supplied).

  • No changes are visible when working at the object level (i.e., with toolkits).

• Datatree merging moved to server for performance

  • Loading Thor databases was pretty slow even if "raw" data loading was used (i.e., database reloading). Moving the merge operation to the server makes datatree merging much faster (5x) for raw data loads. (Previous behavior: the client-side got the extant datatree from the server, merged it with another, and returned the merged datatree to the server.)

  • The server now checks to see if the new tree fits in the current record's location; if so, it uses it rather than creating a new one and dinking the hash lists. When deleting records, the server now coalesces adjacent empty blocks when able.

  • Extra reserved space is inserted following commonly-used cross-references in an adaptive manner. Such space is not subject to garbage collection (except when the database is explicitly crunched).

• Automated indirect key generation

  • The new thorload option -GENERATE_INDIRECT will invoke automatic generation of indirect data keys. Use of indirect data can greatly reduce Thor database size and Merlin search times. Indirect referencing has always be good for computers; now it's convienent for humans.

  • The options (-EXCLUDE_INDIRECT and -INCLUDE_INDIRECT combined with datatype specifications allow precise control over automatic key generation.

  • Automatically generated indirect keys obey all the datatree conventions including subfield delimiters (~).

  • The key generation uses a particularly clever quadruple hash scheme which is fast and efficient in space.

• Component-tuples (PART_NTUPLE normalization)

  • Datafields which have the PART_NTUPLE normalization are treated as vectors of data which have a correspondence to the disconnected components of a structure. The order-correspondence of the vector is maintained on canonicalization. Like other ntuples, any kind of data can be stored in component-tuples (string, integer, real, binary, etc.)

  • Such vectors are very useful for representing things like mole-fractions of mixtures and stoichimetry of reactions.

  • Component fingerprint-tuples may be used to enhance the speed of atom-level searching in large mixtures.

• Fingerprint-tuples

  • The fingerprint program will produce component-tuples of fingerprints (FPP datatype) when given the new -m option.

  • A database may contain both FP and FPP data which will be used as available and needed. For databases of large dot-disconnected mixtures, this can produce a significant increase in screening speed (with a penalty in database/pool size)

• Reaction normalizations

  • USMILES -- when applied to reactions, the unique SMILES normalization preserves reactant/product roles, removes the agent and discards atom maps.

  • AUTOGEN GRAPH -- reaction roles are discarded when making GRAPHs, leaving a normal unique SMILES of the dot disconnected reactants and products. Aside from its normal use in tautomer lookup, this is useful for indexing reactions in the reverse direction.

  • MAKERXNMOL -- takes three tags (reactant, agent, product) and when applied to reaction data, autogen's three dataitems by role (e.g., $RMOL, $AMOL, and $PMOL).

  • ATOM_NTUPLE, BOND_NTUPLE, PART_NTUPLE -- reaction tuples are processed as if the > reaction delimiter was a dot, i.e., tuples apply over the entire reaction..

• Alternative file suffixes

  • Previously thorserver required the primary thor database file to end in ".THOR". A new thorserver option (DATABASE_SUFFIX_LIST) allows a bar-delimited list of alternative suffixes.

  • The default DATABASE_SUFFIX_LIST is ".THOR|.TDB|.tdb"

  • Other database file names are literally as per the primary file, e.g., both base_datatypes.tdb and base_dts.tdb are valid

  • This is required to operate with some filesystems (e.g., ISO-9660 (lowest common denominator CD-ROM format) limits file names 8.3 characters).

• Relative database pathnames in .THOR file

  • Database file names in a .THOR file are now interpreted as relative paths from the directory containing the .THOR file.

  • This makes it easier to move databases around or put them on removable media such as a CD-ROM.

• New "read-only" Thor database property is specified in .THOR file or by file permissions.

  • Thor databases now have a "readonly" property which is saved in the .THOR file and typically set using sthorman or thorchange -setaccess.

  • Databases which are on "readonly" media (e.g., CD-ROMs) are automatically readonly.

  • Absolutely no writing takes place for a readonly database (e.g., no .LCK file is created).

• Live Thor databases can be put on CD-ROMs.

  • "Live" databases can be run directly off their CD-ROM (they don't need to be copied to disk).

  • Given alternative suffixes, relative pathnames and readonly capability, live databases can be "burned" onto CD's even in the very restrictive ISO-9660 format.

• Non-identifiers can be cross-referenced.

  • Non-identifier dataitems in a Thor datatree which is preceded by a slash ('/') are automatically cross-referenced. to the tree root (which may or may not be a SMILES).

  • Allows you to build many-to-many relationships -- powerful but use with caution!

• New "thorfilters" program: thordbinfo

  • Provides database status including header contents

  • With this functionality, the set of thorfilters does everything that sthorman does interactively. Now sh and perl scripters are first class citizens.

  • An example:

                $ thordbinfo chemreact@sol
    
                Enter server access password for sol:thor:dave >> 
                Enter password for database chemreact@sol:thor:* >> 
    
                name: /usr3/thordb/chemreact.THOR
            modified: Fri Dec 13 10:51:46 1996
        datatypes db: chemreact_datatypes
         indirect db: chemreact_indirect
          monomer db: 
         Number TDTs: 389792
        Number Xrefs: 1316390
           TDT bytes: 363036084
          Xref bytes: 276885576
      TDT hash table: 500009
     Xref hash table: 2000003
    Autocrunch limit: 0%
     "HOLD" database: TRUE
           Read-only: FALSE
      Record locking: DISABLED
           

• Lone hydrogens are removed during GRAPH normalization (bugfix).

• Monomer table caching

  • Thor clients connecting to a database with a defined monomer table download the whole monomer table the first time it is used (e.g., for normalization). This can get time-consuming if there are many thousands of monomer definitions.

  • The capability of caching monomer tables in a local directory (e.g., /tmp) was added.

  • The new dt_info() property "monomtime" returns the date and time that a database's monomer table was last modified.

  • These functions are combined to produce the following behavior: if a local directory is specified, the monomer table is cached in that directory only if it doesn't already exist or it has been changed since the table was cached.

  • This is most suitable for remote toolkit applications and for applications which access combinatorial databases over slow lines.

Merlin

• Reaction searching
  • Character screening changed to accomodate new syntax
  • Widgets handle reactions (grins and depict in xvmerlin)

• Tversky similarity search (also dt_fp_tversky())
  • At the extremes, restores the "similarity as superstructure" and "similarity as substructure" metrics missing since the 'ol VAX version v3.6.
  • Provides a user-definable continuous range of super- to sub-structure similarity as described by John Bradshaw it a previous EuroMUG.
  • Very powerful for reaction searching.

• Component-tuple fingerprint optimization