Daylight v4.9
Release Date: 1 February 2008

Name

thordump - directly dump the contents of a THOR database

Unix Synopsis

thordump [options] file.DP

Description

thordump is a program intended for last-ditch situations, where a THOR database is damaged and can't be opened by thorserver. The most likely cause of such corruption is that the computer crashed or the server was killed while the server was actively modifying the database.

thordump takes a single file as its argument, which will normally be the file with the suffix ".DP" (e.g. "$DY_THORDB/mydatabase.DP"); the .DP file contains all of the raw data in the database and can be used to reconstruct the entire database.

When the dump is finished, thordump reports the number of TDTs saved and the number of errors (corrupt sections of the database) encountered. Each time an error is detected, its location and size is reported. See the REJECT_LOG option below.

Before recovering a database with thordump, you should first try to "crunch" it (see sthorman(1), thorcrunch(1)). The crunch operation causes a database's hash tables to be rebuilt; often the damage to a database is only a corrupt hash table.

Because thordump reads the .DP file directly, the Thor and Merlin servers are not used, so the normal login/password procedure is circumvented. If your Thor database files have world read permission, anyone can use thordump to get all of your data. Be sure your database have the correct file permissions.

It is also possible to use thordump to dump the contents of the cross-reference (".DX") datafile. This is never necessary since all the cross-reference information is derived from the .DP file.

Options

-TDT_OUTPUT_FORMAT LIST|DUMP
Specify whether TDTs are written in "list" format (one line per dataitem), or "dump" format (one line per TDT). Default is LIST.
-REJECT_LOG file
If corrupt TDTs are encountered, they are skipped (not printed in the output file). If this file is specified, all corrupt records are saved in the file. It is sometimes possible to recover these records manually. Default is no file (corrupt TDTs counted then discarded).
-SKIP_RECORDS N
Skip the first N TDTs of the input file. A typical use of this option would be to resume some operation on a TDT file (such as fingerprinting) that failed part-way through.
-DO_RECORDS N
Write N records to the output file then quit. You can use SKIP_RECORDS and DO_RECORDS together -- SKIP_RECORDS is applied first, then the the number of TDTs specified by DO_RECORDS is processed.
-SKIP_BYTES N
Skip the first N bytes of the TDT the .DP file. This is used to skip over portions of a file that are so badly damaged that thordump can't get past them. The dump will actually start on the next "sensible" record after the Nth byte. You can use SKIP_BYTES, SKIP_RECORDS and DO_RECORDS together -- SKIP_BYTES is applied first, then the number of TDTs specified by SKIP_RECORDS is skipped, then the the number of TDTs specified by DO_RECORDS is processed.
-MINOR_REPORT N
The interval (number of TDTs) between minor reports. The minor report is a period "." printed on "standard error". N = 0 suppresses the minor report. Default is 10.
-MAJOR_REPORT N
The interval (number of TDTs) between major reports. The major report prints the number of TDTs processed and the number of errors so far, followed by a newline, to "standard error". N = 0 suppresses the major report. If both MINOR_REPORT and MAJOR_REPORT are zero, summary information that is normally printed at the end is also suppressed. Default is 500.

Return Value

Return status is zero if the dump succeeds, or one if a problem is detected. In addition, diagnostic messages may be printed on standard error:

out of memory

There is not enough swap space. If the .DP file itself is corrupt, it may appear that a TDT is very large, thus causing this message.
can't read hash-table data file
The database is corrupted beyond redemption. Very unfortunate.
corrupt hash-table data file
Same idea as above.
Can't write record
An error storing the data that is being dumped. Usually due to no write permission on the file or writing to a full disk.

Files

$DY_ROOT/bin/thordump

Daylight License

No license required.

Related Topics

dayevict(1) daymessage(1) merlindbping(1) merlinload(1) merlinls(1) merlinping(1) merlinwho(1) thorchange(1) thorcrunch(1) thordbping(1) thordelete(1) thordestroy(1) thordiff(1) thorlist(1) thorload(1) thorlookup(1) thorls(1) thormake(1) thorping(1) thorwho(1)

sthorman(1) thorserver(1) merlinserver(1) licensing(5)

Daylight Theory Manual, Daylight System Administration Manual

Bugs

thordump doesn't check for locks on a database; a thorserver may have a database open and be actively writing to it while thordump is trying to dump it. This can potentially cause thordump to report errors even for databases that are not corrupt.

It is tempting to use thordump(1) instead of thorlist(1) or other client programs as a backup mechanism for the database. Don't do this unless you are sure no one is actively writing to the database.

If the files that make up a database are not properly secured by the operating system's permission flags, thordump can be used to read the database's contents directly, thus circumventing the THOR and Merlin server's normal security mechanisms.