Daylight v4.9
Release Date: 1 February 2008

Name

tdtcat - Format TDT files in dump or list format

Unix Synopsis

tdtcat [options] [file ...]

Description

tdtcat reads TDTs (Thor Datatrees) from its input file(s) and copies them to standard output. It removes whitespace characters that are not part of the datatree (e.g. newlines, tabs between dataitems or between datatrees) and prints the output in either "list" format (one dataitem per line) or "dump" format (one datatree per line).

An input file named "-" indicates that standard input is to be used; for example, "tdtcat foo.tdt - bar.tdt" will print foo.tdt, then print from standard input until EOF (end-of-file) is encountered, then will print bar.tdt. If no file is specified, standard input is used.

Options

-TDT_OUTPUT_FORMAT LIST|DUMP
Specify whether TDTs are written in "list" format (one line per dataitem, or "dump" format (one line per TDT).
-OUTPUT_FILE outfile
Specifies the file to which output is to be written. Default is "standard output".
-SKIP_RECORDS N
Skip the first N TDTs of the input file. A typical use of this option would be to resume some operation on a TDT file (such as fingerprinting) that failed part-way through. -SKIP_RECORDS is applied before -DO_RECORDS.
-DO_RECORDS N
Write N records to the output file then quit. A typical use of this option might be in conjunction with the -SKIP_RECORDS option (above) to process a file in "chunks".
-VERBOSE TRUE|FALSE
If TRUE is specified, prints on standard error the names of the input files as they are processed.

For backwards compatibility, the following older "UNIX style" options are supported:

"-d" is the same as "-THOR_OUTPUT_FORMAT DUMP".
"-o file" is the same as "-OUTPUT_FILE file".
"-s N" is the same as "-SKIP_RECORDS N".
"-v" is the same is "-VERBOSE TRUE".

Return Value

Returns 0 on success, 1 if an error is detected.

Examples

Combine two TDT files, print in "list" format:
tdtcat file1.tdt file2.tdt > combined.tdt
Remove timestamps from a TDT file, print in "dump" format (see "BUGS" below):
tdtcat <file.tdt | grep -v "^TS<" | tdtcat -d >out.tdt

Files

$DY_ROOT/bin/tdtcat

Daylight License

No license required.

Related Topics

thorfilters(1)
dayevict(1) daymessage(1) merlindbping(1) merlinload(1) merlinls(1) merlinping(1) merlinwho(1) thorcrunch(1) thordbping(1) thordelete(1) thordestroy(1) thordiff(1) thordump(1) thorlist(1) thorload(1) thorlookup(1) thorls(1) thormake(1) thorping(1) thorwho(1)

Bugs

There is no inherent limit to the length of a TDT, or to the length of any one dataitem in a TDT. Daylight software is designed to handle TDTs and dataitems of any length, but many other programs are not so robust. In particular, UNIX programs such as grep(1), vi(1) and sed(1) are often limited to 1024 characters in a single line, and thus are unsuited for use with TDTs.

tdtcat reads the entire TDT into memory before printing any of it. Thus its process size will grow to at least as large as the largest TDT that must be processed.

If a TDT file contains an unbalanced quote ("), tdtcat will try to find the matching quote regardless of how far it goes. Typically, this will result in it trying to read the entire remainder of the TDT file into memory, resulting in an "OUT OF MEMORY" error.