waitd - wait for incoming data then process it daemon

SYNOPSIS

waitd  [ -g | -i ]

DESCRIPTION

waitd is a daemon that waits for data being delivered to one or more directories, and then initiates processing on that data after its been delivered.

waitd determines what to do based on the configuration file $PASSDIR/incoming. Blank lines in the file, and lines starting with '#', are ignored.  $PASSDIR/incoming is an ASCII file with the following fields:

telem-or-format

Indicates kind of data being delivered; it can be a real telemetry name (e.g., hrpt) or not (e.g., fnmocgrib).

incoming-dir

Directory where files are arriving; if this directory does not exist when waitd is started (or restarted), the corresponding record is inactive.

name-template

Wildcard expression for matching files to be processed. If there are multiple matching files, only the oldest is processed at any given time.

process-name

Process to run on arriving files; the process is run as

  process-name fully-qualified-matching-file-name

If it is not fully qualified, process-name must be in one of the directories in $PATH.

wait-until-age

Matching files must be at least this old (secs) to process.

keep-after

Number of matching files to keep around after processing; excess matching files are removed.

hours-too-old

Get rid of matching files this many hours old, whether they have been processed or not.

waitd logs all of its actions in $PASSDIR/incoming.log. This log file can be read using tlogview or any text editor. The output from processes started by waitd are directed to files with names $PASSDIR/logfiles/incoming.number.letter.  number is the 1-relative number of the record in the $PASSDIR/incoming configuration file.  letter is a letter in the range A-Z. tlogview can be used to easily read these per-process output files.

waitd keeps an ASCII history file around in order to remember what actions have already been taken. This file is called $PASSDIR/incoming.history, and has the following fields:

fully-qualified-file-name

Full pathname of the processed file.

process-name

Name of the process used to process the file.

last-modification-time

Last modification time of the file, in seconds since midnight, January 1, 1970.

process-status

Process status: (0) process started, (1) process terminated in an unknown state, (2) process terminated in a known state, regardless whether it was successful or not.

This file is not intended to be edited by the user. However, removing lines from $PASSDIR/incoming.history before starting waitd will cause waitd to forget certain actions.

waitd supports the following signals:

SIGHUP

Restart the program, re-reading the configuration file.

SIGTERM

Exit program gracefully.

SIGINT

Go into hibernation (idle mode).

SIGCONT

Get out of hibernation (normal mode).

SIGALRM

Skip next sleep step.  SIGALRM can be used by processes wanting immediate action from waitd, e.g., check directories now.

PARAMETERS

-g

Normal mode. waitd automatically puts itself in the background in non-idle mode. waitd will stay in non-idle mode until it receives a signal to do otherwise.

-i

Idle mode. waitd automatically puts itself in the background in idle mode. waitd will stay in idle mode until it receives a signal to do otherwise.

EXAMPLES

The following is a sample record from $PASSDIR/incoming.

gvar /home/xtuser/incoming *.g8 gvar_incproc 60 1 4

The incoming data telemetry/format is gvar. Data files are being delivered to /home/xtuser/incoming, with name extensions '.g8', and are to be processed by the script gvar_incproc. Incoming files more than 60 seconds old are assumed to be completely delivered and ready to process. At most 1 processed file is allowed to remain in the /home/xtuser/incoming directory at any given time. Matching files in /home/xtuser/incoming that are more than 4 hours old are automatically removed, whether or not they have been processed.

FILES

$PASSDIR/incoming, $PASSDIR/incoming.log, $PASSDIR/incoming.history

SEE ALSO

tlogview

NOTES

waitd processes one file per line in $PASSDIR/incoming at a time. If there are N records in $PASSDIR/incoming, there can be at most N processes started by waitd running at the same time. Users can defeat this limitation by using a process that invokes a background sub-process and then exits without waiting for that sub-process to complete.

It is recommended that successful processes remove the input data files when/if they are no longer needed.

waitd can wait for groups of files (e.g., files with the same names, but different extensions) provided these files are delivered sequentially, in the same order all the time. For example, suppose files with extensions .avhrr and .tovs were being delivered to a given directory, and that each .tovs file is always delivered after the corresponding .avhrr file. In this case, the name-template in $PASSDIR/incoming should be '*.tovs'.

Last Update: $Date: 2001/12/14 18:42:16 $