What is TeraPGS?
product-definition GUI daemons processing scripts directories and environment variables
Graphic overview of PGS processing.
TeraPGS (TeraScan Product Generation System) is a system for automatically generating and distributing TeraScan products according to user specifications. TeraPGS is intended to simplify the procedures for creating TeraScan products. In addition to its processing capabilities, TeraPGS can package products and distribute them to various destinations by copying to local computers or by FTPing to remote sites.
Using the TeraPGS GUI, the user can construct product definitions which are saved in TeraScan's $PASSDIR/pgsprocs directory (product-definition files can be identified by their .pgs extensions). Each product definition is a recipe for one or more related TeraScan products. Each product definition can prescribe up to one TDF dataset and up to three picture products (in formats such as JPEG). The product definition also includes a distribution scheme to specify destinations and methods for delivery of the resulting products.
Once a user has set up product definitions, TeraPGS processing scripts will check the definitions each time data is ingested, or ingested data is received from a remote machine, and match the product-definitions to the telemetry received, generating products accordingly.
TeraPGS can wrap a product or set of products either for external or internal delivery. A package for external delivery can be addressed and sent to one or more destinations on the local network or outside the user's network by FTP or remote copy. A package for internal delivery can be locally copied to a specified directory on the user's workstation.
The options available on the TeraPGS GUI will depend on the type of telemetry selected for processing. TeraPGS can work with a wide variety of satellite telemetries: HRPT, GVAR, RTD, PDUS, SeaWiFS, SVISSR, WEFAX, and CHRPT data are all handled by TeraPGS.
In addition to these telemetries, there is a special multigeo data stream (pseudo telemetry) that is really a composite of all available geostationary data processed in the TeraPGS $DATADIR/whole_pass directory. Click here for more information about this telemetry.
TeraPGS can run several product definitions simultaneously on the same satellite pass. Each product definition is assigned a priority by the user. Because TeraPGS can handle multiple product definitions simultaneously, a product definition with a lower priority still has a chance to be run.
An alternative application, PGS Express, is a subset of TeraPGS that invokes custom scripts to generate products and then automatically distributes those products. For more information on PGS Express, click here.
TeraPGS, as well as PGS Express, can work with live data captured on your system, as well as data ingested on a remote system and received on your system. To find out how to link your product definitions to data capture, click here.
Files, Daemons, Scripts, and Functions
The following is a discussion of how TeraPGS is set up on your system and the location of the scripts, daemons, and functions that are running behind the scenes to create products.
These files are all editable, but should only be edited by an experienced user. If these files are corrupted, then you stand a good chance of rendering your system inoperable. Also, while we show a certain file residing in a specific location, it is possible that the file could reside elsewhere on your system.
Picture and Dataset Variables
When you select a telemetry type in TeraPGS, there are both Dataset and Picture variable options. The Picture and Dataset variables displayed on the GUI are telemetry specific. If you select HRPT, for instance, you will see a list of variables that is different from the list you would see if you had selected GVAR as the input telemetry. The file that controls these variables is pgsvars, which resides in $REFDATA/terapgs.
pgsgens, which is also located in $REFDATA/terapgs, controls the options you are presented with in the Image Maker field. Like the Picture and Dataset variables, the Image Maker options are also telemetry specific.
Environment Variables and Directory Structure
The environment variables for TeraPGS are all defined in the tscan.login file, which resides in $PASSDIR.
$ZAF_PATH - The ZAF_PATH variable must be set correctly in order for your TeraPGS GUI to work. The ZAF_PATH setup should be:
setenv ZAF_PATH $REFDATA/Zinc
or something similar. The ZAF_PATH variable should be pointing to the terapgs.znc file. You should have no reason to change this setting, unless you have moved the TeraPGS executable and .znc files.
$DATADIR defines the directory where all the ingested satellite telemetry resides. The directory is broken down by telemetry. The format for the telemetry stored in $DATADIR is yymmdd.hhmm.cnn. So, for example, 980401.1234.n14.avhrr would describe a NOAA-14 pass that was captured on April 1, 1998-2000 at 12:34 PM. The $DATADIR directory is further subdivided, mainly as an organizational tool.
The PGSSPOOL variable defines the temporary directory where data is spooled, or held, while awaiting distribution.
The PGSWORKDIR variable defines the directory where TeraPGS does all its construction. The PGSWORKDIR directory should have at least 200 MB of space available for TeraPGS to use in conjunction with creating product. After TeraPGS is done constructing product, it will clean up the PGSWORKDIR directory and eliminate any unnecessary files. Both PGSSPOOL and PGSWORKDIR variables are defined in the tscan.login file.
Saved TeraPGS Product Definitions
All saved TeraPGS product definitions are stored in $PASSDIR/pgsprocs. Saved TeraPGS product definitions have a .pgs extension. Users can modify these files, if comfortable doing so, with a UNIX editor such as vi.
TeraPGS Directories and Environment Variables
When you first install TeraPGS, you will need to create three directories for TeraPGS to use in constructing products. You will then need to enter the directory information into the system configuration file and define environment variables for the directories in the tscan.login file. The directories you need to make are the Data Directory, the Work Directory, and the Spool Directory. To find out how to set up the TeraPGS directories and environment variables, click here.
Note: Please read the instructions carefully. If you do not set up and define these directories and environment variables properly, TeraPGS will not run on your system.
The TeraPGS distribution daemon, terapgsd, must be started at boot. The daemon functions like a print spooler to distribute products defined in the GUI. The daemon resides in the tscan.rc.local file, which should be in the $PASSDIR directory.
If your system should crash or otherwise not shut down gracefully, you or your system administrator will need to remove the terapgs.lok file located in the $PGSSPOOL directory. Once you have removed the terapgs.lok file, start the distributor by entering terapgsd& at the command line.
pgs_checkd is a daemon used in two places: tscan.rc.local and pgs_ingest. pgs_checkd checks to see if the distribution daemon is running. If terapgsd, is not running, then pgs_checkd makes sure the $PGSSPOOL directory is not overflowing (it will scrub the directory down to a maximum of 100 files), and then starts terapgsd.
A script is a file that acts upon telemetry in your system in a certain way. Prior to TeraPGS, users would run functions on the command line and generate scripts. Once set, the script would be a useful tool in generating products quickly. The problem, of course, comes if you want to modify or change the script in some way. TeraPGS allows you to define products with the GUI interface. But behind it all, TeraPGS still runs scripts to generate the products. The following is a brief description of some of the scripts in TeraPGS and what they do. These scripts are stored in $TSCANROOT/bin. Click here for a graphic overview of PGS processing flow.
pgs_ingest
This script identifies the correct, telemetry-specific ingestor to run and is also responsible for disk utilization. This script is the primary data ingest script and can be linked to data capture.
pgs_waitd_ingest
This script is specified in the Waitd panel of the Configuration Editor as the program to run for remotely ingested data. The waitd daemon monitors a specified incoming directory for a particular telemetry and places the data in the telemetry-specific $DATADIR/wholepass directory. Processing then continues as for live data capture.
pgs_${telem}in
These telemetry-specific scripts (e.g., pgs_hrptin, pgs_modisin) handle ingest, calibration, and navigation of the telemetries. There is also some telemetry-specific processing for each of the telemetry types. pgs_multigeo is a special "ingestor" used to create multi-geostationary image composites.
pgs_pproc
This script handles scheduling and initiation of the rest of the TeraPGS processing tree. It builds a list of product definitions from $PASSDIR/pgsprocs (.pgs files) and custom scripts defined in $PASSDIR/pgscustom (.pgc files) to be run by priority: run pgsprocs first, then custom scripts.
pgs_make
This script controls the generation of both data sets and picture sets.
pgs_simpletdf
This script is responsible for generating data sets containing variables specified for delivery as data, as well as variables used in picture making to include synthesized variables. This script is also responsible for any remapping of the data as well as delivering any data sets scheduled for shipment to the distributor.
pgs_overlay
This script handles the specifics of generating pictures of the datasets with the prescribed overlays (wedge, legend, coast, lat/ lon grid, etc.). It also handles export of the pictures into one of several graphics formats such as TIFF, GIF, Postscript, etc.
pgs_died
This script invokes the distributor daemon (terapgsd) upon failure to complete a task. It allows an "excuse" file to be sent along (usually as e-mail) containing more detailed information on the type of failure.
Copyright © 1998-2000, SeaSpace Corporation. All rights reserved.
Last updated: $Date: 2002/09/17 23:57:04 $