Defining Custom Processing Scripts with PGS Express

PGS Express is a subset of TeraPGS that invokes custom scripts to generate products and then automatically distributes those products. PGS Express handles multiple telemetries (data streams) and can deliver either a dataset (TDF and HDF) or a picture product (GIF, GTIF, JPG, MIFF, PCX, PNG, and TIFF).

PGS Express consists of two GUIs, as well as custom user scripts and the distributor.

PGS Express GUIs

PGS Express Template:

The system administrator or programmer creates a template that identifies a custom processing script, one or more data streams, and an output format.

PGS Express:

The user selects a template, selects a single data stream if the template contains multiple data streams, and defines an area of interest (optional) and distribution point(s) for a product.

Custom Processing Scripts

These are user-created processing scripts that are invoked to do the actual processing and product generation. These scripts must be written so that they accept the following command-line arguments in the order shown: stream, master, output file, input file(s), optional user arguments.

The PGS Distributor

The PGS distributor is a daemon that manages delivery of products. Delivery is independent of the time of product generation and doesn't interfere with product generation. The distributor has these features:

Delivery of multiple products at a time to multiple recipients.
Delivery by one of two means - FTP or copying (remote copy is not available for PGS Express) - with output file naming handled via a template.
Delivery-attempt options.
Options for post-delivery scrubbing of local directories.

Creating a Product Definition

Step 1.	Use pgs_express_template to create a template that specifies a single telemetry/data stream or multiple data streams, the output format for the product created, and the path for the custom script that will be used to create the product(s) for the data stream(s) specified.
Step 2.	In pgs_express, select a template previously created in pgs_express, then select a single data stream and area of interest (AOI), attach a distribution record, and create new product definition.

Start pgs_express_template:

1.	Select File --> New. When prompted, enter new process (template) name and click OK. Then click OK at the next prompt to return to the Custom Script window.
2.	Select the Output Format for the product (hdf, tdf, gif, jpg, etc.)
3.	Supply the Program Path for the custom script that will be run. Use the Browse button to select a path or enter the path directly into the text field.
4.	Select a Data Stream from the list.
5.	Click Accept. This enters the process name you specified earlier into the Title field.
6.	To create additional templates, repeat steps 1-5.
7.	Select File --> Exit (and save) to save the template(s).

Start pgs_express:

1.	Select File --> Load to load an existing product definition to edit.
or
1.	Select a Process Definition (created in pgs_express_template) from the drop down box.
2.	Select Active (default), [Product Type (picture/data - this is not enabled)], Priority (1-5, with 1 the highest priority), Data Stream (will only be one if only one selected in template; if multiple, will be available in drop down box), AOI or define a New one graphically using TeraMaster.
3.	Enter a Title. This becomes the title of the process definition defined here and is selectable from File -> Load.
4.	Enter a User Argument. These are arguments to be passed to and executed by the custom script.
5.	Click on the Distribution button to create a delivery point for the product to be created by the custom script. Then, from the Pick from Master Distribution Database dialog, select a distribution record from the list on the left side of display and click on Add to attach it to the process definition. To change any of the delivery instructions associated with process definition, select a delivery point from the list on the right and click Edit. Click Done to return to main panel.
	Note: To create a distribution record, select Edit --> Edit Master Distribution. Click here for instructions on creating or editing a distribution record.
6.	Select File --> Save. The filename specified here becomes the name of the .pgc process definition file saved in $PASSDIR/pgscustom/process_definitions and also the name of .dist distribution file for this script in $PASSDIR/pgscustom/distributions.

Linking to Capture:

Live Data. pgs_ingest is specified as the Post Process on the autoschedule record in TeraCapCon.: pgs_ingest calls pgs_pproc, which builds a list of custom procedures from $PASSDIR/pgscustom/process_definitions/*.pgc and custom distributions from $PASSDIR/pgscustom/distributions/*.dist. Then, based on data stream and priority, it runs pgs_custom, which runs the custom script, puts the output in $PGSWORKDIR/pgscustom, and then runs pgs_distrib to distribute the products.

Remote Ingested Data. pgs_waitd_ingest is specified on the Waitd panel of the Configuration Editor.: pgs_waitd_ingest is specified as the program to run for a particular telemetry. The user specifies the incoming directory to watch and the filename search template. It will create $DATADIR/whole_pass/<telemetry> if it doesn't exist.

To find out more on how to link your TeraPGS product definitions to data capture, click here.

Last updated: $Date: 2002/09/10 20:21:52 $