TeraScan 3.2 Release Notes

Software Release Date: April 2002

Errata - Status of TeraScan 3.2 since release

This document pertains to the TeraScan 3.2 software release and summarizes the features added and bugs fixed since TeraScan 3.1.2.

Several new functions have been added. New GUIs have also been added for monitoring pass reception, satellite tracking, and running custom scripts using the TeraPGS product generation and distribution capabilities. These new functions and GUIs are indicated by New! in these Release Notes.

Only these Operating Systems will be supported for TeraScan 3.2:

Solaris: 2.5.x, 2.7, and 2.8
Linux: 7.2

This document contains links to specific pages of the TeraScan 3.2 help system. Thus, such links will only be relevant once TeraScan 3.2 has been installed on your system.

A separate document called TeraScan 3.2 Installation Instructions explains how to install the TeraScan 3.2 release and how to configure the system to work with the software.

New and Improved Features

This section summarizes the most notable of the new or improved features of TeraScan 3.2.

angles

If the input dataset contains multi-resolution data, the new optional master_var parameter can be used to specify the variable for which angles are to be computed. The output angles variables will have the same dimensions as the master_var. For examples of usage, see the angles help page in TeraScan 3.2.

ATOVS/TOVS Processing

atovsin--NOAA-16 hirs calibration updated. Daily hirs statistics filenames are now satellite-specific. amsu-a and amsu-b sensor data input updated.

amsua, amsub, and hirs sensor files generated for NOAA-16. HIRS calibration parameters added to NOAA-15 hirs file.

amsu_b_ret--Function changed so it works better with full rev data (e.g., from gac)

tovscdf--Time variable added to output.

Capture

archive--Several new operations have been added: dtwrite and dtread operations allow for archiving of processed datasets. header operation is currently the same as where, except that header information is written to stdout without any titles. mrestore operation restores multiple passes from tape. New optional parameter added: ansi_file_hdr prepends output filename with ANSI header for output of global imaging raw pass data from a TeraScan online pass. For examples of usage, see the archive help page in TeraScan 3.2. Function modified to work with very large telemetry record sizes to accommodate X-band telemetries.

eosam telemetry for terra-1 satellite changed to teradb. See $SATDATA/terra-1/Define.

send_pass and recv_pass--New! functions can send pass data across the net "live" to/from a remote system via a TCP/IP port. For further information, please see the send_pass and recv_pass help pages in TeraScan 3.2.

X-band cross-level pointing has been improved. The cross level has been changed to allow a maximum elevation of 89.999 (formerly 89.99).

Coastlines

New! World Vector Shoreline coastline databases wvsplus.coast and wvsplus.political were added. These databases provide better/newer political boundaries than are available from the Digital Chart of the World (e.g., the Middle East, the Balkans).

dcw.coast was modified to remove rectangular lines east of Tahiti.

convec

convec modified to accept latlon as the input type. A new parameter, distance_units, is used when input is latlon.

DMSP Processing

read_smq11_ssp (U.S. Navy specific)--New! function has been added to read the SSP raw file generated by SMQ-11 systems. read_smq11_ssp generates the same .mi_s, .t1_s, and . t2_s files that rtdin generates -- the difference is that the raw data comes from the SMQ-11 processor, which is not really raw RTD telemetry. The SMQ-11 processor strips the SSP records from the raw telemetry and puts it in a separate file.

Diskspace Management Improved

Directory names specified for diskhogs can now begin with environment variables, allowing scrubbing for directories such as $DATADIR/whole_pass and $PASSDIR/logfiles.

The default $PASSDIR/config/diskhogs.config defines scrubbing of $PASSDIR/logfiles, $PASSDIR/logfiles/mtrack, and $PASSDIR/logfiles/pgs directories. diskhogs does not allow recursive scrubbing in /bin, /lib, /usr, /etc, /opt, /dev, /var, /sbin, and /boot, so multiple entries are required in the diskhogs .config file to keep $PASSDIR/logfiles under control.

NOTE: tsched runs the function diskhogs2 hourly. diskhogs2 reads the $PASSDIR/config/diskhogs.config file and runs diskhogs for each configuration.

Additional directories can be scrubbed using this method as well. For example, you can control the size of $DATADIR/whole_pass by modifying diskhogs.config as follows:
[$DATADIR/whole_pass]
recursive: yes
wildcard: *
remove_tdf_only: no
free_mbytes: 0
max_mbytes: 800
max_age_hours: 0
read_time: no

eof

New parameters across_vars, new_var_name, and burst_modes added so that eof works across channels and can automatically burst out individual nodes.

FY-1C Processing

chrptin--Optional sort_by_var parameter removed..

General Utilities

Better Ellipsoid Support. All master functions now support flattening and equator radius parameters to define an ellipsoidal earth. This forces the datum to be the same between two packages. Recommended only for expert user translating from one package to another.

A new projection--Normalized LRIT Geo--has been added for use with LRIT/digital WEFAX data.

gsubset now works with masters. gsubset can now wait until gvarin has finished processing the subset area before exiting. For further information, please see the gsubset help page in TeraScan 3.2.

findsat and sensorll have been modified to allow simultaneous earth localtion for multiple satellites. For example, you can register NOAA data to GOES data without a master projection (in sensor projection).

make_tsuser script has been modified to create a new local user account from scratch (if one does not already exist) and then configure that account as a TeraScan user. Note: For NIS, NIS+, or YP accounts, you should make the account first via normal means and then run this script. For more on how to use this script to make a TeraScan user, please see the Creating a TeraScan User Account help page in TeraScan 3.2.

tscaninfo--New! script that reports useful information about your TeraScan system: the version of TeraScan software and the packages installed, and system information such as the O.S., the hostname, the hostID, the bus type, etc. This information can then be provided to Customer Support if necessary. For more extensive system information, run as tscaninfo full. For further information, please see the tscaninfo help page in TeraScan 3.2.

GMS Capture/Processing

gmsin has been modified to use simplified mapping blocking for earth location when other sources of earth location are unavailable in the data (e.g., satellite attitude, etc; latitude, longitude, line, sample)

manamsched has been modified to accumulate two identical copies of the manam message so that manam scheduling is not affected by bit errors.

geo_simple

geo_simple now reads INSAT image data in Air Force SDHSUIS data format, and outputs that data in TeraScan dataset format. New parameters have been added to allow users to override bad dates and times: user_date_time, pass_date, start_time. For further information, please see the geo_simple help page in TeraScan 3.2.

HRPT Processing

avin/hrptin/avcal changed to support simultaneous NOAA KLM ch3a and ch3b:

hrptin and avin--New optional parameter which_ch3 added to determine which NOAA KLM channel 3 to include in the output.

avcal--Two new output variables cal_good_3a_lines and cal_good_3b_lines have been added for save_cal_vars=yes.

Additional changes:

hrptin--New HRPT formats added: oldlacp, pre-KLM GAC data; selfdef, Self Defense Agency format from Japan; tenbit, raw unpacked HRPT 10-bit data, with no extra bits between words.

hrptin--Optional sort_by_var parameter removed..

dcsin and hrptin--New parameter dcs_ident_file added to identify the ASCII lookup table for computing extended 28-bit DCS platform id's.

DCS processing now works for NOAA-15 and NOAA-16.

For further information, please see the avcal, avin, dcsin, and hrptin help pages in TeraScan 3.2.

HY-1 Ingest

hyin--New! function to ingest COCTS and CCD data from the HY-1 satellite. Input telemetry data can be read from the pass disk, from standard UNIX disk files, or from a tape. Output is COCTS dataset and raw CCD file. The user can specify whether raw data should be de-randomized. User can define the suffix of the output data. Output filename format: yyyy.jjj.hhmm.<satel>.<suffix>.

HY-1 directory added to $SATDATA.

Image Rendering (non-interactive)

rgbscale--New! function has been added for scaling and exporting 24-bit true color images For further information, please see the rgbscale help page in TeraScan 3.2.

imblank--New! function has been added for creating background images that can be injected with metafile overlays and used as a mask For further information, please see the imblank help page in TeraScan 3.2.

sinject--New! function (a version of xinject that does not require the X-windows server to be running) has been added. The biggest difference between xinject and sinject is text output. xinject uses raster fonts from the X-windows font server; sinject uses Hershey stroke fonts (stored in $REFDATA/fonts/Hershey), which are more limited. xinject recognizes different font families used in the different input metafiles. sinject currently always uses Hershey Roman stroke text. For further information, please see the xinject/sinject help page in TeraScan 3.2.

expim--New parameter false_color has been added to specify that output image should be based on 3 variables. Only TIFF, JPEG, and GEOTIFF formats are currently supported for false_color For further information, please see the expim help page in TeraScan 3.2.

Import/Export

tdftohdf/hdftotdf--have been modified so they are near perfect inverses of each other. Thus, if:

A -> todftohdf -> hdftotdf -> B

then A and B are identical (or nearly identical).

nctdf--New! native netcdf importer. tdfnc/nctdf are near perfect inverses of each other. For further information, please see the tdfnc and nctdf help pages in TeraScan 3.2.

Binary import/export functions impbin, impbin1, and expbin have a new optional parameter reverse_bytes to handle byte-swapping (input/output data that does not match the import/export machine's byte order). The reverse_bytes parameter tells these functions whether or not the data to be imported/export is byte-swapped. In addition, if reverse_bytes=yes, then instantiate=yes is forced. For further information, please see the impbin, impbin1, and expbin help pages in TeraScan 3.2. Also see the import help page for more about byte-swapping and getting data ready for import.

impgrib2 has been modified to work with more types of GRIB data, particularly GRIB files concatenated in various time/elevation orders.

Launchpad

Some new Launchpad buttons have been added to open new applications: Tracking Monitor, StatPass, Satellite Tracker, etc. Some documentation buttons have been updated.

Logfiles

The main TeraScan log files--schedlog, pgslog, and incoming.log--now have an .html extension. They contain HTML constructs and are browsable.

tlogview GUI has been modified to accommodate these changes.

Master functions

altmaster--New! function that clones a master file, retaining the area covered but with the option of changing the pixel resolution. altmaster is intended for use by scripts that generate sensor-specific products registered to user-specified areas of interest (i.e., master files). altmaster can be used to change the master file's resolution so it is consistent with the resolution of the input sensor data. For further information, see the altmaster help page in TeraScan 3.2.

mastersub--New parameters master_margin and subset_margin added. These parameters allow the user to make sure the subset is a little bigger than the minimum, so subsequent navigation does not move the subset off the master. For further information, please see the mastersub help page in TeraScan 3.2.

A new map projection--Normalized LRIT Geo--has been added to TeraScan for use with LRIT/digital WEFAX data.

MODIS Processing (for systems with a TeraScan MODIS license)

modisin--Underwent major modification. It now supports live ingest and can read either CADU or PDS data, or read directly from the pass disk. Channel numbers of format modis_ch_## were changed to modis_ch## to match IMAPP-converted output. New parameters added: on_pass_disk, pass_number, reed_solomon, delta_500m, delta_250m, use_master, master_file, full_width, num_scans. Parameters delta_sample, num_lines removed; replaced by delta_500m/delta_250m and num_scans. Additionally, modisin was changed to clip dimension and remove wasted space. For further information, please see the modisin help page in TeraScan 3.2.

modis_bt--New! function corrects calibrated data output by IMAPP for the panoramic bowtie effect due to the behavior of the MODIS sensor cross-track scanning mirror. This effect leads to a scan-to-scan overlap of pixels at scan angles greater than 25 degrees. For further information, please see the modis_bt help page in TeraScan 3.2.

modis_color-- New! function generates water-leaving radiances from calibrated MODIS data. It implements the SeaWiFS-type ocean color algorithms to derive chlorophyll concentrations and aerosol optical depth. For further information, please see the modis_color help page in TeraScan 3.2.

mod_chlor-- New! function generates chlorophyll values from MODIS color variables. It computes the chlorophyll concentration using the normalized water-leaving radiances at 443 and 530 nanometers (nLws). For further information, please see the mod_chlor help page in TeraScan 3.2.

modis_turbidity-- New! function computes the water column turbidity index, NDVI, reflectance, and areosol optical thickness from calibrated, bowtie-corrected MODIS data. For further information, please see the modis_turbidity help page in TeraScan 3.2.

MODIS_RADIANCE-- New! Script converts IMAPP-calibrated HDF scaled integers to TDF MODIS radiance units (W/m^2/micrometer/steradian/micron). For further information, please see the MODIS_RADIANCE help page in TeraScan 3.2.

modis_bright-- New! function converts radiance units to brightness temperatures for MODIS channels 20 through 36 (17 emissive bands). For further information, please see the modis_bright help page in TeraScan 3.2.

OCM Processing (for systems with a TeraScan OCM license)

ocmin--Function restructured. New parameters on_pass_disk and pass_number add ability to ingest directly from the pass disk as well as from standard UNIX files. Parameter calibrate has been added to allow user to specify whether radiometric calibration should be performed on the data (performs the same calibration as ocmcal). If calibrate=yes (default), byte_output specifies whether to save the output as 8-bit or 16-bit integer data. Parameters start_sample and num_samples have been added so that ocmin reads the 3730 samples of active data starting from sample 1144 or all 6020 samples. Parameter num_lines added to allow user to specify the number of OCM lines to extract from the oceansat-1 pass. For further information, please see the ocmin help page in TeraScan 3.2.

ocmcal--Based on information provided by ISRO, function modified to automatically use the correct min and max for each channel of output calibrated data.

ocm_color--New! function generates water-leaving radiances from calibrated OCM data. It implements the SeaWiFS-type ocean color algorithms to derive chlorophyll concentrations and aerosol optical depth. For further information, please see the ocm_color help page in TeraScan 3.2.

ocm_chlor--New! function generates chlorophyll values from OCM color variables. It computes the chlorophyll concentration using the normalized water-leaving radiances at 443 and 530 nanometers (nLws). For further information, please see the ocm_chlor help page in TeraScan 3.2.

Pass Reception Monitoring

statpass--New! GUI that reports status by telemetry for the previous, current, and next scheduled pass. Command line: tstatpass. Available from TeraScan Launchpad.

For the non-GUI version, called by statpass, a satellite mode (statpass -s) was added to report status for the previous, current, and next scheduled pass for each satellite.

Tracking Monitor--New! GUI that presents a continually updated status of antenna tracking and signal reception. The information it displays for each pass received is also stored in a file (in the directory $PASSDIR/logfiles/mtrack) that can be replayed through the GUI for diagnostic purposes. Command line: trackmon. Available from TeraScan Launchpad.

NOTE FOR X-BAND SYSTEMS: qtrackeye has been renamed to trackmon and updated to work with the new mtrack files that contain the tracking history previously stored in dtrack files.

Satellite Tracker--New! GUI that allows multiples satellites to be tracked and displayed simultaneously, along with multiple receiver sites and the sun's location. Orbit plot updates are performed every thirty seconds. All satellites, receiver sites, and the sun are plotted on a base map of the earth in either an orthographic projection or a Mercator projection. Command line: tracksat. Available from TeraScan Launchpad.

rxcheck--New! receiver diagnostic command rxcheck works with all receivers (e.g., HR series, TR100, XR100, Alcatel, Aydin). The rxcom function is retained for use with HR series receivers (HR99, HR100, HR200, HR250, HR300).

rxcircle

A new metafile generator rxcircle generates a coverage circle than can be used in scripts.

Satellite TV (stv)

Satellite TV has been completely reconfigured. New features include:

TV Guide mode--lists incoming and future schedule passes. Graphically displays the orbit of the satellite(s) you are monitoring.

Watch mode--from which you can view the incoming or online pass in Single, Dual, or Color mode. Histogram equalization replaces multiple enhancement controls in previous version of Sat TV. New Zoom feature.

View Online Pass menu choice from the File menu opens the View Pass dialog box from which you can select a pass stored online and view pass coverage before you select it.

Satellite TV now supports chrptin telemetry for the fy-1c satellite, cocts telemetry for the hy-1 satellite, and teradb telemetry for the terra-1 satellite.

For further information, please see the Satellite TV help page in TeraScan 3.2.

Scripting Functions

getconfig--New optional parameter file added. This can be any configuration file in the "settings" format. If the file is not a fully qualified path, it is assumed to be in $PASSDIR.

passinfo--New optional parameter archive_header added to list the information contained in the archive tape header file for a given pass. If the archive_header parameter is used, the pass_number parameter is ignored. The recommended way to use this parameter is as follows:
```
      set HDR = `archive operation=header dev_name=$TAPE`
      set X = `passinfo archive_header="$HDR" item=... `
```

Added new item estlines to return the estimated number of scan lines in the pass. If num_lines=0, estlines will estimate lines based on the duration of the pass.

Added two new items satzenith and sunzenith. Existing items satelev and sunelev cut off at elev=0; they are not allowed to go negative. New zenith items have full 180-degree range: 0 = straight up, 180 = straight down.

For further information, please see the passinfo help page in TeraScan 3.2.

SeaWiFS Processing

New! A software solution for SeaWiFS image decryption is now available from Orbimage. The software is called the OrbView Ground Processor (OGP).

The OGP software has been integrated into rdframe for real-time decryption of SeaWiFS data scheduled for capture as swhrpt telemetry.

A new TeraScan function ogpreplay has been added to decrypt delayed-mode (swcrpt) TeraScan SeaWiFS pass data using the OGP package. For further information, please see the ogpreplay help page in TeraScan 3.2. Note: ogpreplay cannot be used unless the OGP package has been installed.

seawifsin--Optional parameter sort_by_var removed.

TeraPGS

PGS Express--A New! application has been added for running user programs or scripts using the TeraPGS product generation and distribution capabilities. This application consists of custom user scripts and two GUIs:
pgs_express_template--used to define a new user program to the system, with selection of one or more data streams and a dataset or picture output format (HDF, TDF, JPEG, GIF, etc.). (This can also be invoked as pgs_express -t.)

pgs_express--allows users to run the defined program for a single data stream, an area of interest (AOI), and linked to one or more distribution points.
Both GUIs are available from the TeraScan Launchpad. For an overview of this application, see the pgs_express help page in TeraScan 3.2.
Two example custom user scripts are provided and defined in the pgs_express configuration file $PASSDIR/pgscustom/configuration:

pgs_geo_clouds provides cloud top statistics and cloud motion winds from geostationary satellites. Output is TDF for gvar, pdus, and svissr telemetry.

pgs_two_color_ir provides a two-color IR image, with blue scale over water and green over land. Output is JPEG for hrpt, chrpt, rtd, gvar, pdus, svissr, and teradb telemetry/data streams.

The function cloudvec has been modified to more efficiently work with pgs_geo_clouds.

A new script pgs_hrpt_fog has been added. It creates a low cloud image for NOAA passes containing the 3.7 micron AVHRR channel 3, available during the nighttime portion of the pass. For nighttime regions, it creates a low cloud product. For daytime regions, it creates a visible product. Often these appear in conjunction if the pass is on the terminator: a low cloud product on one side of the terminator and a visible product on the other side. To specify this product, select lowcloud from the list of dataset or picture variables.

Note: There must be an avhrr_ch3 in the dataset. If there is an avhrr_ch3a, instead, it will not process the data.

terapgsd now logs in $PASSDIR/pgslog when delivering a product.

The ingest portion of PGS has been redesigned so that all ingesters do HTML logging and generate the same (or better) messages.

pgs_hrptin has been modified to scrub the $DATADIR/overlay/tovs directory as well as $DATADIR/whole_pass/hrpt. The interpretation of the file path specified in the distribution record has been changed. A pathname that begins with a "/" is assumed to be an absolute pathname rather than a relative pathname.

TeraVision Sample Data

SkewT data that can be displayed with data from either the "California" or "Ship Tracks" data types has been added to the TeraVision sample data. For further information on displaying SkewT data in TeraVision, please see the SkewT help page in TeraScan 3.2.

timemerge

New! function timemerge has been added to merge together unregistered pass data received at different sites from the same satellite and sensor. The function merges all the included variables from the input files along their "time" dimension, removing duplicate lines. For further information, please see the timemerge help page in TeraScan 3.2.

tsdiff

tsdiff modified so that user no longer needs to specify an output file name when report_only=yes.

X-band Framing (for systems with an appropriate license)

writsx--modified to frame teradb CADUs "on the fly."

Bugs Fixed

This section summarizes bugs fixed since TeraScan 3.1.2.

emath series functions

Functions fixed so optimizer does not compress temporary arrays that are used trivially; e.g., y1=x, y2=y1, y3=y1.

emathc fixed so it links from library libtcan.a instead of libutils.a, which no longer exists.

embed

embed fixed so wildcards will work when same_name=yes.

FY-1C Processing

chrptin on Linux systems was not able to read from disk file nor could it do thermal calibration. This has been fixed.

geo_simple

geo_simple fixed so short (16-bit) data is no longer fatal. Missing lines are filled with bad_val.

impasc

impasc was unable to import 2D data on Linux systems. This has been fixed.

llgrid

llgrid fixed so that it will output labels for fractional latitude spacing.

manamsched

manamsched fixed to correct end- of- year wrap problem.

setname

setname wasn't working well with subsets. The computed time did not take into account offset from start and/or subsampling. This has been fixed.

Scripting support

A script with an incorrect hard-coded parameter such pass_number or tape_device would cause an infinite loop. This has been fixed.

TeraVision 3D Datasets

The previous version of TeraVision was modified to load 3-dimensional datasets created without dimension variables. An error in the code caused complex data types to show incorrect slices for selection. For example, when you tried to load TOVS wind_vectors data, the slices appeared as 0 or 1 rather than levels such as 100, 200, etc. TeraVision now properly sees the data as complex data.
If you have previously defined such data types, you will need to delete the data type and create a new one.

Known Bugs

This section summarizes the bugs known to be part of TeraScan 3.2. Applicable operating system(s) and whether for "New" install or "Upgrade" are noted in parentheses.

TeraPGS

"Dry Run" option pops up an empty data viewer, which claims there is no data loaded; nothing is listed under "Available Layers" option. When the data viewer is closed using File -> Exit, the viewer has a segmentation fault that crashes TeraPGS. (Linux) (New, Upgrade)

TeraVision

Selecting the end point of a single vector within Survey crashes TeraVision. (Linux) (New, Upgrade)

Tkconfig

The Configuration Editor cannot be used for Linux systems to add a framer, bitsync, receiver, or pass partitions. To add hardware or modify the configuration, you will need to edit the $PASSDIR/system.config file. To add pass partitions, you can edit the $PASSDIR/BAST file. (Linux) (New, Upgrade)

When the "navigation" parameter is set to "moving" in the $PASSDIR/system.config file, the entry in the GUI is incorrectly shown to be "fixed." (All Solaris) (New, Upgrade)

Tlogview

When the user selects File -> Mail, this sends the entire logfile to SeaSpace Customer Support. The user is not currently given the choice of sending the entire file or the displayed list only. (All Solaris, Linux) (New, Upgrade)

Tracksat

Undo: After the map has been re-centered, Undo does not always work. (All Solaris, Linux) (New, Upgrade)

STV

When selecting a live or online pass to view, STV attempts to run the ingester and then crashes. (Linux) (New, Upgrade)

Viewing OCM passes is not currently supported. (Linux) (New, Upgrade)

When an online pass is selected for viewing, STV does not automatically switch to the "watch" mode. (All Solaris, Linux) (New, Upgrade)

Infrared data may not be displayed "inverted." Therefore, clouds may appear black instead of white. (All Solaris, Linux) (New, Upgrade)

Coastlines are improperly superimposed over a zoomed in image. (All Solaris, Linux) (New, Upgrade)

TeraCapCon

Under "Edit Scheduled Pass," the "start time" option seems to stick sporadically when using the "up" arrow to select. (All Solaris, Linux) (New, Upgrade)

For very long passes (greater than 20 minutes), a maximum of 20 lines is displayed in the Show Coverage panel. (All Solaris, Linux) (New, Upgrade)

Miscellaneous

There appear to be inaccuracies with GOES sounder data processed in TeraScan 3.2. The GOES sounder processing software is currently under revision. No release date is available at this time. (All Solaris, Linux) (New, Upgrade)