IMAPP Processing Example

Step [1]:

For a level 0 PDS file, use the script pgs_modis_calib to create a TeraScan TDF dataset that contains 36 calibrated and earth-located MODIS channels. The processing includes geolocation, calibration, and conversion of separate HDF files for 250m, 500m, and 1000m resolution to one TDF file.

Geolocation and calibration are done scan by scan and are reported for the 1000m lines (e.g., scan_number: 498 is reporting for line 4990, the number of 1000m lines in our example dataset).

When the calibrated HDF files are converted to TDF, the scales and offsets are used to convert the data from scaled integer to radiance units.

To run pgs_modis_calib, change to the IMAPPv1.2 directory. The TDF will be output to the directory defined as $DATADIR in the script.

scrum% pgs_modis_calib /level0/Terra/t1.00306.1622.pds
.
.
.
pgs_modis_calib.csh: Starting Level0 to Level1a processing Mon Apr 16 21:57:38 GMT 2001...
Packet with invalid length found, processing will stop at:  2000-11-01T16:34:56.152920Z
Start time:  2000-11-01T16:22:39.077758Z 
Stop time:   2000-11-01T16:34:56.152920Z 
time length: 737.075162 seconds 
-------------------------------------
level1a_file: zzz.pds
pgs_modis_calib: Successful Level0 to Level1a conversion Mon Apr 16 22:08:25 GMT 2001...
pgs_modis_calib: Starting Geolocation Mon Apr 16 22:08:25 GMT 2001...
scan_number: 0
scan_number: 1
scan_number: 2
scan_number: 3
.
.
.
 
pgs_modis_calib: Starting Calibration Mon Apr 16 22:34:21 GMT 2001..
scan_number:  0
scan_number:  1
scan_number:  2
scan_number:  3
.
.
.
 
pgs_modis_calib: Starting HDF to TDF conversion Tue Apr 17 00:38:31 GMT 2001..
 
[Converting 250m reflective channels:]
 
1 0.0275656 316.972
2 0.0105175 316.972
 
[Converting 500m reflective channels:]
 
1 0.0201061 316.972
2 0.0177292 316.972
3 0.00382121 316.972
4 0.00238923 316.972
5 0.000691114 316.972
 
[Converting 1km reflective channels:]
 
1 0.0069692 316.972
2 0.00463601 316.972
.
.
.
14 0.00664826 316.972
15 0.00268764 316.972
 
[Converting 1km emissive channels:]
 
1 6.2624e-05 2730.58
2 0.00314951 2730.58
.
.
.
15 0.00016187 2730.58
16 0.000108402 2730.58
Successful L0 -> L1B conversion.

Now, change to the directory that contains the output TDF and look at the contents of the dataset. Note that channels 13 and 14 have both high and low channels due to their SNR (signal-to-noise ratio) requirements.

Note the different scales and offsets. This enables the 250m, 500m, and 1000m channels, with different dimensions, to be stored in the same dataset using the same earth location.

Note also at the end of the contents the information for each of the channels, including radiance/reflectance scales and offset from the calibration.

scrum% contents t1.00306.1622.RADIANCE_tdf

Contents of File: t1.00306.1622.RADIANCE_tdf    Page 1

Dimension         Size    Coord         Scale      Offset    Variable
 250M_line        19960    y             0.25      -0.375
 250M_sample       5416    x             0.25      -0.375
 500M_line         9980    y              0.5       -0.25
 500M_sample       2708    x              0.5       -0.25
 1000M_line        4990    y                1           0
 1000M_sample      1354    x                1           0
 .
 .
 .

Variable        Type            Units
 modis_ch01      short           W/(m^2*micrometer*steradian)
 modis_ch02      short           W/(m^2*micrometer*steradian)
 modis_ch03      short           W/(m^2*micrometer*steradian)
 modis_ch04      short           W/(m^2*micrometer*steradian)
 .
 .
 .
 modis_ch13L     short           W/(m^2*micrometer*steradian)
 modis_ch13H     short           W/(m^2*micrometer*steradian)
 modis_ch14L     short           W/(m^2*micrometer*steradian)
 modis_ch14H     short           W/(m^2*micrometer*steradian)
 .
 .
 .
 modis_ch35      short           W/(m^2*micrometer*steradian)
 modis_ch36      short           W/(m^2*micrometer*steradian)

Variable        Dimension       Size
 modis_ch01      250M_line       19960
 modis_ch01      250M_sample     5416
 modis_ch02      250M_line       19960
 modis_ch02      250M_sample     5416
 modis_ch03      500M_line       9980
 modis_ch03      500M_sample     2708
 modis_ch04      500M_line       9980
 modis_ch04      500M_sample     2708
 modis_ch05      500M_line       9980
 modis_ch05      500M_sample     2708
 modis_ch06      500M_line       9980
 modis_ch06      500M_sample     2708
 modis_ch07      500M_line       9980
 modis_ch07      500M_sample     2708
 modis_ch08      1000M_line      4990
 modis_ch08      1000M_sample    1354
 .
 .
 .
 modis_ch13L     1000M_line      4990
 modis_ch13L     1000M_sample    1354
 modis_ch13H     1000M_line      4990
 modis_ch13H     1000M_sample    1354
 modis_ch14L     1000M_line      4990
 modis_ch14L     1000M_sample    1354
 modis_ch14H     1000M_line      4990
 modis_ch14H     1000M_sample    1354
 modis_ch15      1000M_line      4990
 modis_ch15      1000M_sample    1354
 .
 .
 .

Variable            BadValue    ValidMin    ValidMax       Scale      Offset
 modis_ch01           -32768      -32768       32767           1           0
 modis_ch02           -32768      -32768       32767           1           0
 modis_ch03           -32768      -32768       32767           1           0
 modis_ch04           -32768      -32768       32767           1           0
 .
 .
 .

Variable     Attribute       Type      Units   Value
 modis_ch01   long_name       byte              Earth View 250M Reflective Solar Bands Scaled Integer
 modis_ch01   band_names      byte              1,2
 modis_ch01   radiance_scales float             0.0273799 0.0112384
 modis_ch01   radiance_offsets float             316.972 316.972
 modis_ch01   radiance_units  byte              Watts/m^2/micrometer/steradian
 modis_ch01   reflectance_scales float             5.27148e-05 3.50265e-05
 modis_ch01   reflectance_offsets float             316.972 316.972
 modis_ch01   reflectance_units byte              none
 modis_ch01   corrected_counts_scales float             0.126194 0.126194
 modis_ch01   corrected_counts_offsets floa              316.972 316.972
 modis_ch01   corrected_counts_units byte             counts
 .
 .
 .

Step [2]:

Use modis_bt to create a dataset containing calibrated channels 1 through 16 corrected for the bowtie effect of the MODIS sensor (channels 1, 3, and 4 are useful for RGB images; channels 8, 9, 10, 11, 12, 13H, 15, and 16 are ocean color channels). Note the change of variable names to indicate the data is bowtie corrected.

scrum% modis_bt
in/out files   : char(255) ? t1.00306.1622.RADIANCE_tdf t1.00306.1622.bt
channels       : int ( 36) ? [3 7 10 14 17 21 25 28 32 35]1^16

>*> modis_ch01b: Nl 19960, Ns 5416
   250m Channel: 01(00) - Line    0
   250m Channel: 01(00) - Line  200
   250m Channel: 01(00) - Line  400
   250m Channel: 01(00) - Line  600

>*> modis_ch03b: Nl 9980, Ns 2708
   500m Channel: 03(00) - Line    0
   500m Channel: 03(00) - Line  100
   500m Channel: 03(00) - Line  200
   500m Channel: 03(00) - Line  300

>*> modis_ch08b: Nl 4990, Ns 1354
   1000m Channel: 08(00) - Line    0
   1000m Channel: 08(00) - Line  100
   1000m Channel: 08(00) - Line  200
   1000m Channel: 08(00) - Line  300
.
.
.
modis_bt: Finished bowtie Correction, closing files...

Click here for an example of an image before and after bowtie correction.

Now, look at the contents of the bowtie-corrected dataset. Note the scale and offset.

scrum% contents t1.00306.1622.RADIANCE_bt

Contents of File: t1.00306.1622.RADIANCE_bt    Page 1

Dimension         Size    Coord         Scale      Offset    Variable
 1000M_line        4990    y                1           0
 1000M_sample      1354    x                1           0
 500M_line         9980    y              0.5       -0.25
 500M_sample       2708    x              0.5       -0.25
 250M_line        19960    y             0.25      -0.375
 250M_sample       5416    x             0.25      -0.375 
 .
 .
 .

Variable        Type            Units
 modis_ch01b     short           W/(m^2*steradian*micrometer)
 modis_ch02b     short           W/(m^2*steradian*micrometer)
 modis_ch03b     short           W/(m^2*steradian*micrometer)
 modis_ch04b     short           W/(m^2*steradian*micrometer)
 modis_ch05b     short           W/(m^2*steradian*micrometer)
 modis_ch06b     short           W/(m^2*steradian*micrometer)
 modis_ch07b     short           W/(m^2*steradian*micrometer)
 modis_ch08b     short           W/(m^2*steradian*micrometer)
 modis_ch09b     short           W/(m^2*steradian*micrometer)
 modis_ch10b     short           W/(m^2*steradian*micrometer)
 modis_ch11b     short           W/(m^2*steradian*micrometer)
 modis_ch12b     short           W/(m^2*steradian*micrometer)
 modis_ch13Lb    short           W/(m^2*steradian*micrometer)
 modis_ch13Hb    short           W/(m^2*steradian*micrometer)
 modis_ch14Lb    short           W/(m^2*steradian*micrometer)
 modis_ch14Hb    short           W/(m^2*steradian*micrometer)
 modis_ch15b     short           W/(m^2*steradian*micrometer)
 modis_ch16b     short           W/(m^2*steradian*micrometer)

Data processed with modisin tends to be well navigated because modisin takes into consideration attitude information. However, data processed with IMAPP is not as well navigated and usually needs to be adjusted for time.

Navigate the data using navbox2 and nav2. (Remember to use pitch and yaw on the nav2 command line.)

If the dataset you are navigating includes the 250m channels, use channel 2 for navigation. It has the best land/water interface. (If your data does not include channel 2, use channel 16, a near IR channel, instead.)

Note: If your data cannot be navigated automatically, click here for details of manual navigation.

scrum% navbox2
in/out files   : char(255) ? t1.00306.1622.bt boxes
variable       : char( 31) ? [] modis_ch02b
coast_file     : char(255) ? [wdb2.cil] dcw.coast
box_height     : int       ? [100] 50
box_width      : int       ? [50] 50
min_cross_dev  : real      ? [2.5]
min_feature    : int       ? [15]
max_regions    : int       ? [4]
min_region_perc: int       ? [10]
min_coast_pts  : int       ? [50]
min_sat_elev   : real      ? [0]
min_good_boxes : int       ? [1]
rad.tdf: Looking for boxes in the LS area [1 4990] x [1 1354]
rad.tdf: 692 boxes selected

scrum% nav2 use_pitch=yes use_yaw=yes
input file     : char(255) ? t1.00306.1622.RADIANCE_tdf
expr_vars      : char(255) ? modis_ch02b
expression     : char(255) ? x1
box_file       : char(255) ? boxes
coast_file     : char(255) ? [wdb2.cil] dcw.coast
max_line_shift : int       ? [10] 50
max_samp_shift : int       ? [10] 50
min_good_boxes : int       ? [1]
fix_attitude   : char(  3) ? [yes]
reset_sat      : char(  3) ? [yes] n
method         : char( 19) ? [correlate] corr edge
min_coast_pts  : int       ? [50]
min_edge_diff  : real      ? 1
min_edge_perc  : real      ? [95]
min_corr       : real      ? [95]
interp_corr    : char(  3) ? [yes]
min_feature    : int       ? [5]
fixed_thresh   : char(  3) ? [no]
stdev_units    : real      ? [2]
num_bins       : int       ? [100]
estimate_range : char(  3) ? [yes]

t1.00306.1622.RADIANCE_tdf: Old Attitude: (0.00 0.00 0.00), Time Adj: 0.00
.
.
.
Box 688:  [19806,19855] x [5109,5158]   84.0% edge fits
Box 689:  [19806,19855] x [5109,5158]  Standard dev units 1.75868
Box 689:  [19806,19855] x [5134,5183]  Standard dev units 1.6191
Box 690:  [19831,19880] x [4859,4908]   86.1% edge fits
Box 690:  [19831,19880] x [4859,4908]  Standard dev units 1.72074
Box 691:  [19880,19930] x [1684,1733]   80.6% edge fits
Box 691:  [19880,19930] x [1684,1733]  Standard dev units 1.57484
Box 692:  [19906,19955] x [5159,5208]   87.0% edge fits
Box 692:  [19906,19955] x [5159,5208]  Best match at max shift

t1.00306.1622.RADIANCE_tdf: 90 out of 692 boxes used ...
Mean Square Error: Old (32.47 5.67), New (2.47 4.33)

t1.00306.1622.RADIANCE_tdf: Roll/Pitch/Yaw : (-0.01 -0.08 0.11)
t1.00306.1622.RADIANCE_tdf: Time Adjustment: -1.30

Step [3]:

OPTIONAL. Skip to Step 4 if processing time and disk space are not a consideration.

Instead of processing the entire dataset, you might want to continue processing only a portion of the data. The example creates a master file (satmaster) with a starting time several minutes after the start time of the pass and then uses that master to subset the data (mastersub).

mastersub is useful when products for several areas are to be generated from a given pass.

Display the input dataset and determine the desired starting latitude of the data (we've chosen a latitude of 37 N). Then, use the function ephem to display a list of time steps of the pass and select the time step that correlates most closely with the desired starting latitude. (pcap_time is the start time of the pass.)

scrum% ephem
receiver_lat   : char( 15) ? [32 59.10 N] 43.07 N
receiver_lon   : char( 15) ? [117  0.84 W] 89W
satellite      : char( 10) ? terra-1
sensor         : char(  5) ? [modis]
pass_date      : char( 15) ? [2001/05/01] 2000/11/01
pcap_time      : char( 15) ? 16:22:39
time_step      : int       ? [30]
printout       : char(  3) ? [no]
terra-1 modis  Ephemeris  Receiver: 43  4.20 N,  89  0.00 W   Page 1

Date      Time   Elevation  Azimuth  Latitude    Longitude   Radius

2000/11/01  16:22:00    0.36   21.69  65 23.91 N   66 31.47 W  7075.39
2000/11/01  16:22:30    2.17   23.16  63 41.07 N   68  2.35 W  7075.57
2000/11/01  16:23:00    4.10   24.82  61 57.49 N   69 23.70 W  7075.77
.
.
.
2000/11/01  16:29:00   34.87  104.91  40 43.82 N   79 22.78 W  7078.64
2000/11/01  16:29:30   33.01  118.14  38 56.24 N   79 56.52 W  7078.91
2000/11/01  16:30:00   29.94  129.47  37  8.52 N   80 28.97 W  7079.19
2000/11/01  16:30:30   26.33  138.63  35 20.69 N   81  0.27 W  7079.46
.
.
.

Now, use satmaster to create a master file based on the earth location of the pass we are using, with a start time later than the actual start of the pass (16:22:39 --> 16:30:00).

The scan lines for the ocean color channels (1000m channels) are 1354 samples wide. Estimate the number of 1000m lines and accept the default num_samples.

View the output master file. If the master does not cover the desired area, determine the correct number of lines to use and rerun satmaster as necessary.

scrum% satmaster
output file    : char(255) ? [Master] SATM
satellite      : char( 11) ? terra-1
sensor         : char(  5) ? [modis]
geo_correct    : char(  3) ? [no]
pass_date      : char( 15) ? [2001/04/30] 2000/11/01
start_time     : char( 15) ? 16:30:00
delta_line     : int       ? [1]
num_lines      : int       ? 1800
delta_sample   : int       ? [1]
start_sample   : int       ? [1]
num_samples    : int       ? [1354]

Use mastersub to subset the data using the master you just created. The data will be kept in sensor scan. Instantiate the data to copy the variable data from the input dataset to the output dataset.

scrum% mastersub
in/out files   : char(255) ? t1.00306.1622.bt t1.00306.1622.msub
include_vars   : char(255) ? [] 
master_file    : char(255) ? [Master] SATM
instantiate    : char(  3) ? [no] y
t1.00306.1622.msub: range 11893 - 19128 250M_line
t1.00306.1622.msub: range 5947 - 9564 50M_line
t1.00306.1622.msub: range 2974 - 4782 1000M_line
t1.00306.1622.msub: range    1 - 5416 250M_sample
t1.00306.1622.msub: range    1 - 2708 500M_sample
t1.00306.1622.msub: range    1 - 1354 1000M_sample

Step [4]:

Now, use the script get_modis_color.csh to extract the ocean color channels from the calibrated data and rename the channels (e.g., from modis_ch08b to ch1, etc.) for use by the ocean color function modis_color.

scrum% get_modis_color.csh t1.00306.1622.msub
.
.
.
varname old_var_name=modis_ch08b new_var_name=ch1 t1.00306.1622.clr_input
varname old_var_name=modis_ch09b new_var_name=ch2 t1.00306.1622.clr_input
varname old_var_name=modis_ch10b new_var_name=ch3 t1.00306.1622.clr_input
varname old_var_name=modis_ch11b new_var_name=ch4 t1.00306.1622.clr_input
varname old_var_name=modis_ch12b new_var_name=ch5 t1.00306.1622.clr_input
varname old_var_name=modis_ch13Hb new_var_name=ch6 t1.00306.1622.clr_input
varname old_var_name=modis_ch15b new_var_name=ch7 t1.00306.1622.clr_input
varname old_var_name=modis_ch16b new_var_name=ch8 t1.00306.1622.clr_input
.
.
.
t1.00306.1622.clr_input
Successful created color file.

Use the ocean color function modis_color to derive the water-leaving radiances that will be used to create a chlorophyll product.

The example uses the default climatology file.

scrum% modis_color
in/out files   : char(255) ? t1.00306.1622.clr_input t1.00306.1622.clr
climatology    : char(  3) ? [yes]
met_file       : char(255) ? [ecmwfclim.tdf]
interp_met     : char(  3) ? [yes]
delta_x        : int       ? [8]
delta_mod      : int       ? [48]
stray_lit      : char(  3) ? [no]
out_of_band    : char(  3) ? [no]
compute_ndvi   : char(  3) ? [no]
fill_bad       : char(  3) ? [no]

Processing:t1.00306.1622.clr_input 16:22:39.078 2000/11/01: Lines: 1809
Samples: 1354
Line    0
Line  100
Line  200
Line  300
Line  400
Line  500
Line  600
Line  700
.
.
.

Look at the contents of the ocean color dataset. Note the units, scale, and offset.

scrum% contents t1.00306.1622.clr

Contents of File: t1.00306.1622.clr    Page 1

Variable        Type            Units
 nLw_412         short           mW/(cm^2*steradian*micrometer)
 nLw_443         short           mW/(cm^2*steradian*micrometer)
 nLw_490         short           mW/(cm^2*steradian*micrometer)
 nLw_530         short           mW/(cm^2*steradian*micrometer)
 nLw_550         short           mW/(cm^2*steradian*micrometer)
 La_670          short           mW/(cm^2*steradian*micrometer)
 La_865          short           mW/(cm^2*steradian*micrometer)

Variable            BadValue    ValidMin    ValidMax       Scale      Offset
 nLw_412              -32768      -32768       32767       0.001           0
 nLw_443              -32768      -32768       32767       0.001           0
 nLw_490              -32768      -32768       32767       0.001           0
 nLw_530              -32768      -32768       32767       0.001           0
 nLw_550              -32768      -32768       32767       0.001           0

Use the function mod_chlor to derive chlorophyll from the water-leaving radiances.

scrum% mod_chlor
in/out files : char(255) ? t1.00306.1622.clr t1.00306.1622.CHL
Processing line 0
Processing line 100
Processing line 200
Processing line 300
Processing line 400
.
.
.
Processing line 1600
Processing line 1700
Processing line 1800

Look at the contents of the output dataset and note the scale, and offset.

scrum% contents t1.00306.1622.CHL

Contents of File: t1.00306.1622.CHL    Page 1

Variable        Type            Units
 Improved_chl    short            mg m^3        

Variable        Dimension       Size
 Improved_chl    line            1800
 Improved_chl    sample          1354

Variable            BadValue    ValidMin    ValidMax       Scale      Offset
 Improved_chl         -32768      -32768       32767      0.0001           0

Step [5]:

Register the data to a map and projection. First, create the area of interest using master and then remap the data using fastreg. The image will be oriented exactly as the master, and will have exactly the same size as the master [see "Using a Master at Registration"].

scrum% master
output file    : char(255) ? [Master]
projection     : char( 13) ? rec
center_lat     : char( 15) ? 28 N
center_lon     : char( 15) ? 81 W
num_lines      : int       ? 1000
num_samples    : int       ? 1000
pixel_width    : real      ? [1.1132]
pixel_height   : real      ? [1.1132]
rotate_angle   : real      ? [0]
move_center    : char(  3) ? [no]

scrum% fastreg
in/out files   : char(255) ? t1.00306.1622.CHL t1.00306.1622.CHL.r
master_file    : char(255) ? [Master] 
include_vars   : char(255) ? []
poly_size      : real      ? [100]
t1.00306.1622.clr.r: Improved_chl:  [   1,  125] X [   1,  500]
t1.00306.1622.clr.r: Improved_chl:  [   1,  125] X [ 501, 1000]
t1.00306.1622.clr.r: Improved_chl:  [ 126,  250] X [   1,  500]
t1.00306.1622.clr.r: Improved_chl:  [ 126,  250] X [ 501, 1000]
.
.
.

Mask out areas of land in the chlorophyll image. Use lsmask to create the landmask and write it as a variable to the master file and emath2 to apply the landmask to the chlorophyll variable.

scrum% lsmask
master_file    : char(255) ? [Master]
mask_database  : char(255) ? [landmask.wdb2]
master_var     : char(255) ? [master]
mask_var       : char(255) ? [lsmask]
Master master lat/lon: [23.00 33.00] x [-86.662 -75.34]
landmask.wdb2: loading database ...
Master: writing lsmask ...

scrum % emath2
in/out files   : char(255) ? Master t1.00306.1622.CHL.r t1.00306.1622.CHL.m
file1_vars     : char(255) ? [] lsmask
file2_vars     : char(255) ? [] Improved_chl
expression     : char(255) ? x1 == 0 ? x2 : badval  
var_name       : char( 31) ? chl_msk
var_units      : char( 31) ? []
var_type       : char( 15) ? [float]


scrum% contents t1.00306.1622.CHL.m

Contents of File: t1.00306.1622.CHL.m    Page 1