The FourCurveAtmosphere Plugin

This atmosphere plugin provides a low fidelity, physics-driven atmospheric modeling capability in DIRSIG5.

Summary

Originally developed as a companion to the ChipMaker plugin, the FourCurveAtmosphere model was designed to provide fidelity somewhere between a fully analytical approximation of the atmosphere and physics-driven representation produced by atmospheric radiative transfer models (e.g. MODTRAN, 6S, etc.). The specific need with ChipMaker simulations was that the overhead of computing a full physics-driven atmosphere model for a given solar and sensor geometry, atmospheric condition, etc. would dwarf the radiometric image calculation for smaller image chips. Hence, an approach to leveraged a pre-computed database was desired.

The general assumption with this model is that the sensor is generally overhead and looking down at the world.

Simulations requiring a full model of the atmosphere should use the NewAtmosphere plugin. However, if that level of fidelity is not required, then this plugin serves as an alternative that requires less computation to be spent on atmospheric calculations.

This capability is currently divided into two major components:

The fourcurve_builder tool, which interfaces with the externally developed physics-based atmospheric models to construct an HDF5 database of atmospheric data tables.
- A detailed interface control document (ICD) for the atmospheric database can be found here.
The FourCurveAtmosphere plugin, which uses the HDF atmospheric database to drive the atmospheric contributions in a DIRSIG5 execution.

Approach

This plugin models the atmosphere using four spectral curves, which are a function of the solar geometry angle and sensor view geometry. The four curves are:

Ground-reaching solar irradiance [watts / (cm² micron)]
Path radiance per meter [watts / (cm² sr meter micron)]
Path extinction per meter [1 / meter]
Hemispherical sky irradiance [watts / (cm² micron)]

For each solar declination angle, the database contains a series of four curve "sets" that are a function of the sensor declination angle and relative azimuth angle between the solar and sensor path.

In the original version of this model, the path transmission and radiance were not a function of the sensor view geometry. Hence, the model was simply 4 curves for each solar geometry. The next iteration of the model added the view geometry dependence, which makes the number of spectral curves for each solar geometry much greater than 4. However, the "four curve" model name was retained.

The tables are computed for a sensor at a nominal altitude (see the nominal_altitude variable in the database configuration).

The spectral path radiance and path extinction for a request solar and sensor path geometry is then length scaled using a uniform optical depth scaling approach.

The scaling is never extrapolated more than 110% of the path length used to generate the table data. If your sensor altitude exceeds the nominal database altitude by more than 110%, then you should generate a new database.

Data Model

The database for the model is stored in a Hierarchical Data Format (HDF) file. The data model for this plugin is fairly straightforward. A single HDF database can contain one or more atmospheric conditions (conditions). A set of conditions captures a specific combination of atmospheric state variables, including air temperature, air pressure, humidity, aerosol type and aerosol densities distributions within the atmosphere. For example, in terms of a MODTRAN driven atmospheric condition, the state might be the atmosphere model, aerosol model aerosol visibility supplied to MODTRAN. All the conditions in the database share:

A set of spectral samples that define the wavelengths at which the spectral curves are sampled,
A set of solar declination samples that define the solar declination angles that are sampled,
A set of path declination samples that define the angles the sensor declination angles that are sampled, and
A set of relative azimuth samples that define the relative angles between the sensor view and solar illumination geometry.

Each conditions in the database has a distinct and non-null name. For each condition, there is a set of solar declination angles. Within each solar declination angle, there is the angularly constant data (ground reaching solar irradiance, ground reaching sky irradiance and path extinction) and the angularly sample data (the path radiance).

Database
+- SpectralSamples
+- SolarDeclinationAngles
+- PathDeclinationAngles
+- RelativeAzimuthAngles
+- Conditions
   +- mls_urban_5k
      +- SolarDeclination0
         +- AngularConstantData
         +- PathRadiance
      +- SolarDeclination1
         +- AngularConstantData
         +- PathRadiance
         ...
      +- SolarDeclination1
         +- AngularConstantData
         +- PathRadiance
   +- mls_urban_23k
      +- SolarDeclination0
         +- AngularConstantData
         +- PathRadiance
      +- SolarDeclination1
         +- AngularConstantData
         +- PathRadiance
         ...
      +- SolarDeclination1
         +- AngularConstantData
         +- PathRadiance

The interface control document for this file can be found here.

Data Usage

The plugin that uses the database performs operations using the following general logic:

During initialization, the user-requested conditions are loaded
During a state change (e.g. the start of a capture):
1. The ephemeris plugin is used to figure out the current solar geometry.
2. The AngularlyConstantData and PathRadiance tables are loaded from the SolarDeclination group that is the closest, nearest neighbor match to the current solar declination angles.
3. The loaded tables are spectrally resampled to match the current spectral state (e.g. wavelengths being modeled by the current sensor).
During the radiometry calculations, atmospheric requests are handled as follows:
1. When the solar irradiance is requested, the loaded solar irradiance data in the AngularlyConstantData table is used.
2. When the sky irradiance is requested, the loaded sky irradiance curve data in the AngularlyConstantData table is used.
3. When a path transmission is requested, the loaded path extinction data in the AngularlyConstantData table is used with the length of the path.
4. When a path radiance is requested, the path radiance table is bi-linearly interpolated in the zenith and azimuth dimensions using the sensor view declination angle, relative azimuth angle to the sun and the length of the path from the PathRadiance table.

Warning and Error Conditions

The plugin currently supports the following warning conditions

If the altitude of the sensor exceeds the nominal altitude of the database by more than 120%, a warning is issued.

The plugin currently supports the following error conditions:

If the spectral state does not overlap the spectral coverage (e.g. SpectralSamples) of the database, an error is reported.

Plugin Input

The inputs to the plugin is a JSON object that is embedded directly into the JSIM file where the plugin is requested (via the inputs section). Below is a example of the plugin configuration:

Example setup for the FourCurveAtm plugin.

  ...
  "plugin_list": [
    ...
    {
      "name": "FourCurveAtmosphere",
      "inputs": {
        "database": "./user_defined_atm.hdf",
        "conditions": "mls_rural_10km",
        "remove_sensor_path": false
      }
    },
    ...
  ]
  ...
}

database (optional)

This variable points to an existing atmospheric database file to be used. When not provided, the default database is used (see $DIRSIG_HOME/lib/data/atm/fourcurve_vis_nir_swir_10km.hdf). If a the provided filename does not use absolute or relative path, the file is searched for in the following locations:

The $DIRSIG_HOME/lib/data/atm folder, then
The users Documents/DIRSIG/atmospheres folder.

Any user generated databases can be stored in the user’s Documents/DIRSIG/atmospheres folder. This avoids needing to hard code the absolute path to these alternative database files.

conditions (required): This variable contains the name of the atmospheric conditions inside of the database to use for defining the curves. Only one set of conditions can be used in a single simulation.

remove_sensor_path (optional): This boolean controls if the sensor path (the atmospheric path between the sensor and the first thing intersected in the scene(s)) should be "removed" during the simulation. Removal means setting the path transmission to 1.0 and the path radiance to 0.0 for all wavelengths. This effectively converts the conventional sensor reaching radiance to a surface leaving radiance. This option is provided as a means to emulate the output of some atmospheric compensation algorithms.

Databases

The plugin will look for databases using the following search logic:

A database with an absolute or relative path (e.g., it starts the /, ./, ../, C:\, etc.), is looked for using that path.
Otherwise the requested database file is looked for:
- In the lib/data/atm folder inside the DIRSIG installation, then
- In the user’s Documents/DIRSIG/atm folder.

It is encouraged to use the user’s folder to store alternative database files since files in this folder will not be updated or replaced by a DIRSIG software update.

Default Database

Each DIRSIG release includes a default database that contains a variety of conditions. This database is stored in $DIRSIG_HOME/lib/data/atm and a more detailed README file is located there with details about the database.

The default database contains 30 separate atmospheric conditions, encompassing the permutations of the following three main variables:

Profiles: Tropical, Mid-Latitude Summer (MLS) and Mid-Latitude Winter (MLW)
Aerosols: Rural and Urban aerosols
Visibilities: 5, 10, 15, 30 and 50 km

The corresponding condition names are:

Mid-Latitude Summer (MLS,) Rural aerosols
- mls_rural_5km
- mls_rural_10km
- mls_rural_15km
- mls_rural_30km
- mls_rural_50km
Mid-Latitude Summer (MLS,) Urban aerosols
- mls_urban_5km
- mls_urban_10km
- mls_urban_15km
- mls_urban_30km
- mls_urban_50km
Mid-Latitude Winter (MLW), Rural aerosols
- mlw_rural_5km
- mlw_rural_10km
- mlw_rural_15km
- mlw_rural_30km
- mlw_rural_50km
Mid-Latitude Winter (MLW), Urban aerosols
- mlw_urban_5km
- mlw_urban_10km
- mlw_urban_15km
- mlw_urban_30km
- mlw_urban_50km
Tropical, Rural aerosols
- trop_rural_5km
- trop_rural_10km
- trop_rural_15km
- trop_rural_30km
- trop_rural_50km
Tropical, Urban aerosols
- trop_urban_5km
- trop_urban_10km
- trop_urban_15km
- trop_urban_30km
- trop_urban_50km

The spectral coverage of the default database is 0.350 → 2.550 microns at 0.001 micron sampling.

The angular sampling of the default database is as follows:

Solar declination is sampled from 0 → 80 degrees at 10 degree sampling.
Path declination is sampled at 0, 10, 20, 30, 40, 60 and 80 degrees.
Relative azimuth between the sun and the view path is sampled from 0 → 180 degrees at 45 degree sampling.

The nominal altitude of the default database is 10 km.

Below is an example .jsim input using the mls_rural_50km condition of the default database:

Using the default database with the Mid-latitude Summer (MLS) and rural aerosols with 50 km visibility conditions.

  ...
  "plugin_list": [
    ...
    {
      "name": "FourCurveAtmosphere",
      "inputs": {
        "conditions": "mls_rural_50km"
      }
    },
    ...
  ]
  ...
}

Generating a Database

Included in the DIRSIG distribution is a program called fourcurve_builder that can be used to generate HDF databases. The reasons you might consider building an alternative to the default database include:

Add a condition that doesn’t exist in the default database.
Change the nominal altitude used in the default database.
Change the spectral coverage used in the default database.

The input for the fourcurve_builder program is a JSON object that can either be given in an input file via the -i or --input flags or through the standard input stream. An example of the JSON object is given below:

Example database builder input file using a MODTRAN user profile and ModtranTape backend with external tape5 files.

{
  "bandpass" : {
    "units"   : "microns",
    "minimum" : 0.380,
    "maximum" : 2.520,
    "delta"   : 0.001
  },
  "backend" : {
    "name": "modtran_tape",
    "modtran_profile": "MODTRAN6"
  },
  "conditions": [
    {
      "name": "mls_rural_5km",
      "backend_input": {
        "tape5_filename": "tp5/mls_rural_5km.tp5"
      }
    },
    {
      "name": "mls_rural_10km",
      "backend_input": {
        "tape5_filename": "tp5/mls_rural_10km.tp5"
      }
    },
    {
      "name": "mls_rural_25km",
      "backend_input": {
        "tape5_filename": "tp5/mls_rural_25km.tp5"
      }
    }
  ],
  "nominal_altitude" : 10000,
  "solar_declination_samples" : [ 0, 10, 20, 30, 40, 50, 60, 70, 80 ],
  "path_declination_samples" : [ 0, 20, 40, 60, 80 ],
  "relative_azimuth_samples" : [ 0, 45, 90, 135, 180 ]
}

The bandpass defines the spectral range and sampling that the database will be created for. The backend configuration is identical to those used with the NewAtmosphere plugin, and specifies what backend model to use to generate the curves. The conditions array defines the set of atmospheric conditions that will be available in the database. When using the MODTRAN tape backend, the backend_input for each condition is the name of the MODTRAN tape5 file that describes the atmosphere for that condition.

Since HDF does not allow slashes in data set or group names, any slashes (/) in the condition names are replaced by colons (:).

The example file below uses some advanced features in the MODTRAN backend including:

Use of environment variables for the paths to the MODTRAN executable and DATA folder (see modtran_exe and modtran_data).
Use of parametric configurations for each MODTRAN conditions.

Example database builder input file using a MODTRAN environment variables and the ModtranTape backend with parameterized inputs.

{
  "backend" : {
    "name" : "modtran_tape",
    "modtran_exe" : "${MODTRAN_EXE}",
    "modtran_data" : "${MODTRAN_DATA}",
    "modtran_min_delta" : 0.1
  },
  "bandpass" : {
    "units"   : "microns",
    "minimum" : 0.350,
    "maximum" : 2.550,
    "delta"   : 0.001
  },
  "path_declination_samples" : [ 0, 10, 20, 30, 40, 60, 80 ],
  "relative_azimuth_samples" : [ 0, 45, 90, 135, 180 ],
  "conditions": [
    {
      "name": "trop_rural_5km",
      "backend_input": {
        "tape5_parameters": {
          "atmospheric_model": "Tropical",
          "co2_mixing_ratio_ppm": 410,
          "boundary_aerosol_model": {
            "type": "Rural",
            "parameters": {
              "visibility_km": 5.0
            }
          },
          "multiple_scattering": {
            "type": "Disort",
            "parameters": {
              "streams": 32
            }
          }
        }
      }
    },
    {
      "name": "trop_rural_10km",
      ...
    }
  ]
}

Configuration Options

The following variables are optional in the input JSON:

nominal_altitude: This is the nominal altitude in meters used for the path transmission and radiance calculations. The default value is 10,000 meters (10 km).

Prefer the command-line --nominal_altitude option at generation time to hard coding the altitude in the configuration. This allows you to use the configuration for multiple altitudes.

solar_declination_samples: This is an array of the solar declination angle samples in degrees. The default angles are 0 → 80 degrees at 10 degree sampling.
path_declination_samples: This is an array of the path declination angle samples in degrees. The default angles are 0 → 80 degrees at 10 degree sampling.
relative_azimuth_samples: This is an array of the relative azimuth angle samples between the solar and sensor path in degrees. The default angles are 0 → 180 degrees at 10 degree sampling. Due to assumed symmetry, there is no need to sample 0 → 360 degrees.

Command-Line Options

For more details, see the command-line help for fourcurve_builder.

Command-line help for the fourcurve_builder tool.

$ fourcurve_builder -h
Usage: fourcurve_builder [options]

Builder for FourCurveAtmosphere HDF databases.

Options:
  -h/--help
	Display this help and exit.
  -v/--version
	Display build and version info and exit.
  --log_level string
	Sets the minimum logging level (debug, info, warning, error, critical, off). Defaults to info.
  -i/--input string
	The input filename (.json), defaults to reading from stdin.
  -o/--output string
	The output filename (.hdf), defaults to 'atm.hdf'.
  -j/--max_processes int
	The maximum number of backend atm model processes to launch.
  -s/--save_files
	Save the backend atm model input/output files.
  -n/--name string
	The name to give to the database.
  -a/--author string
	The author name to embed in the database.
  --nominal_altitude float
	Nominal altitude (meters)
  --exec string
	Execution mode to use. (local, mpi)

Demos

The FourCurveAtm1 demo contains a simple demonstration of how to use both the plugin and fourcurve_builder. It contains the above example of fourcurve_builder JSON in the atm.json file and produces a video of a simple scene over the course of a day to show how the atmosphere varies as a function of solar declination angle.