The atm_builder tool constructs the atmospheric databases that are utilized by the DIRSIG5 NewAtmosphere plugin.

Summary

The atm_builder tool and NewAtmosphere plugin were created to provide a new foundation for supporting for physics-based atmospheric models in DIRSIG5. The NewAtmosphere and atm_builder toolchain attempt to address a number of limitations of the DIRSIG4-era make_adb and "classic" atmosphere model combination in a variety of ways.

The figure below attempts to illustrate the relationship between NewAtmosphere plugin and atm_builder program in the context of the overall simulation. This figure also attempts to show the relationship to other components of the simulation including the ephemeris plugin, sensor plugins(s), and scenes used in the simulation. The atm_builder interacts with the loaded sensor plugin(s) to gather information about the temporal and spectral states that will need to be provided by the database. The builder also interacts with the loaded scenes to gather information about the scene locations and bounding boxes.

flow
Figure 1. Relationship between the atm_builder program and other components of the simulation.

Approach

The design of the HDF database and the approach to populating the data for that database was strongly influenced by the lessons learned with the DIRSIG4-era make_adb tool. The general structure of the database is based on data tables that are organized into "states", which parallel the spectral and temporal states used inside DIRSIG5. Each "state" represents a specific spectral sampling and window in time. Multiple states arise from unique combinations of these spectral and temporal

  • An RGB instrument will request a set of wavelengths specific to that sensor but when it is capturing data over a long period of time, then the atmosphere will need to be temporally sampled over that time window. This results in a set of states that all have the same spectral sampling but are for different points in time.

  • In contrast, a multi-sensor payload with an RGB camera, SWIR camera, MWIR camera and LWIR camera might only collect data at a single point in time, but will require at unique state for each camera due to address the multiple spectral windows. This results in a set of states that have the same temporal sampling but different spectral sampling.

  • And a complex simulation might have multiple sensors (different spectral and temporal samplings) that result in many spectrally and temporally unique states.

Note In addition to important aspects of the atmosphere changing with time (specifically, the position of the sun and resulting atmospheric scattering), each unique temporal state might also result in needing to predict a unique spatial and/or angular region of the atmosphere due to dynamic platform motion.

The atm_builder tool leverages a set of backend models that provide a generic interface to atmospheric radiative transfer models (e.g. MODTRAN, etc.). The builder creates generic requests for the backend models, and these backends translate the request into inputs for the respective external model. An example request is "get the spectral irradiance from the sun reaching a given latitude, longitude and altitude at a specific date and time". The builder creates a list of requests and then submits them to the backend, which allows the backend to parallelize these requests (if possible).

The atm_builder tool constructs it’s list of requests using the following approach to determine the needs of the various sensors configured in the simulation:

  • The sensor plugins for the simulation are loaded and initialized.

  • A list of temporal states spanning the sensors is constructed.

    • A given sensor might have a read-out rate that results in 10s, 100s or 1000s of unique times during the simulation. However, these times might only be separated by fractions of a second during which we would not expect the atmosphere to change. In that case, it would be inefficient to rerun an atmosphere model (e.g. MODTRAN) only to get the same results.

    • Hence, there is a time resolution that is used to temporally partition these times into quantized times between which we do expect the atmospheric values to change significantly. The user has control over this temporal quantization (see the time_delta variable discussed later in this manual).

  • Once the set of unique temporal states have been determined, a set of combined spectral+temporal states is created.

    • For each time window, the sensors that will be imaging during that window are identified and a unique spectral+temporal combo state is created for each sensor that has a unique spectral sampling.

    • Additionally, all the captures (e.g. focal plane readouts) that are in the time window for each sensor is determined.

  • At the end of the combo state generation process, the tool has a set of combo states, where each state represents a unique time, a unique set of wavelengths and a unique angular region of the atmosphere (driven by the fields-of-view (FOV) of each capture mapped to the state).

The list of combined spectral+temporal states form the top-level of the hierarchical database. The next level in the database addresses the needs of the scenes in the simulation. Specifically, what altitudes need to be considered when the atmospheric models are run. For example, a space-based sensor looking down on the Earth might see all the way down the ground, so the altitude of the ground scene needs to be established. That scene might span a large range of altitudes due to mountains that are represented in the scene. In that case, we might want to make requests of the backend atmosphere models to know something like the ground reaching solar irradiance at multiple altitudes so that values can be appropriately interpolated. To establish these key altitudes, the bounding boxes for all of the scenes used in the simulation are loaded. This allows the builder to get a handle on the spatial extents of the virtual world that will be modeled. However, it is not necessary to consider the differences in solar reaching irradiance, or path transmission to the top and bottom of a scene that is only a few meters tall. So the key altitudes to utilize are quantized to avoid unnecessary backend model requests. The following approach is used to generate the final list of key altitudes.

  • Iterating across the scenes, the minimum and maximum Z for each scene is extracted and added to a list candidate altitudes.

  • The list of altitudes across all scenes is sorted to account for vertical overlaps in altitude ranges.

  • The lowest altitude in the sorted list is added as the first key altitude.

  • Moving up through the higher altitudes in the sorted list, new altitudes are added to the key altitude list each time one exceeds the previously stored altitude by a given range (see the altitude_delta variable discussed later in this manual).

scene altitudes
Figure 2. Using the scene bounding boxes to establish key altitudes (H0, H1, H2 and H3).
Tip The optional --altitude_list command-line variable and the altitude_list JSON configuration variable will override this approach.
Important The optional <groundaltitude> in the scene file allows the user to explicitly define the altitude of the scene that should be interpreted as the minimum or "ground" altitude. This avoids the shortcomings with assuming that the minimum, average, etc. Z of the scene is the "ground". When a scene HDF contains the GroundAltitudeOverride (an optional file attribute that carries the <groundaltitude> from the scene file), the algorithm outlined above will use this override value rather than the minimum Z from the bounding box.

The list of combined spectral+temporal states and the list of key altitudes are fed to request generator, which will submit the various backend requests needed to satisfy the needs of the simulation.

Program Usage

The atm_builder tool is what will build the HDF atmospheric database. It loads the simulation configuration, determines what backend calculations need to be performed and then has the backend execute those calculations. Below is the command-line usage message when either no arguments or the --help option is supplied:

$ atm_builder --help
Usage: atm_builder [options] jsim_filename
Database builder for NewAtmosphere plugin

Options:
  -h, --help                              Displays this help.
  -v, --version                           Displays version information.
  --verbose                               Enable verbose output of the tool
  --no_run                                Do not run the backend atmospheric
                                          model (aka 'dry run' mode)
  --save_files                            Save input/output files generated by
                                          the backend atmospheric model
  --exec <mode>                           Execution mode to uses (local, mpi)
                                          [default=local]
  --time_delta <T_delta>                  Set the interval between time steps
                                          (default: 900 seconds
  --altitude_minimum <H_min>              Set the minimum scene altitude sample
                                          (default: 0 meters)
  --altitude_delta <H_delta>              Set the interval between scene
                                          altitude samples (default: 500 meters)
  --altitude_list <H_list>                Set the list of scene altitudes
                                          samples to use (default: N/A)
  --fov_min_deltas <X,Y>                  Set the minimum angle between sensor
                                          FOV paths (default: 5,5 degrees)
  --sky_zenith_sampling <min,max,delta>   Set the zenith (declination) sampling
                                          for sky paths (default: 7.5 -> 82.5 @
                                          15 degrees)
  --sky_azimuth_sampling <min,max,delta>  Set the azimuth sampling for sky
                                          paths (default: 0 -> 360 @ 30 degrees)
  --max_range <R_max>                     Set the maximum range within the FOV
                                          paths (default: N/A)
  --max_processes <N>                     Set the maximum number of backend
                                          model processes (default: <virtual
                                          cores>)
  --log_level <level>                     Set the minimum logging level. Values
                                          are: debug, info, warning, error,
                                          critical, off

Arguments:
  jsim_filename               JSIM filename

General operation entails supplying the name of the JSIM file to the tool, along with any options.

$ atm_builder example.jsim

Command-Line Options

The following command-line options provide additional utility:

--no_run or --no_run=<true,false>

This option will tell the backend to configure all the backend runs, but it will not execute them. The input files will be preserved after the program completes. The default value is false.

--save_files or --save_files=<true,false>

This option will tell the backend to save all the input and output files generated to execute the associated model (overrides the save_files option in the JSON configuration). The default value is false.

--time_delta=T_delta

This option will set the time resolution (in seconds) of the database (overrides the time_delta variable in the JSON configuration). The default value is 900 seconds (15 minutes).

--altitude_minimum=H_min

This option is used to override the altitude (in meters) for the lowest altitude in the altitude list used to generate the database.

--altitude_delta=H_delta

This option will set the altitude resolution (in meters) of the database (overrides the altitude_delta variable in the JSON configuration). The default value is 500 meters (0.5 km).

--altitude_list=H_list

This option will define the list of sample altitudes (in meters) in the database. This option overrides the algorithm that determines the sample altitudes from the scenes and ignores the altitude_minimum and altitude_delta variables used by that algorithm. (overrides the altitude_list variable in the JSON configuration).

--fov_min_deltas=X,Y

This option specifies the minimum angle between path samples within the field-of-view (FOV) of any given capture. The default values are 5 and 5 degrees in zenith and azimuth.

--sky_zenith_sampling=min,max,delta

This option specifies zenith (declination) sampling of the sky. The default values are 7.5 → 82.5 degres @ 15 degree intervals (e.g., 7.5,82.5,15.0).

--sky_azimuth_sampling=min,max,delta

This option specifies azimuth sampling of the sky. The default values are 0 → 360 degres @ 30 degree intervals (e.g., 0,360,30.0).

--max_range=R_max

This option is used when the sensor FOV includes the horizon and is used to constrain the length (in meters) of paths computed by MODTRAN for DIRSIG paths above the horizon.

--max_processes=N

This option will tell the backed the maximum number of external model processes it is allowed to create (overrides the process_count variable in the JSON configuration). The default value is the number of virtual cores on the host computer.

--log_level=<L>

This option allows the user to configure the verbosity of the tool output. The available options are debug, info, warning, error, critical and off. The default is info. Setting this to debug will yield addition information that can be useful for debugging purposes.

Configuration

The NewAtmosphere plugin configuration in the JSIM file is provided to the atm_builder program to create the database used at run time. The configuration of the NewAtmosphere plugin (which is used by atm_builder) is described in detail in the plugin manual. The configuration of the supported atmospheric model backends is described here.