This guide is meant to help users understand and resolve common warnings and errors generated by DIRSIG and it’s companion utilities.

MODTRAN Related

Configuration Verification Failures

At the start of execution, the atmospheric database builders (make_adb, atm_builder, etc.) perform a multi-step configuration check on the supplied MODTRAN configuration soon after the program starts but before it starts to run MODTRAN for the specific cases needed for your simulation. The point of this check is to identify specific configuration issues before getting into the data generation phase of the execution. The checks focus on commonly occuring mistakes made in the configuration provided to DIRSIG. These checks include:

  1. Verifying the MODTRAN executable provided exists,

  2. Verifying the MODTRAN executable provided is a program,

  3. Verifying the MODTRAN DATA directory provided exists,

  4. Verifying the MODTRAN DATA directory is a directory,

  5. Verifying that a simple MODTRAN input can be read,

  6. Verifying that this simple MODTRAN input can be run, and

  7. Verifying that the expected MODTRAN output files are produced.

When correctly configured, this series of checks will produce the following series of messages:

Verifying MODTRAN configuration:
    Checking if MODTRAN program path exists ... passed!
    Checking if MODTRAN program is executable ... passed!
    Checking if MODTRAN data path exists ... passed!
    Checking if MODTRAN data path is a directory ... passed!
    Reading a simple MODTRAN example ... passed!
    Running a simple MODTRAN example ... passed!
    Checking for MODTRAN output files ... passed!

If any of the checks fail, an informative error should be provided that identifies the problem. For example:

Verifying MODTRAN configuration:
    Checking if MODTRAN program path exists ...
checkAtmModels: ERROR!
    Invalid path for MODTRAN executable
    File does not exits!
    Executable path = '/opt/Mod4v3r1/Mod4v3r1.exe'

In this case, the provided path to the MODTRAN executable was invalid. The user should verify the correct path and update the relevant profile in their user settings or input configuration file. Similar errors are generated for checking the existance of the DATA directory provided and permissions for the executable and data directory. For example:

Verifying MODTRAN configuration:
    Checking if MODTRAN program path exists ... passed!
    Checking if MODTRAN program is executable ...
checkAtmModels: ERROR
    MODTRAN is not executable by this user!
    Executable path = '/opt/Mod4v3r1/Mod4v3r1.exe'

This error indicates that the path to the MODTRAN executable might be valid but the user does not have permissions to run the program.

Providing specific solutions for the large variety of reasons that MODTRAN might not run the simple test scenario is not possible. Instead, the following specific troubleshooting tips related to a failed test execution are provided:

  • If the "running a simple example" check fails, look in the test_run folder for the input tape5 file that was used and any MODTRAN output files.

  • Depending on the DIRSIG program, the version of MODTRAN, etc. specific errors might be captured in the following files:

    • For DIRSIG5-era generators, look in modtran_stdout.txt and modtran_stderr.txt.

    • For DIRSIG4-era generators, look for the modtran.log file.

    • Any MODTRAN version specific warning or error files (e.g., warnings.txt).

    • If it exists, the tape6 or <root>.tp6 file can contain errors.

Could not open MODTRAN output file

Example

A make_adb run fails with the following (or similar) error:

Example of a missing MODTRAN output file error from make_adb.
make_adb: OPEN ERROR
    Could not open MODTRAN output file!
    Filename = solar_path_00000/tape7.scn
    Exiting...

Or a atm_builder or fourcurve_builder run fails with the following (or similar) error:

Example of a missing MODTRAN output file error from atm_builder.
[error] Backend failed to peform request:
[error] ModtranTapeBackend::readGeneralPathResult:
[error] Unable to open MODTRAN output file '/tmp/test/modtran_runs0/2/tmp.7sc'
[error] unable to open file: No such file or directory

The folder containing the file might vary (solar_path_xxxxx, sensor_path_xxxxx, sky_path_xxxxx, modtran_runs0, etc.) but the key element is that a file named tape7.scn could not be opened.

Explanation

This file is one of the standard MODTRAN output files that the make_adb, atm_builder, fourcurve_builder, etc. programs try to read in. If this file cannot be opened, it means it was never generated. Which means that MODTRAN (a) never started or (b) started but did not complete.

Resolution

There are numerous reasons why MODTRAN might not start or might not complete and this guide cannot individually address all of them. Instead, we should start with finding out what MODTRAN might have reported when attempting to run this case.

If you were running the DIRSIG4-era make_adb tool:

  • Start by looking at the modtran.log file for your simulation. This will contain a series of messages related to running MODTRAN behind the scenes. It may also have captured error messages were generated by MODTRAN that would normally have gone to the console. Such an error (if present) would likely explain the specific issue clearer than a long list of possible issues would here.

  • It can then be valuable to look in the folder itself (in this case, solar_path_00000). That folder should contain the tape5 file that was generated by the make_adb program.

    • If MODTRAN completely failed to start, then there won’t be any additional files present and the modtran.log file will have the most useful information.

    • If MODTRAN ran, but failed to complete then there will be a tape6 file present, and potentially a tape7 file present, but the tape7.scn file is not present. In this case, you want to check the modtran.log file but also the tape6 file for errors and/or warnings.

    • Some versions of MODTRAN produced a warnings.txt file, which might contain useful information.

    • The user can always attempt to manually run MODTRAN in this folder to directly observe any MODTRAN warnings and/or errors. Be sure to run the exact same executable that make_adb is using (i.e., the executable path from the respective MODTRAN profile from the user’s Preferences/Settings).

Important If using MODTRAN6 with the suggested wrapper script approach, then the MODTRAN output files might still be mod6_tmp.tp6, mod6_tmp.tp7 and mod6_tmp.7sc if the script hasn’t run yet or tape6, tape7 and tape7.scn if it has run or tried to run. Either way, look at the modtran.log and either mod6_tmp.tp6 or tape6 (whichever is present) for more information.

If you were running the DIRSIG5-era atm_builder, fourcurve_builder, etc. tools:

  • The modtran_runs folder(s) contain sub-folders for each MODTRAN simulation. In those sub-folders, you will find the input and output files for each MODTRAN simulation.

    • Each sub-folder will contain (at a minimum) a README.txt file that has a plain-text summary of the simulation and a tmp.tp5 file (the primary input file for MODTRAN).

    • If MODTRAN completely failed to start, then there won’t be any additional files present and the modtran_stdout.txt and modtran_stderr.txt files that captured the output messages for MODTRAN should have the most useful information.

    • If MODTRAN ran, but failed to complete then there will be a tmp.tp6 file present, and potentially a tmp.tp7 file present, but the tmp.7sc file is not present. In this case, you want to check the modtran_stdout.txt and modtran_stderr.txt files but also the tmp.tp6 file for errors and/or warnings.

    • Some versions of MODTRAN produced a warnings.txt file, which might contain useful information.

    • The user can always attempt to manually run MODTRAN in this folder to directly observe any MODTRAN warnings and/or errors. Be sure to run the exact same executable that atm_builder, etc. was using (i.e., the executable path from the respective MODTRAN profile from the user’s Preferences/Settings or the executable path in the backend setup in the JSIM file).

Spectral resolution is too fine

Example

A MODTRAN simulation fails with a message indicating that the "spectral resolution is too fine" (normally output by the CHKRES function in MODTRAN):

Example resolution check error.
STOP Error in CHKRES:  Spectral resolution is too fine.

Depending of the version of MODTRAN, this error might have been captured by the logs created by DIRSIG of the MODTRAN console output or it might appear in the tape6 (or xxx.tp6) output file.

Explanation

When this error is generated, it means that the requested spectral output requested in the MODTRAN input file exceeded the spectral resolution of the band model being used. Unless otherwise specified via advanced options in the input, MODTRAN will automatically pick the appropriate band model. In the case of using the automatic selection mode (the most commonly used), then the error indicates that the requested resolution exceeded that of the highest resolution band model file available. If your input explicitly requests a specific band model file, then the resolution exceeds that supported by the requested file.

Resolution

There are several possible solutions to this problem:

  1. If the user-supplied input (e.g., tape5 file) explicitly requests a band model file, select an alternate band model file that supports the requested resolution. Or let MODTRAN choose the best band model file automatically.

  2. If using an old version of MODTRAN (e.g., MODTRAN4), upgrade to the latest version (e.g., MODTRAN6) to gain access to the higher resolution band model and dozens of other improvements.

  3. Decrease the spectral resolution of the sensor to meet the resolution of the available band models.

Not enough spectral points

Example

A MODTRAN simulation fails with a message indicating that it "needs more spectral points" (normally output by the CHKRES function in MODTRAN):

Example spectral points check error.
STOP Error in CHKRES: Need more spectral points.

Depending of the version of MODTRAN, this error might have been captured by the logs created by DIRSIG of the MODTRAN console output or it might appear in the tape6 (or xxx.tp6) output file.

Explanation

When this error is generated, it most likely means that the spectral bandpass requested in the MODTRAN input file had no width or defines a single spectral sample. MODTRAN cannot run when the minimum and maximum wavelength (or wavenumber) are the same (zero width) or with a single spectral sample.

Resolution

This is most likely because the DIRSIG sensor model was configured to have the respective minimum and maximum wavelength be the same or the spectral sampling results in a single point (i.e., the spectral sampling interval or delta is equal to the difference betwen the minimum and maximum). DIRSIG and MODTRAN are inherently spectral models, so requesting more than a single spectral sample is required. In most cases, we want 10s or 100s of spectral samples to capture the variablility in a channel.

The resolution to the problem is to define spectral bandpasses in the configured sensors with more than 1 (ideally many more than 1) spectral samples. Since each DIRSIG sensor plugin has its own way of defining the spectral bandpass, there is no universal solution. However, the following quick guidance is provided for the most commonly used sensor plugins:

  • For the BasicPlatform plugin, one (or more) of the focal plane bandpasses in the .platform file has either the minimum equal to the maximum, or the delta set to the difference between the minimum equal to the maximum.

  • For the OrthoImage, SphericalCollector, etc. plugins, the bandpass variable in the input JSON has either the minimum equal to the maximum, or the delta set to the difference between the minimum equal to the maximum.

Tangent height cannot be reached

Example

A MODTRAN simulation fails with a message indicating that the "tangent cannot be reached" (normally output by the FINDMN function in MODTRAN):

Example tangent height error.
Warning from FINDMN:  H2ALT (= 0.00000E+00km) is less than the tangent
                      height (= 4.28090E+00km) and cannot be reached.

Explanation

When this error is generated, it means that the requested path in MODTRAN never intersected the atmosphere. This usually indicates that the user has configured an exo-atmospheric sensor that has a FOV that includes the earth limb (an off nadir view that sees both the earth and space). This specific (but terse) error is indicates that a path was requested from H1 to H2 but the tangent height of the path (lowest altitude) wasn’t reachable.

Resolution

There is no resolution to this problem, because MODTRAN is not intended to be used to view the limb of the earth or the day/night terminator. Due to MODTRAN’s internal round earth model and path refraction it is difficult for DIRSIG to accurately predict which paths that will intersect the earth, skim the horizon, skim the top of the atmosphere, etc. Hence, DIRSIG is currently unable to detect these conditions prior to sending them to MODTRAN and tell the user that the viewing geometry is unsupported.

NewAtmosphere Plugin Related

The following warnings and errors are specific to the NewAtmosphere plugin.

Backend Request Failure

The "backend" modules interface DIRSIG5’s various atmospheric database builders with atmospheric radiative transfer models. Each backend is an adapter between generic atmospheric radiative transfer requests and the specific model that they interface with (e.g., MODTRAN). Due to the complex nature of the interaction, a myriad of errors can occur. Using one possible error scenario, this section attempts to provide the user with a guide to identify the specific error that occurred in their situation.

Example

During the building of a new atmospheric database using the atm_builder tool, an error similar to that below is generated:

Version: 2022.27 (d968ad6) built on Jul  6 2022 07:02:14
Copyright 2015-2022 Rochester Institute of Technology

...
[output deleted for documentation purposes]
...

Generating data for spectral+temporal state #0 (226 requests)
    Sun zenith/azimuth = 45.000000,90.000000
    Moon zenith/azimuth = 45.000000,270.000000
    Moon phase angle = 0.000000 (fraction = 0.500000)
Multi-threaded atm model request runner is using 10 threads (processes)

Exception encountered while building database!
backend failed to perform request:
MODTRAN output file "/Users/dirsig/test/modtran_runs0/127/tmp.7sc" is empty

This is considered a fatal error and will halt execution.

Explanation

The error indicates that the builder attempted to run MODTRAN and that attempt failed. In this case, the specific MODTRAN run in the modtran_runs0/127 folder was the one that failed. The modtran_runs0 folder is a temporary folder created by atm_builder within which it constructs a series of sub-folders for each MODTRAN run to execute. The run sub-folders are numbered based on their position in the request queue. In this case, run 127 was the setup that failed. Various errors might be encountered that would indicate the MODTRAN run did not execute as expected. However, these are usually related to one or more of the MODTRAN output files is missing or empty. In example above, the tmp.7sc file was found to be empty.

There are many reasons why a MODTRAN run can fail that are unrelated to your simulation. These primarily have to do with the configuration of MODTRAN in the backend:

  • The path to the MODTRAN executable is wrong (see the modtran_exe variable in the backend configuration).

  • The path to the MODTRAN DATA directory is wrong (see the modtran_data variable in the backend configuration).

Backend configuration issues should start with a review of the backend setup options.

Resolution

To diagnose the root cause and hopefully address the nature of the problem, we will look at the files in the run sub-folder. In this case the name of the folder where the run failed was modtran_runs0/127 but the exact name of the sub-folder can obviously vary. Browse to this folder and look at what files exist. The list of files is the first indicator of where things went wrong.

The folder listing below illustrates the minimal set of input files for MODTRAN that were generated by the atm_builder tool.

Input files for a MODTRAN4 run produced by the atm_builder tool.
-rw-r--r--  1 dirsig dirsig   253 Jul  6 07:22 README.txt
-rw-r--r--  1 dirsig dirsig     3 Jul  6 07:22 modroot.in
-rw-r--r--  1 dirsig dirsig   553 Jul  6 07:22 tmp.tp5

If this is all the files contained in the folder, then MODTRAN never ran and the problem is most likely in the backend configuration.

Note The README.txt file contains a human readable summary of the MODTRAN configured in this folder. The modroot.in file is the "root" file for MODTRAN (in this case, the name corresponds to using MODTRAN4) and the tmp.tp5 is the actual MODTRAN input file.

If the folder listing contains the files modtran_stderr.txt and/or modtran_stdout.txt, then MODTRAN was run and failed. Most likely, some of the MODTRAN output files will also exist in in this case. Below a long/detailed listing of the files you might find in the folder in this case if MODTRAN ran but failed:

Input/Output files for a failed MODTRAN4 run.
-rw-r--r--  1 dirsig dirsig   253 Jul  6 07:22 README.txt
-rw-r--r--  1 dirsig dirsig     3 Jul  6 07:22 modroot.in
-rw-r-----  1 dirsig dirsig   138 Jul  6 07:22 modtran_stderr.txt
-rw-r-----  1 dirsig dirsig     0 Jul  6 07:20 modtran_stdout.txt
-rw-r--r--  1 dirsig dirsig     0 Jul  6 07:22 tmp.7sc
-rw-r--r--  1 dirsig dirsig     0 Jul  6 07:20 tmp.7sr
-rw-r--r--  1 dirsig dirsig     0 Jul  6 07:22 tmp.mc
-rw-r--r--  1 dirsig dirsig   553 Jul  6 07:22 tmp.tp5
-rw-r--r--  1 dirsig dirsig  1940 Jul  6 07:22 tmp.tp6
-rw-r--r--  1 dirsig dirsig     0 Jul  6 07:22 tmp.tp7
-rw-r--r--  1 dirsig dirsig     0 Jul  6 07:22 tmp.tp8

As we can see in this detailed listing, many of the MODTRAN output files are zero length (empty), which corresponds to the error specifically used in this example (tmp.7sc is empty).

The first place to start is looking at the modtran_stderr.txt and modtran_stdout.txt. In this case, the error file is not empty (it is 138 bytes long) and most likely contains an error produced by MODTRAN.

$ more modtran_stderr.txt
Note: The following floating-point exceptions are signalling: IEEE_UNDERFLOW_FLAG
STOP Error in CHKRES:  Spectral resolution is too fine.
Note For guidance on how to resolve this specific error see this section.

If the modtran_stderr.txt and modtran_stdout.txt files do not contain any useful error messages, then you should look in other locations.

  • Look at end of the tmp.tp6 file for warnings and/or errors.

  • Check if there is a tmp.wrn ("warning" file)

If none of the MODTRAN output files contain a useful error, then it is possible the error was written to the output and (somehow) not captured in the modtran_stderr.txt and modtran_stdout.txt files. In this case, we suggest you manually run MODTRAN from the command-line to observe any warnings or errors produced.

Missing Spectral/Temporal State

Example

During the execution of the simulation, the following error is generated:

NewAtmosphere::updateState:
    Missing spectral/temporal state in atmosphere database!
    Filename = 'example.adb.hdf'

This is considered an error and will halt execution.

Explanation

When the atmospheric database was generated (see the atm_builder utility), the sensor plugin(s) associated with the simulation defined a set of spectral and temporal states that the database was created to support. A spectral state is a set of wavelengths and a temporal state is a date/time. Each table in the database is for a given sensor (the spectral state) and time (the temporal state) and this error indicates that during the simulation a spectral and temporal state combination was looked for in the atmospheric database but it could not be found.

This can arise from changes to the simulation since the atmospheric database was created. For example:

  • The spectral description of one of the sensor plugins was altered. For example, the spectral bandpass or sampling of a bandpass was changed.

  • The temporal description of one of the sensor plugins was altered. For example, the start time, end time and/or duration that a sensor plugin is collecting changed.

  • A new sensor plugin was added to the simulation or a new camera, instrument, focal plane, etc. was added to an existing sensor plugin.

Resolution

The atmospheric database needs to be reconstructed to reflect the additional spectral and/or temporal states required by the sensor plugin(s) in the simulation.

BasicPlatform Plugin Related

The following warnings and errors are specific to the BasicPlatform plugin.

Unevenly Sampled Bandpass

Example

During the setup of a sensor plugin, the following error message is generated:

readBandpass: Bandpass is not evenly sampled! [0.4 - 0.705] @ 0.01

This is considered an error and will halt execution.

Explanation

The BasicPlatform plugin attempted to create the list of wavelengths for the supplied bandpass but discovered that the range defined by the min and max cannot be evenly divided by the delta. In this example, we can see that the delta (0.010) would not evenly sample the range from 0.400 to 0.705. Specifically, we could see that if we start computing the wavelengths starting at the minimum (e.g., 0.400, 0.410, 0.420 …​) that when we get to the end of the bandpass the samples don’t include the supplied maximum (e.g., 0.680, 0.690, 0.700 and 0.710 does not include 0.705).

Resolution

There are two ways to resolve this problem:

  1. Adjust the bandpass minimum or maximum to agree with the delta. In this case, we could leave the minimum and delta as is and adjust the maximum to be 0.700 or 0.710.

  2. Adjust the delta to evenly sample the range defined by the minimum and maximum. In this case, we can adjust the delta to be 0.005 which would produce a wavelength list that includes the minimum and maximum (e.g., 0.400, 0.405, 0.410 …​ 0.695, 0.700 and 0.705).

Channel Truncation

Example

During the setup of a sensor plugin, the following warning message is generated:

Channel::setup: WARNING!
    Channel is truncated by the bandpass!
    Name = 'Red'
    Bandpass = [0.40000 - 0.70000] microns
    Channel  = [0.32400 - 0.57600] microns

This is only a warning and will not halt execution.

Explanation

You have defined a channel that has a given spectral extent but the spectral bandpass associated with the focal plane truncates that channel. When the platform file generating this warning is loaded into the graphical Platform Editor the channel plot in the Basic Capture Editor helps to visualize the problem:

clipped channel1
Figure 1. Channel plot in the DIRSIG Platform Editor shows the Red channel being clipped at 0.400 microns.

The Red channel is the highlighted Gaussian shaped channel to the far left and you can see that the 0.400 to 0.730 bandpass is clipping off the leading tail of that channel.

Note This warning can be generated for multiple channels and applies to both tabulated and functional channel shapes.

Resolution

To resolve this warning, you will want to adjust the bandpass minimum (or maximum) so that it correctly spans the channels associated with the focal plane. In this example, the minimum of the bandpass needs to be changed to a shorter wavelength to avoid clipping the Red channel. Below shows the channel plot with the bandpass minimum adjusted from 0.400 down to 0.370 and which resolves the issue:

clipped channel2
Figure 2. Channel plot in the DIRSIG Platform Editor with an appropriate bandpass.

Scene Related

Scene/Sensor Spectral Overlap

Example

When a simulation starts, you receive an error that resembles the following:

While checking spectral consistency:
State wavelength range: 0.45 - 1.5 [um]
Scene wavelength range: 0.30 - 1.4 [um]

This is considered an error and will halt execution.

Explanation

The "state" wavelength range corresponds to the spectral bandpass for one of the sensor plugins in your simulation. For example, if you are using BasicPlatform plugin then this would correspond to the bandpass associated with one of the focal planes. The "scene" wavelength range corresponds to the scene used in your simulation. When someone builds a scene, they might not have enough data to populate the spectral databases at all wavelengths. As part of the scene meta-data, the scene builder can indicate which wavelengths the spectral database for the scene supports.

What the error is highlighting is that you are using a sensor that is looking at wavelengths that are not supported by the scene. Specifically, the scene has spectral coverage out to 1.4 microns but the sensor wants to simulate out to 1.5 microns.

Resolution

There are limited solutions for this situation:

  1. Since the scene does not contain the spectral data to support the sensor, we can improve the scene databases to include that data and then update the meta-data spectral coverage description to indicate the expanded coverage.

  2. Since the scene doesn’t support the sensor’s needs, use a different scene.

  3. Since the scene doesn’t support the sensor’s needs, use a different sensor.