This document is supposed to help users debug issues with MODTRAN when that software is run by the DIRSIG model.

Because MODTRAN doesn’t come with a robust, standardized, well documented application programmers interface (API), software like DIRSIG must integrate with MODTRAN through MODTRAN’s standard file interfaces. For a variety of reasons, this coupling can be problematic. For example, different versions of MODTRAN can behave differently when an error occurs or report that same error differently. In other cases, the format of various input and output files change and software that reads and writes those files must be updated.

Note Although the MODTRAN input and output files are documented in the MODTRAN manual, this mechanism falls short at being a robust API for a variety of reasons. The biggest limitation is the lack of a consistent and formalized error reporting mechanism that can be easily interpreted by the calling program. Recent versions of MODTRAN are improving this, but formal APIs have a consistent behavior when errors are produced.

Because this model integration is largely handled as a "black box" to the user, it seems like magic when it works but is very frustrating and obfuscated when it does not. This document is designed to help users in the later case. This document will guide the user through a step by step process to understanding how DIRSIG and MODTRAN interact and how common problems with that interaction can be resolved.

MODTRAN Software

Versions

At this time, DIRSIG has been tested to work with the following versions of MODTRAN:

  • MODTRAN4, Version 3, Release 1 (MODTRAN 4v3r1)

    • This is one of the last releases of the MODTRAN4 software. We strongly recommend not using any version prior to this release because of numerous bugs in the software, important improvements to the software, etc.

  • MODTRAN5, 2.0 and 2.2 (MODTRAN 5.2.0 and MODTRAN 5.2.2)

    • These are the most common releases of MODTRAN5 we encounter in the wild at the time of this writing. Both versions have been used with DIRSIG on all of the supported platforms.

  • MODTRAN6

    • This is the latest version of MODTRAN and it features some changes to the interface. Until DIRSIG is updated to utilize the new interface, users will need to utilize the temporary "wrapper" solution described below.

Important Although MODTRAN 4v3r1 works with DIRSIG, it is strongly recommended that the user employ MODTRAN5 or MODTRAN6 with DIRSIG. For more information, visit http://www.modtran.org.

These versions of MODTRAN have been found to work on Windows, Mac OSX and UNIX platforms.

Known Issues

Using MODTRAN6 with DIRSIG

This latest version of MODTRAN marks a major milestone in the continued development of one of the most popular and trusted codes for simulating radiative transfer in the atmosphere. In addition to important science related advancements, this latest code also includes significant improvements to the general usability of the software. This includes a new graphical user interface (GUI) and the introduction of a formal application programmer interface (API), which let’s codes like DIRSIG interact with MODTRAN in a far more robust way than previous versions allowed.

The major development in the interface area is a shift from the old "tape5" style inputs to a new JSON (JavaScript Object Notation) style input. In addition to improving the general readability of the input, the JSON document format is much easier to read in, modify and write back out. The old tape5 format required software like DIRSIG to know a great deal about the format of the input file in order to correctly read in, modify and write back out a set of MODTRAN inputs. If a new variable was added to an existing input "card", a new input "card" added, etc. then the DIRSIG code needed to be modified to know to read it in and write it back out, even if it never modified those variables. The JSON document style input now employed by MODTRAN6 can be read into a generic structure, the variables of interest can be modified in a straight-forward way, and the document written back out without needing to code extensive logic that understands the meaning and impact of every variable.

However, DIRSIG has not been updated to use the new JSON input format yet. Until DIRSIG is updated, our short-term solution is to make a script that wraps around the MODTRAN executable and perform some basic file manipulations so that the script makes MODTRAN6 "look like" older versions of MODTRAN. As a result, MODTRAN6 can be used with any existing version of DIRSIG without any code modifications to DIRSIG or installation updates for the end-user. The example script below assumes DIRSIG wrote to the file tape5, it renames that file to mod6_tmp.tp5 and then launches MODTRAN6 supplying mod6_tmp.tp5 as a command-line argument. Note that this rename approach was selected over the option of converting the input file to JSON in order to avoid the runtime associated with that conversion. Additionally, DIRSIG expects the output files to be named tape6, tape7, etc. so this script also renames the output files after MODTRAN6 has finished execution.

Here is a basic Bourne/Bash script version of the wrapper than can be utilized on Linux or macOS:

#!/bin/sh
#
# MODTRAN6 wrapper for Linux and MacOS for DIRSIG4
#

# This variable points to MODTRAN6 bin folder
BIN_DIR=/usr/local/MODTRAN6.0.0/bin/linux

# This variable points to the MODTRAN6 DATA folder.  Note that this
# is redundant to what is stored in the MODTRAN profile in the DIRSIG
# settings. But it MUST be correctly specified here because it is a
# required command-line option that is not passed in by DIRSIG.
DATA_DIR=/usr/local/MODTRAN6.0.0/DATA

# This variable defines the basename of the temporary MODTRAN intput
# and output files. This makes it easy to change if we find the current
# one is a bad choice.
TMP_BASENAME=mod6_tmp

# Step #1:
# Rename "tape5" to a file with a ".tp5" extension so we
# can provide a it as a command-line argument in Step #2.
mv tape5 ${TMP_BASENAME}.tp5

# Step #2:
# Run the MODTRAN6 "console" executable (not the GUI!) by specifying the
# name of our renamed "tape5" file.  We also provide the path to the DATA
# folder.
${BIN_DIR}/mod6c_cons ${TMP_BASENAME}.tp5 ${DATA_DIR}

# Step #3:
# Rename all the output files to what DIRSIG4 expects them to be.
mv ${TMP_BASENAME}.tp5 tape5
mv ${TMP_BASENAME}.tp6 tape6
mv ${TMP_BASENAME}.tp7 tape7
mv ${TMP_BASENAME}.7sc tape7.scn

# Step #4:
# Remove other output files we don't care about.
rm ${TMP_BASENAME}.psc

The user must modify the script to reflect the location of their MODTRAN6 installation. Specifically, the paths assigned to the BIN_DIR and DATA_DIR variables in the script.

On our systems, we have named this script mod6_wrapper.sh, given it execute permissions and placed it in the MODTRAN6 bin folder with the distributed executables. To use the script, you should add a new MODTRAN profile to your DIRSIG user settings (see the "MODTRAN" tab in the settings/preferences dialog available in the main simulation editor). Rather than point the DIRSIG profile at the MODTRAN executable, you should select the name of this script instead.

Note The solution for running MODTRAN6 on Windows is esentially the same. In this case, the user shoudl create a Windows .BAT file that performs the same operations as the UNIX script detailed above.

Installation Location

The MODTRAN input files generated by DIRSIG will include the path to the MODTRAN DATA folder containing the band model, correlated-k, etc. database files. The field for this folder path in MODTRAN is limited to 80 characters.

Important Do not install MODTRAN in a folder that results in an absolute path to the DATA folder that is greater than 80 characters.

Mapped Network Drives under Windows

If you are using DIRSIG and/or MODTRAN on a Windows machine that employs a mapped network drives, a variety of problems have been reported. A "mapped network drive" in Windows is when a shortcut to a shared drive is created that does not contain a drive letter. For example, rather than a user’s home folder being Z:\joeuser, the mapped name might be \\home\joeuser.

Important Do not use a mapped network drive to store simulation inputs. Consider copying files to a local disk or use the direct path to the files (one that includes the Windows drive letter).

MODTRAN Installation

This first section discusses the installation of MODTRAN itself. Although the MODTRAN software comes with installation documentation, we will highlight some important aspects here.

Installation Layout

A typical MODTRAN installation is composed of a single file hierarchy that contains the MODTRAN executable, supporting databases, test cases and documentation. The screenshot below shows the typical layout of files and folders in a MODTRAN5 installation (this organization is common to Windows, Mac OSX and UNIX).

images/modtran/windows_folder.png
Figure 1. Screenshot of a standard MODTRAN5 installation folder (Windows).
images/modtran/mac_folder.png
Figure 2. Screenshot of a standard MODTRAN5 installation folder (Mac OSX).

Documentation

The MODTRAN "manual" that focuses on the input file format and how to enable specific features is usually included as a PDF document (shown above as MODTRAN5.2.2.pdf).

Another helpful document is the Frequently Asked Questions (FAQ) document, which includes lots of useful information about the model (shown above as FAQmay2012.pdf).

Databases

The DATA folder contains database files used by MODTRAN at run-time. This includes the band model files (databases of the spectral absorption for atmospheric gases), correlated-K coefficient files and background reflectance spectrum. If you are compiling MODTRAN from source, then some of these databases need to be generated. This will be discussed in another part of this document.

Source Code

The src file contains the source code to the MODTRAN software. On some target platforms the user is expected to compiled MODTRAN from the source code (for example, UNIX). On others, the software comes pre-compiled for the target platform.

Test Cases

The TEST folder contains typically 50+ test cases that can be used by the end-user as examples for specific scenarios, or to verify correct operation of the model.

Pre-Compiled versus Compile Your Own

As of MODTRAN 5.2.0, both the Windows and Mac OSX releases had the MODTRAN program, band model files, etc. pre-packaged and ready to use:

  • The Windows release has both 32-bit and 64-bit versions

    • As a reminder, the 32-bit version can be used on a 64-bit system, but not the other way around.

  • The Mac OSX release has only a 32-bit Intel version.

    • The 32-bit version runs fine on modern Macs, which are almost exclusively 64-bit.

    • If you are still using a PowerPC (PPC), then you will need to follow the instructions to compile he software from source, which will require a FORTRAN compiler.

On the UNIX platform, the software must be compiled from the source code. Follow the instructions included with MODTRAN on how to perform this task.

Console vs. Window Executables on Windows

A typical Windows installation comes with two executables. One is for use from a non-interactive console session (see Mod5.2.2cons.exe in the example file listing above), and the other from interactive session (see Mod5.2.2qwin.exe in the example file listing above). Make sure that DIRSIG is using the "console" executable (the one that contains cons in the name). More information about the difference between these two applications can be found in the MODTRAN documentation.

Dynamic Libraries on Mac OSX

The Mac OSX target platform is one where the MODTRAN distribution comes pre-compiled for the end-user. However, it is pre-compiled using the gfortran compiler, which introduces dependencies for two external, dynamic libraries (they are libgcc_s.1.dylib and libgfortran.3.dylib). These libraries are included with MODTRAN at the top of the installation folder (at the same level as the MODTRAN executable). A common problem with the Mac OSX distribution is that the MODTRAN executable cannot be run from a folder other than the installation folder. This is because the MODTRAN executable does not know where to find these dynamic library files when execution is started from another folder. This creates a problem for 3rd party software like DIRSIG that always starts MODTRAN from a folder other than the installation folder.

To test if this is a problem with your MODTRAN installation, simply try to run the MODTRAN executable from your home folder via the command line (for example, from Terminal.app). If your copy of MODTRAN is installed in /Applications/Mod5.2.2 and the name of the MODTRAN5 executable is Mod90_5.2.2.exe, then try to run the executable directly with the following syntax:

$ cd $HOME
$ /Applications/Mod5.2.2/Mod90_5.2.2.exe

If you get error messages about missing dynamic libraries for libgcc or libgfortran from the runtime loader (dyld), then your installation has this problem. For example:

$ /Applications/Mod5.2.2/Mod90_5.2.2.exe
dyld: Library not loaded: libgfortran.3.dylib
  Referenced from: /Applications/Mod5.2.2/Mod90_5.2.2.exe
  Reason: image not found
Trace/BPT trap: 5

The solution to this problem is to wrap the MODTRAN executable with a shell script that tells the executable the absolute path back to the dynamic library files. This shell script simply needs to add the correct folder path to the DYLD_LIBRARY_PATH variable. An example script is shown below:

#!/bin/sh

# define where MODTRAN lives and the executable name in that location
export MODTRAN_FOLDER=/Applications/Mod5.2.2
export MODTRAN_EXE=Mod90_5.2.2.exe

# check if the dynamic library path variable exists already
if [ -n $DYLD_LIBRARY_PATH ]
then
    # make a new variable with the MODTRAN folder in it
    export DYLD_LIBRARY_PATH=${MODTRAN_FOLDER}
else
    # add the MODTRAN folder to the existing variable
    export DYLD_LIBRARY_PATH=${DYLD_LIBRARY_PATH}:${MODTRAN_FOLDER}
fi

# run the MODTRAN executable with this environment
${MODTRAN_FOLDER}/${MODTRAN_EXE}
Note The user made need to modify the MODTRAN_FOLDER and MODTRAN_EXE variables to reflect the installation folder and MODTRAN executable on their system.

A copy of this script that has been customized with the specifics of your MODTRAN version and installation location can be placed in the MODTRAN installation folder with an appropriate name (for example, run_modtran.sh). The user’s DIRSIG preferences should then point to this script rather than the bare MODTRAN executable.

Populating the DATA Folder

The MODTRAN program relies on a set of band model, correlated-K coefficient, etc. databases to run. These files are stored in the DATA folder within the MODTRAN installation folder.

On the Windows and Mac platform, these databases come pre-built and ready for use. If you are deploying MODTRAN on the UNIX platform, or you have compiled MODTRAN from the source code for any reason, then you will need to create the binary database files yourself. Follow the instructions included with MODTRAN on the correct process to perform this task.

Testing the MODTRAN Installation

The MODTRAN installation folder contains a folder named TEST, which contains 50+ test scenarios for use with MODTRAN. It is best to test the correct operation of your MODTRAN installation by attempting to run some of those test cases. Specifically, verifying that those test cases can be run in a folder outside the MODTRAN installation folder is important. Consider the following test procedure:

  • Make an empty folder to run the test case in.

  • Copy one of the test case "tape5" files (the file with the .tp5 file extension) into your empty folder. Change the name of the copied file to be simply tape5.

  • Start up a command-line session (CMD, Terminal.app, xterm, or whatever is appropriate for your system).

  • Change folders (for example, using the cd command) to your test folder containing the tape5 file you just copied.

  • Attempt to run the MODTRAN executable from the command line using the full path to the installation folder.

  • If the program runs successfully, you should have a tape6 file that ends without error messages (the last line usually contains a copy of the CARD 5 line from the input tape5 file).

Note If one or more of the pre-packaged MODTRAN test cases cannot be run outside of the installation folder as was described above, attempt to resolve the issue before attempting to have DIRSIG utilize MODTRAN.

Setting up DIRSIG to use MODTRAN

The setup of DIRSIG to use MODTRAN is described in the MODTRAN Setup section of the DIRSIG Installation Guide.

Common Failures and Solutions

This section will outline a common set of failures and possible remedies.

If MODTRAN does not run correctly, consider the following steps to understand the nature of the problem:

Are any MODTRAN output files present?

The tape5 file is the input file, but when MODTRAN is run it should generate a set of output files including tape6 and tape7. If you do not see either of these files, then chances are MODTRAN could not be started.

  • Are your preferences pointing to the right MODTRAN executable?

  • Can the MODTRAN executable be run from the simulation folder? Sometimes shared/dynamic libraries are not correctly setup for MODTRAN to be run outside the installation folder. See specific notes earlier in this document about this common problem with the executables distributed for the Mac OSX platform.

Is there a tape6 file but not a tape7 or tape7.scn file?

If this is the case, then MODTRAN was successfully started, but could not run correctly with the supplied input.

  • Look at the end of the tape6 file for error messages

  • Look for a warnings.txt file for error messages

The most common message that a user will receive is that DIRSIG (either from the dirsig driver or the make_adb program) is that the tape7.scn file could not be read.