The BasicPlatform Plugin

This sensor plugin performs spatial and temporal sampling differently than DIRSIG4 did. Specifically, DIRSIG4 used discrete sampling approaches to separately sample the spatial and temporal dimensions of flux onto a given pixel and DIRSIG5 simultaneously samples the spatial and temporal dimensions with uniformly distributed samples.

In DIRSIG4, the amount of sub-pixel sampling was controlled on a per focal plane basis using rigid N x M sub-pixel sampling. Furthermore, delta sampling vs. adaptive sampling was a choice for the user. In contrast the BasicPlatform plugin uniformly samples the detector area and the number of samples used for each pixel is adaptive and driven by the convergence parameters. In general, the entire spatial response description in the input file is ignored by this plugin.

The DIRSIG4 temporal integration feature included an integration time and an integer number of samples to use across that time window. The temporal sampling approach brute-force resampled the entire detector for each sample, which resulted in a proportional increase in run-time. The DIRSIG5 BasicPlatform plugin ignores the number of temporal samples and instead uniformly distributes the ray times across the integration time window.

This is the instrument type used for passive EO/IR simulations. This instrument type features the ability to model multiple focal planes with the simple and raw capture methods available to dictate how indicident flux is captured and written to the output.

This instrument is used when a mono-, bi- or multi-static LIDAR instrument is being modeled. It generally extends the BasicInstrument capability by adding the capture of time-of-flight data via the lidar capture method.

This instrument is used to collect platform or mount relative location and orientation telemetry during a simulation. It can be used to emulate the collection of global position system (GPS) and/or inertial measurement unit (IMU) data.

This is the most commonly used method. This method allows the user to define how the incident radiance is spectrally, spatially and temporally integrated. It provides support for basic apetures, spectral response functions and the ability to convert the incident radiance to digital counts (DCs) through a internal detection model.

This mode is typically used when the user plans on using their own external detection model. The output of this method is a spatially and spectrally "oversampled" (more samples than the ultimate output) data cube. A spatially oversampled incident radiance field provides an external optics and detection model with sub-pixel samples in order to perform modulation transfer function (MTF) calculations and resample to the final sensor resolution. The spectrally oversampled dimension provides enough resolution for spectral integration into channels that might vary across the focal plane.

This capture method is used with laser radar receivers, and adds to the general feature set of the "simple" capture method the ability to output the incident flux as a function of time. Unlike the previously discussed passive capture methods that output an ENVI image file, this capture method outputs spatial x spatial x time data cubes using the DIRSIG BIN format.

By default, the BasicPlatform plugin using the Simple capture method produces an at aperture radiometric data product that is by default in spectrally integrated radiance units of watts/(cm² sr).

Depending on the various options set, the units produced by the Simple capture method can change. For example:

The flux and area units can also be maniupulated in the Output Files tab of the Simple Capture Editor dialog in the graphical user interface.

If an aperture is defined, the resulting acceptance angle of the aperture is incorporated and the radiance at the aperture (watts/(cm² sr)) is converted to an irradiance at the focal plane (watts/cm²). This acceptance angle is defined via the G# (see Schott), which is a function of the focal length, aperture diameter and aperture throughput (geometric and optical).

The options for aperture are defined in the <properties> element of the <instrument> and in the Advanced tab of the Instrument Editor:

The F# of the system is then defined as:

\begin{equation} F\# = \frac{L_f}{D_A} \end{equation}

where L_f is the focal length and D_A is the aperture diameter. The G# is then defined as:

\begin{equation} G\# = \frac{1 + (4 \cdot F\#^2)}{\tau_A \pi} \end{equation}

with units of per steradian (1/sr).

The at focal plane irradiance (E) can then be computed from the at aperture radiance (L) as:

\begin{equation} E = \frac{L}{G\#} \end{equation}

A change in quantity from radiance to irradiance results in a change of units as well as the per steradian term has been removed:

For example, an at aperture radiance in watts/(cm² sr) will now be a at focal plane irradiance in watts/cm².

Enabling "temporal integration" will integrate the seconds in the flux term resulting in (depending on the flux term settings):

The options for temporal integration are defined in the <capturemethod> element of a <focalplane> and in the Clock tab of the Focal Plane Editor:

It is not uncommon for spectrometers to output units in spectral radiance since the data is usually treated as a spectrum. In this case, the signal onto the detectors is spectrally integrated by the individual channel but the data delivered to users is calibrated and normalized by the integral of the channel.

\begin{equation} L_C = \frac{\int L_i(\lambda) R_C(\lambda) d\lambda}{\int R_C(\lambda) d\lambda} \end{equation}

where L_C is the spectral radiance for a channel, L_i is the incident spectral radiance and R_C is the spectral response of the channel at the wavelength. Hence, the spectral integration (numerator) removes the per wavelength units but the normalization (denominator) adds it back in.

If the internal detector model is configured, the output units are simply in digital counts (DCs).

To assist in understand the interaction of the various units options, the following table is provided with a subset of possible combinations:

The parameters gather by the various truth collectors are stored into a 3-D image cube, where each "band" in the image contains information about a specific image formation parameter.

The truth collectors can be configured in the user interface, or manually within the .platform file. Similar to the options for output image files produced by the focal plane capture methods, the user can specify the output file "schedule", which allows for:

The XML schema for the truth collection in the .platform file is as follows:

where each focal plane can have one or more <truthcollection> elements configured. The <collectorlist> element contains one or more <collector> elements that configure truth collectors by providing either an individual collector name (e.g., materialindex) or the name of a truth set (e.g., material).

It should be noted that some truth collectors include additional variables. For example, the abundance collector takes either a material ID (label) or a geometry tag.

In the material abundance mode, the <materials> element is provided and contains a comma separated list of material IDs for target materials for which sub-pixel abundances will be collected:

In the tagged geometry abundance mode, the <tags> element is provided and contains a comma separated list of tags for target geometries for which sub-pixel abundances will be collected:

The DIRSIG5 radiance convergence algorithm powers the adaptive sampling approach to the simulation solution. It allows pixels that are less complex spatially, spectrally or temporally to use fewer samples (paths), and conversely more complex pixels to use more samples (paths). The algorithm initializes itself with an estimate of the pixel radiance using a minimum number of paths (N_min). It then incrementally adds more paths to the radiance solution and compares the radiance at the reference wavelength from the previous N-1 sample solution to the current N sample solution. If the radiance difference is greater than the threshold, then another sample is added to the solution until the threshold is satisfied or the maximum number of samples (N_max) is reached.

One of the failure corner cases for this algorithm is when there is a significant sub-pixel contribution that is missed by the minimum number of samples. For example, a small object within a pixel that is a major contribution to the pixel due to it having a high reflectance or a high temperature. If the object is small (in terms of area), it could easily be missed by a low number of minimum samples (N_min). Hence, the solution could converge after only N_min samples with all the samples having randomly sampled the portion of the pixel represented by something other than this small but important object. This corner case can also manifest with larger targets that contain a small but significant component. For example, a small chrome feature on a car that reflects a solar glint in the reflective region or a hot exhaust pipe in the thermal region. Increasing the minimum number of samples would increase the likelihood that such small but important contributors are not missed, but increasing the minimum number of samples for all the pixels can have a dramatic impact of run-time.

Ideally, we want a way to automatically identify specific pixels that contain these corner cases and utilize more samples in those pixels only. Although it is not automatic, this can be accomplished by using the hypersampling feature that allows the simulation to dynamically increase the N_min and N_max samples used in the convergence algorithm for select pixels. To utilize the feature, the user needs to label important targets and specify a sampling multiplier that increases the sample rates when any of the labeled targets are within a pixel. When a pixel IFOV overlaps these labeled targets, the N_min and N_max values will be increased by a user-defined multiplier.

The ideal pinhole camera approximation employed by this plugin can be enhanced via the use of the built-in lens distortion model. This common 5-parameter distortion model allows the user to introduce common distortions including barrel and pin cushion. The governing equation has 3 radial terms (k₁, k₂ and k₃) and 2 tangential terms (p₁ and p₂) that define the transform of an undistorted point (x,y) on the focal plane into a distorted point (x',y').

\begin{eqnarray} x' &=& x \cdot (1 + k_1 r^2 + k_2 r^4 + k_3 r^6) + 2p_1 xy + p_2(r^2 + 2x^2)\\ y' &=& y \cdot (1 + k_1 r^2 + k_2 r^4 + k_3 r^6) + 2p_2 xy + p_1(r^2 + 2y^2) \end{eqnarray}

Units for x and y are in focal lengths and centered about the principal point of the instrument.

Since DIRSIG is a reverse ray-tracing approach, the plugin is performing the inverse of this mapping to compute the appropriate rays to sample the correct regions of the object plane. It should be noted that these equations are not invertable, so the inversion is performed using an iterative solution to find the undistorted point (x,y) from a distorted point (x',y') on the focal plane.

The 5-parameter model can be configured via the GUI:

The distortion model can also be configured via the <distortion> element insider the <properties> of a given instrument.

The parameter set above produces a pronounced "pincushion" distortion when looking at a checkerboard pattern.

See the LensDistortion1 demo for a working example.

Time-delayed integration (TDI) is a strategy to increase signal-to-noise ratio (SNR) by effectively increasing the integration time for each pixel. Rather than using a single pixel with a long integration time, TDI uses a set of pixels with a short integration time that will image the same location over a period of time. This is usually utilized in a pushbroom collection system, where a 1D array is scanned across a scene using platform motion to construct the 2nd dimension of the image. TDI is typically accomplished by using a 2D array in pushbroom mode and using the 2nd dimension of that 2D array as TDI "stages" that will be used to re-image the same location as the system as the array is scanned by the platform in the along-track dimension. The figure below illustrates this concept ^[1].

In order to use the TDI feature, you need to have the following components setup in the <focalplane> configuration in the .platform file:

Below is an example excerpt from a .platform file that has these components configured. In this case, the array has 12 stages of TDI (see the <yelementcount> element.

See the TimeDelayedIntegration1 demo for a working example.

As an alternative to assigning a focal plane one (or more) channels that are modeled at every pixel, the option also exists to generate mosaiced output of a color filter array (CFA) that can be demosaiced as a post processing step. The currently supported CFA patterns include the Bayer and TrueSense patterns. These are configured by adding the <channelpattern> section to the <spectralresponse> for a given focal plane. Each channel pattern has a list of pre-defined "filters" that must be mapped to the names of channels defined in the <channellist>. Examples for each pattern are shown in the following sections.

The user can supply an "active area mask" for all the pixels in the array via a gray scale image file. The brightness of the pixels are assumed to convey the relative sensitivity across the pixel area. A mask pixel value of 0 translates to zero sensitivity and a mask pixel value of 255 translated to a unity sensitivity. Hence, areas of the pixel can be masked off to model front-side readout electronics or to change the shape of the pixel (e.g. circular, diamond, etc.). The mask is used to override the uniform pixel sample with an importance based approach. Therefore, the active area image can contain gray values (values between 0 and 255) that indicate the pixel has reduced sensitivity in that specific area.

The active area mask is supplied via the <activearea> XML element within the <focalplane> element (see example below):

The <image> element specifies the name of the 8-bit image file (JPG, PNG, TIFF, etc.) used to drive the pixel sampling.

Note that this pixel area sampling can be combined with the PSF feature described here.

The convergence parameters used to define how many paths/pixel are used is typically overridden at the simulation level via the command-line --convergence option. This command-line option sets the default convergence parameters used for every capture in every sensor plugin defined in the simulation. However, when multiple sensors, multiple focal planes, etc. are included in a single simulation, there can be a desire to not want the same convergence parameters for every capture. For example, defining unique parameters for each focal plane array would be appropriate in cases where you have arrays that are capturing dramatically different wavelength regions (e.g., visible vs. long-wave infrared) and the magnitude of the sensed radiance differs or the noise equivalent radiance of the detectors differs. It is also handy when defining something like a low-light sensor that will always be used in low radiance magnitude conditions.

By default, the BasicPlatform plugin is producing an at aperture radiometric data product that is by default in spectrally integrated radiance units of watts/(cm² sr). Depending on the various options set, these units can change. For example:

The default output radiometric data products are provided so that users can externally perform modeling of how specific optical systems, detectors and read-out electronics would capture the inbound photons and produce a measured digital count.

For users that do not wish to perform this modeling externally, the internal detector model described here provides the user with the option to produce output data that is in digital counts and features Shot (arrival) noise, dark current noise, read noise and quantization. The general flow of the calculations is as follows:

The internal detector model must be configured by manually editing the .platform file to add the <detectormodel> element and few other elements. The following .platform file excerpt an be used as a template:

The user can currently describe the modulation transfer function (MTF) of the optical system via a single Point Spread Function (PSF). This function describes the spread of a point (zero area) in the object plane onto the focal plane after passing through the optical system.

Note that a single PSF is provided and hence, this is currently not a position dependent PSF. It should also be noted that the PSF is wavelength constant for the wavelengths captured by the respective focal plane (different PSF configurations can be assigned to different focal planes).

This feature is enabled by manually editing a DIRSIG4 .platform file and injecting the <psf> XML element in the <focalplane> element. The PSF is used in a 2-step importance sampling scheme to emulate the convolution of the PSF with the pixel area. In general, more samples per pixel are required to faithfully reproduce the impacts of the PSF on the output. Not that for PSF that is highly structured and/or very wide (spans many pixels), the maximum number of paths per pixel might need to be increased (see the DIRSIG5 manual for details).

There is also an ability to have this plugin launch an external process to perform operations on the output image. The <processing> element can be added to each focal plane (inside the <capturemethod> element) and can include one or more "tasks" to be performed on the supplied schedule.

The schedule options include:

The example below is meant to demonstrate how a (fictitious) program called demosaic can be run at the end of each capture. The options (provided in order by the <argument> elements) are specific to the program being executed:

The special string $$IMG_BASENAME$$ represents the radiometric image file being generated. In this example, the <imagefile> indicates that a file with the basename of truesense and a file extension of .img will be produced for each capture, resulting in filenames such as truesense_t0000-c0000.img, truesense_t0000-c0001.img, etc. When this processing task is triggered after the first capture, the $$IMG_BASENAME$$ string will be replaced by the string truesense_t0000-c0000. Subsequent processing calls will be automatically called with the appropriate filename that is changing from capture to capture.

The <removefile> option allows you to request that the current file produced by DIRSIG be removed after the processing task is complete. In the case of the example above, after the DIRSIG image has been demosaiced, the original mosaiced image could be removed (to save disk space, etc.). The default for this option is false.

For a more complex example, consider wanting to automatically generate a video file from a simulation. In this example, we will use the ffmpeg utility to encode a sequence of individual frames into a video. To achieve this, we will use the "file per capture" output file schedule (to produce a separate image file for each capture frame) and then define two processing tasks:

While the simulation is running, the <message> will be displayed when each processing task is executed:

An example of this video encoding scenario is included in the TrackingMount1 demo (see the encode.platform and encode.sim files).

This sensor plugin is typically used to model a camera that is focused at inifinity, where all rays pays through a single focus point. However, the user can optional specify a finite focus distance in a camera modeled by the plugin, which triggers an advanced sampling of the user-defined aperture. The variable that enables this feature is the <focusdistance> in the instrument’s <properties> element. In order to enable this feature, the <aperturediameter> must also be defined.

A working example of this feature is provided in the FocusDistance1 demo.