NEF BRDF Extraction Tutorial

The NEF database contains bi-directional reflectance function (BRDF) and bi-directional transmittance function (BTDF) data for a large number of materials. Most of these material descriptions were created by fitting lab BRDF/BTDF measurements to a variety of parametric models, including the standard Hapke model and the "modified" Beard-Maxwell (MBM) model. Previous DIRSIG4 integrations with the NEF involved duplicating the code for the Hapke model and the NEF’s unique MBM model as native BRDF models in DIRSIG and importing the corresponding parameters from the NEF file-based material descriptions.

When NEF version 12 was released, it reflected a major software overhaul effort, which included the migration from a file-based database to an SQL database. The updated NEF database and utility software now included a Java-based application programming interface (API) — the same API used by the graphical user interface (GUI) provided with software. The API included tools to compute the BRDF for a defined source and view geometry. However, using the API was too slow for DIRSIG4 to leverage for run-time calculations. Although the API could disclose which BRDF model is used for a given material, it did not provide direct access to the BRDF model parameters which could be used by DIRSIG4’s existing implementations of Hapke and Modified Beard-Maxwell. Without a computationally efficient way to perform BRDF calculations at run-time, support for the NEF database in DIRSIG4 came to an end.

When DIRSIG5 was created, one of the key design decisions was to avoid developing, validating and maintaining dozens of BRDF model implementations in the new code base. In addition, we desired a workflow that could ingest measured BRDF data more efficiently (potentially without employing a parametric model for fitting). And finally, DIRSIG5 would need more than just the BRDF "evaluation" calculation (e.g. the BRDF for a given source and view geometry), it would need efficient ways to perform two additional tasks:

These requirements resulted in the development of Spherical Quad Trees (SQTs) in DIRSIG5. The SQT mechanism solved all of these requirements by utilizing a data-driven representation of hemispherical or spherical data that could be quickly evaluated, integrated and sampled. A toolbox of utility programs were created to import data into the SQT formatted files, and extract data from SQT formatted files for data visualization and evaluation. In DIRSIG5, all parameterized BRDF models are angularly sampled and then imported into this data-centric SQT format for use at run-time. A key tool in the SQT data workflow is the raw2sqt tool, which imports angularly sampled spectral data from a documented, ASCII/Text file and optimizes into a binary SQT file. Since the Java-based NEF API provided a way to sample any BRDF model in the database and write it into one of these ASCII/Text intermediate files, an integration mechanism was now possible once again.

The nef2raw tool used in this tutorial uses the NEF’s Java-based API to extract two components of the spectral-angular material description from a material in the NEF database:

Although the API supports evaluating the BRDF at any wavelength, the reality is that underlying description tends to have model parameters at no more than 5 wavelengths: the UV, VIS, NIR, MWIR and LWIR. This reflects that fact that although the DHR can vary rapidly by wavelength, the shape of the BRDF generally does not. Extracting the full BRDF at 1000’s of wavelengths would result in a lot of data to be stored. Hence, the approach in the nef2raw tool is to extract the BRDF at 23 wavelengths spread across the UV, VIS, NIR, SWIR, MWIR and LWIR regions. In addition, the DHR is extracted spectrally and fused with the BRDF in a similar manner as the built-in NEF calculations. As a result we have found that most materials can be captured in a data representation that only requires 2-10 MB per material.

Below is an outline of the workflow:

To use this workflow, the following software components are required to be installed:

The "working folder" is the folder where the workflow will run. The location of this folder is not important as long as the user has write permissions for this folder. For the sake of this tutorial, we created a folder named nef2raw in the user’s desktop folder.

Since you will need to modify the script for each material to extract, you will need to copy the nef2raw.java and nef2raw.bat file from the DIRSIG installation extras folder into your working folder. The nef2raw.java file should not need to be updated, but the nef2raw.bat file needs to be updated to (1) specify the path to your NEF installation and (2) change the material name (identifier) to be extracted. An example nef2raw.bat file is show below. The first line (@echo off) disables printing the command being executed to the output. The second line (starting with javac -cp) compiles the Java program for running. The third and final line (starting with java -cp) runs the program with the NEF "parameters" file as the first argument and the name of the material to be extracted as the second argument:

You will need to edit the nef2raw.bat file to reflect the path to your NEF installation. Make sure the example path (e.g. C:\NEFDS16110F_PC ) is updated to reflect the path to the NEF installation for your computer. Furthermore, make sure the name of the "properties" file (in the Properties subfolder) reflects the version installed.

In the nef2raw.bat file, the third (final) command (starts with java -cp) is the one that runs the extraction program, and the 2nd (final) argument is the NEF name (identifier) of the material to be extracted. For this tutorial, we want to extract two materials: (1) a background material and (2) a target material. This could be accomplished by running the script as-is once for one material, editing the script to change the material and then running the script again for the other material. However, a simpler approach is to simply repeat the 2nd command for each material we want to extract.

For this tutorial, we will extract the following two materials:

Hence, the updated script looks like:

With the script updated, the script can now be run to perform the extraction from the database:

After the extraction has run, your working folder should contain .raw and .dhr file pairs for both materials.

The .dhr files contain the extracted DHR for the two materials. These two curves are plotted below to show the differences in the magnitude between the bright concrete and dark black paint:

The next step is to convert the ASCII/Text "raw" BRDF measurements into optimized, binary SQT files. This is performed by running the raw2sqt utility included with the DIRSIG software:

The last step is to update the Brdf2 demo scene to use these two extracted materials. If you have not already downloaded that demo, then do so now. The remainder of this tutorial will assume you extracted the demo into the Brdf2 folder in your desktop folder.

Graphically editing the material file to add the SQT reflectance property is a forthcoming feature. For now, the best approach is to edit the material file in a text editor to make the required changes. For this tutorial, we will be updating both materials in the scene. The first material (ID = 105) is the material applied to the "ground" and the second material (ID = test_mat) is applied to the vehicle. To make the changes, the following edits need to be made for each material:

The example below shows the edits made to the 105 (ground) material:

The test_mat material should be similarly updated, but assigned the corresponding .sqt and .dhr filenames for the 1455UUUPNT material:

With the scene updates completed, the scene can be run. For DIRSIG5, recompile the scene with scene2hdf and then run the simulation:

The image below shows the output of the updated simulation. It should be noted that the simulation was updated to use a 640 x 480 camera (versus the original 320 x 240 camera) and the image below was created using a gamma = 2.0 scaling (e.g. envitopnm --autoscale=gamma --gamma=2 ...). Note the concrete color of the background and the glossy character of the black paint on the target vehicle:

The image below is a similar rendering of the original Brdf2 material configuration: