skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Differentiable Programming for Hyperspectral Unmixing using a Physics-based Dispersion Model
Hyperspectral unmixing is an important remote sensing task with applications including material identification and analysis. Characteristic spectral features make many pure materials identifiable from their visible-to-infrared spectra, but quantifying their presence within a mixture is a challenging task due to nonlinearities and factors of variation. In this paper, spectral variation is considered from a physics-based approach and incorporated into an end-to-end spectral unmixing algorithm via differentiable programming. The dispersion model is introduced to simulate realistic spectral variation, and an efficient method to fit the parameters is presented. Then, this dispersion model is utilized as a generative model within an analysis-by-synthesis spectral unmixing algorithm. Further, a technique for inverse rendering using a convolutional neural network to predict parameters of the generative model is introduced to enhance performance and speed when training data is available. Results achieve state-of-the-art on both infrared and visible-to-near-infrared (VNIR) datasets, and show promise for the synergy between physicsbased models and deep learning in hyperspectral unmixing in the future.  more » « less
Award ID(s):
1909192
PAR ID:
10193714
Author(s) / Creator(s):
; ; ; ; ;
Date Published:
Journal Name:
European Conference on Computer Vision (ECCV)
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; (, Remote Sensing)
    null (Ed.)
    Alaska has witnessed a significant increase in wildfire events in recent decades that have been linked to drier and warmer summers. Forest fuel maps play a vital role in wildfire management and risk assessment. Freely available multispectral datasets are widely used for land use and land cover mapping, but they have limited utility for fuel mapping due to their coarse spectral resolution. Hyperspectral datasets have a high spectral resolution, ideal for detailed fuel mapping, but they are limited and expensive to acquire. This study simulates hyperspectral data from Sentinel-2 multispectral data using the spectral response function of the Airborne Visible/Infrared Imaging Spectrometer-Next Generation (AVIRIS-NG) sensor, and normalized ground spectra of gravel, birch, and spruce. We used the Uniform Pattern Decomposition Method (UPDM) for spectral unmixing, which is a sensor-independent method, where each pixel is expressed as the linear sum of standard reference spectra. The simulated hyperspectral data have spectral characteristics of AVIRIS-NG and the reflectance properties of Sentinel-2 data. We validated the simulated spectra by visually and statistically comparing it with real AVIRIS-NG data. We observed a high correlation between the spectra of tree classes collected from AVIRIS-NG and simulated hyperspectral data. Upon performing species level classification, we achieved a classification accuracy of 89% for the simulated hyperspectral data, which is better than the accuracy of Sentinel-2 data (77.8%). We generated a fuel map from the simulated hyperspectral image using the Random Forest classifier. Our study demonstrated that low-cost and high-quality hyperspectral data can be generated from Sentinel-2 data using UPDM for improved land cover and vegetation mapping in the boreal forest. 
    more » « less
  2. ; ; (, Scientific Reports)
    Abstract Compared with traditional Fabry–Perot optical filters, plasmonic color filters could greatly remedy the complexity and reduce the cost of manufacturing. In this paper we present end-to-end demonstration of visible light spectroscopy based on highly selective plasmonic color filter array based on resonant grating structure. The spectra of 6 assorted samples were measured using an array of 20 narrowband color filters and detected signals were used to reconstruct original spectra by using new unmixing algorithm and by solving least squares problem with smoothing regularization. The original spectra were reconstructed with less than 0.137 root mean squared error. This works shows promise towards fully integrating plasmonic color filter array in imagers used in hyperspectral cameras. 
    more » « less
  3. ; ; ; ; ; ; ; (, Plant Methods)
    Abstract Background Remote sensing instruments enable high-throughput phenotyping of plant traits and stress resilience across scale. Spatial (handheld devices, towers, drones, airborne, and satellites) and temporal (continuous or intermittent) tradeoffs can enable or constrain plant science applications. Here, we describe the technical details of TSWIFT (Tower Spectrometer on Wheels for Investigating Frequent Timeseries), a mobile tower-based hyperspectral remote sensing system for continuous monitoring of spectral reflectance across visible-near infrared regions with the capacity to resolve solar-induced fluorescence (SIF). Results We demonstrate potential applications for monitoring short-term (diurnal) and long-term (seasonal) variation of vegetation for high-throughput phenotyping applications. We deployed TSWIFT in a field experiment of 300 common bean genotypes in two treatments: control (irrigated) and drought (terminal drought). We evaluated the normalized difference vegetation index (NDVI), photochemical reflectance index (PRI), and SIF, as well as the coefficient of variation (CV) across the visible-near infrared spectral range (400 to 900 nm). NDVI tracked structural variation early in the growing season, following initial plant growth and development. PRI and SIF were more dynamic, exhibiting variation diurnally and seasonally, enabling quantification of genotypic variation in physiological response to drought conditions. Beyond vegetation indices, CV of hyperspectral reflectance showed the most variability across genotypes, treatment, and time in the visible and red-edge spectral regions. Conclusions TSWIFT enables continuous and automated monitoring of hyperspectral reflectance for assessing variation in plant structure and function at high spatial and temporal resolutions for high-throughput phenotyping. Mobile, tower-based systems like this can provide short- and long-term datasets to assess genotypic and/or management responses to the environment, and ultimately enable the spectral prediction of resource-use efficiency, stress resilience, productivity and yield. 
    more » « less
  4. ; ; ; ; ; ; ; ; (, Proc. SPIE 10497, Imaging, Manipulation, and Analysis of Biomolecules, Cells, and Tissues XVI)
    Autofluorescence has historically been considered a nuisance in medical imaging. Many endogenous fluorophores, specifically, collagen, elastin, NADH, and FAD, are found throughout the human body. Diagnostically, these signals can be prohibitive since they can outcompete signals introduced for diagnostic purposes. Recent advances in hyperspectral imaging have allowed the acquisition of significantly more data in a shorter time period by scanning the excitation spectra of fluorophores. The reduced acquisition time and increased signal-to-noise ratio allow for separation of significantly more fluorophores than previously possible. Here, we propose to utilize excitation-scanning of autofluorescence to examine tissues and diagnose pathologies. Spectra of autofluorescent molecules were obtained using a custom inverted microscope (TE-2000, Nikon Instruments) with a Xe arc lamp and thin film tunable filter array (VersaChrome, Semrock, Inc.) Scans utilized excitation wavelengths from 360 nm to 550 nm in 5 nm increments. The resultant spectra were used to examine hyperspectral image stacks from various collaborative studies, including an atherosclerotic rat model and a colon cancer study. Hyperspectral images were analyzed with ENVI and custom Matlab scripts including linear spectral unmixing (LSU) and principal component analysis (PCA). Initial results suggest the ability to separate the signals of endogenous fluorophores and measure the relative concentrations of fluorophores among healthy and diseased states of similar tissues. These results suggest pathology-specific changes to endogenous fluorophores can be detected using excitationscanning hyperspectral imaging. Future work will expand the library of pure molecules and will examine more defined disease states. 
    more » « less
  5. ; ; ; ; ; (, Bioinformatics)
    Abstract MotivationSpectral unmixing methods attempt to determine the concentrations of different fluorophores present at each pixel location in an image by analyzing a set of measured emission spectra. Unmixing algorithms have shown great promise for applications where samples contain many fluorescent labels; however, existing methods perform poorly when confronted with autofluorescence-contaminated images. ResultsWe propose an unmixing algorithm designed to separate fluorophores with overlapping emission spectra from contamination by autofluorescence and background fluorescence. First, we formally define a generalization of the linear mixing model, called the affine mixture model (AMM), that specifically accounts for background fluorescence. Second, we use the AMM to derive an affine nonnegative matrix factorization method for estimating fluorophore endmember spectra from reference images. Lastly, we propose a semi-blind sparse affine spectral unmixing (SSASU) algorithm that uses knowledge of the estimated endmembers to learn the autofluorescence and background fluorescence spectra on a per-image basis. When unmixing real-world spectral images contaminated by autofluorescence, SSASU greatly improved proportion indeterminacy as compared to existing methods for a given relative reconstruction error. Availability and implementationThe source code used for this paper was written in Julia and is available with the test data at https://github.com/brossetti/ssasu. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email