Most applications of multispectral imaging are explicitly or implicitly dependent on the dimensionality and topology of the spectral mixing space. Mixing space characterization refers to the identification of salient properties of the set of pixel reflectance spectra comprising an image (or compilation of images). The underlying premise is that this set of spectra may be described as a low dimensional manifold embedded in a high dimensional vector space. Traditional mixing space characterization uses the linear dimensionality reduction offered by Principal Component Analysis to find projections of pixel spectra onto orthogonal linear subspaces, prioritized by variance. Here, we consider the potential for recent advances in nonlinear dimensionality reduction (specifically, manifold learning) to contribute additional useful information for multispectral mixing space characterization. We integrate linear and nonlinear methods through a novel approach called Joint Characterization (JC). JC is comprised of two components. First, spectral mixture analysis (SMA) linearly projects the high-dimensional reflectance vectors onto a 2D subspace comprising the primary mixing continuum of substrates, vegetation, and dark features (e.g., shadow and water). Second, manifold learning nonlinearly maps the high-dimensional reflectance vectors into a low-D embedding space while preserving manifold topology. The SMA output is physically interpretable in terms of material abundances. The manifold learning output is not generally physically interpretable, but more faithfully preserves high dimensional connectivity and clustering within the mixing space. Used together, the strengths of SMA may compensate for the limitations of manifold learning, and vice versa. Here, we illustrate JC through application to thematic compilations of 90 Sentinel-2 reflectance images selected from a diverse set of biomes and land cover categories. Specifically, we use globally standardized Substrate, Vegetation, and Dark (S, V, D) endmembers (EMs) for SMA, and Uniform Manifold Approximation and Projection (UMAP) for manifold learning. The value of each (SVD and UMAP) model is illustrated, both separately and jointly. JC is shown to successfully characterize both continuous gradations (spectral mixing trends) and discrete clusters (land cover class distinctions) within the spectral mixing space of each land cover category. These features are not clearly identifiable from SVD fractions alone, and not physically interpretable from UMAP alone. Implications are discussed for the design of models which can reliably extract and explainably use high-dimensional spectral information in spatially mixed pixels—a principal challenge in optical remote sensing.
NASA’s Earth Surface Mineral Dust Source Investigation (EMIT) mission seeks to use spaceborne imaging spectroscopy (hyperspectral imaging) to map the mineralogy of arid dust source regions. Here we apply recent developments in Joint Characterization (JC) and the spectral Mixture Residual (MR) to explore the information content of data from this novel mission. Specifically, for a mosaic of 20 spectrally diverse scenes, we find: (1) a generalized three-endmember (Substrate, Vegetation, Dark; SVD) spectral mixture model is capable of capturing the preponderance (99% in three dimensions) of spectral variance with low misfit (99% pixels with <3.7% RMSE); (2) manifold learning (UMAP) is capable of identifying spatially coherent, physically interpretable clustering relationships in the spectral feature space; (3) UMAP yields results that are at least as informative when applied to the MR as when applied to raw reflectance; (4) SVD fraction information usefully contextualizes UMAP clustering relationships, and vice-versa (JC); and (5) when EMIT data are convolved to spectral response functions of multispectral instruments (Sentinel-2, Landsat 8/9, Planet SuperDove), SVD fractions correlate strongly across sensors, but UMAP clustering relationships for the EMIT hyperspectral feature space are far more informative than for simulated multispectral sensors. Implications are discussed for both the utility of EMIT data in the near-term and for the potential of high signal-to-noise (SNR) spaceborne imaging spectroscopy more generally, to transform the future of optical remote sensing in the years and decades to come.
more » « less- Award ID(s):
- 2226649
- PAR ID:
- 10497003
- Publisher / Repository:
- MDPI
- Date Published:
- Journal Name:
- Remote Sensing
- Volume:
- 15
- Issue:
- 9
- ISSN:
- 2072-4292
- Page Range / eLocation ID:
- 2295
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
-
more » « less
A geologic map is both a visual depiction of the lithologies and structures occurring at the Earth’s surface and a representation of a conceptual model for the geologic history in a region. The work needed to capture such multifaced information in an accurate geologic map is time consuming. Remote sensing can complement traditional primary field observations, geochemistry, chronometry, and subsurface geophysical data in providing useful information to assist with the geologic mapping process. Two novel sources of remote sensing data are particularly relevant for geologic mapping applications: decameter-resolution imaging spectroscopy (spectroscopic imaging) and meter-resolution multispectral shortwave infrared (SWIR) imaging. Decameter spectroscopic imagery can capture important mineral absorptions but is frequently unable to spatially resolve important geologic features. Meter-resolution multispectral SWIR images are better able to resolve fine spatial features but offer reduced spectral information. Such disparate but complementary datasets can be challenging to integrate into the geologic mapping process. Here, we conduct a comparative analysis of spatial and spectral scaling for two such datasets: one Airborne Visible/Infrared Imaging Spectrometer—Classic (AVIRIS-classic) flightline, and one WorldView-3 (WV3) scene, for a geologically complex landscape in Anza-Borrego Desert State Park, California. To do so, we use a two-stage framework that synthesizes recent advances in the spectral mixture residual and joint characterization. The mixture residual uses the wavelength-explicit misfit of a linear spectral mixture model to capture low variance spectral signals. Joint characterization utilizes nonlinear dimensionality reduction (manifold learning) to visualize spectral feature space topology and identify clusters of statistically similar spectra. For this study area, the spectral mixture residual clearly reveals greater spectral dimensionality in AVIRIS than WorldView (99% of variance in 39 versus 5 residual dimensions). Additionally, joint characterization shows more complex spectral feature space topology for AVIRIS than WorldView, revealing information useful to the geologic mapping process in the form of mineralogical variability both within and among mapped geologic units. These results illustrate the potential of recent and planned imaging spectroscopy missions to complement high-resolution multispectral imagery—along with field and lab observations—in planning, collecting, and interpreting the results from geologic field work.
-
more » « less
The monitoring of agronomic parameters like biomass, water stress, and plant health can benefit from synergistic use of all available remotely sensed information. Multispectral imagery has been used for this purpose for decades, largely with vegetation indices (VIs). Many multispectral VIs exist, typically relying on a single feature—the spectral red edge—for information. Where hyperspectral imagery is available, spectral mixture models can use the full VSWIR spectrum to yield further insight, simultaneously estimating area fractions of multiple materials within mixed pixels. Here we investigate the relationships between VIs and mixture models by comparing hyperspectral endmember fractions to six common multispectral VIs in California’s diverse crops and soils. In so doing, we isolate spectral effects from sensor- and acquisition-specific variability associated with atmosphere, illumination, and view geometry. Specifically, we compare: (1) fractional area of photosynthetic vegetation (Fv) from 64,000,000 3–5 m resolution AVIRIS-ng reflectance spectra; and (2) six popular VIs (NDVI, NIRv, EVI, EVI2, SR, DVI) computed from simulated Planet SuperDove reflectance spectra derived from the AVIRIS-ng spectra. Hyperspectral Fv and multispectral VIs are compared using both parametric (Pearson correlation, ρ) and nonparametric (Mutual Information, MI) metrics. Four VIs (NIRv, DVI, EVI, EVI2) showed strong linear relationships with Fv (ρ > 0.94; MI > 1.2). NIRv and DVI showed strong interrelation (ρ > 0.99, MI > 2.4), but deviated from a 1:1 correspondence with Fv. EVI and EVI2 were strongly interrelated (ρ > 0.99, MI > 2.3) and more closely approximated a 1:1 relationship with Fv. In contrast, NDVI and SR showed a weaker, nonlinear, heteroskedastic relation to Fv (ρ < 0.84, MI = 0.69). NDVI exhibited both especially severe sensitivity to unvegetated background (–0.05 < NDVI < +0.6) and saturation (0.2 < Fv < 0.8 for NDVI = 0.7). The self-consistent atmospheric correction, radiometry, and sun-sensor geometry allows this simulation approach to be further applied to indices, sensors, and landscapes worldwide.
-
Hyperspectral imaging systems are becoming widely used due to their increasing accessibility and their ability to provide detailed spectral responses based on hundreds of spectral bands. However, the resulting hyperspectral images (HSIs) come at the cost of increased storage requirements, increased computational time to process, and highly redundant data. Thus, dimensionality reduction techniques are necessary to decrease the number of spectral bands while retaining the most useful information. Our contribution is two-fold: First, we propose a filter-based method called interband redundancy analysis (IBRA) based on a collinearity analysis between a band and its neighbors. This analysis helps to remove redundant bands and dramatically reduces the search space. Second, we apply a wrapper-based approach called greedy spectral selection (GSS) to the results of IBRA to select bands based on their information entropy values and train a compact convolutional neural network to evaluate the performance of the current selection. We also propose a feature extraction framework that consists of two main steps: first, it reduces the total number of bands using IBRA; then, it can use any feature extraction method to obtain the desired number of feature channels. We present classification results obtained from our methods and compare them to other dimensionality reduction methods on three hyperspectral image datasets. Additionally, we used the original hyperspectral data cube to simulate the process of using actual filters in a multispectral imager.more » « less
-
Pixel-level fusion of satellite images coming from multiple sensors allows for an improvement in the quality of the acquired data both spatially and spectrally. In particular, multispectral and hyperspectral images have been fused to generate images with a high spatial and spectral resolution. In literature, there are several approaches for this task, nonetheless, those techniques still present a loss of relevant spatial information during the fusion process. This work presents a multi scale deep learning model to fuse multispectral and hyperspectral data, each with high-spatial-and-low-spectral resolution (HSaLS) and low-spatial-and-high-spectral resolution (LSaHS) respectively. As a result of the fusion scheme, a high-spatial-and-spectral resolution image (HSaHS) can be obtained. In order of accomplishing this result, we have developed a new scalable high spatial resolution process in which the model learns how to transition from low spatial resolution to an intermediate spatial resolution level and finally to the high spatial-spectral resolution image. This step-by-step process reduces significantly the loss of spatial information. The results of our approach show better performance in terms of both the structural similarity index and the signal to noise ratio.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.3390/rs15092295
