skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Innovative data reduction and visualization strategy for hyperspectral imaging datasets using t-SNE approach
Abstract Visible hyperspectral imaging (HSI) is a fast and non-invasive imaging method that has been adapted by the field of conservation science to study painted surfaces. By collecting reflectance spectra from a 2D surface, the resulting 3D hyperspectral data cube contains millions of recorded spectra. While processing such large amounts of spectra poses an analytical and computational challenge, it also opens new opportunities to apply powerful methods of multivariate analysis for data evaluation. With the intent of expanding current data treatment of hyperspectral datasets, an innovative approach for data reduction and visualization is presented in this article. It uses a statistical embedding method known as t-distributed stochastic neighbor embedding (t-SNE) to provide a non-linear representation of spectral features in a lower 2D space. The efficiency of the proposed method for painted surfaces from cultural heritage is established through the study of laboratory prepared paint mock-ups, and medieval French illuminated manuscript.  more » « less
Award ID(s):
1743748
PAR ID:
10443554
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
Pure and Applied Chemistry
Volume:
90
Issue:
3
ISSN:
0033-4545
Page Range / eLocation ID:
493 to 506
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; (, Remote Sensing)
    null (Ed.)
    Alaska has witnessed a significant increase in wildfire events in recent decades that have been linked to drier and warmer summers. Forest fuel maps play a vital role in wildfire management and risk assessment. Freely available multispectral datasets are widely used for land use and land cover mapping, but they have limited utility for fuel mapping due to their coarse spectral resolution. Hyperspectral datasets have a high spectral resolution, ideal for detailed fuel mapping, but they are limited and expensive to acquire. This study simulates hyperspectral data from Sentinel-2 multispectral data using the spectral response function of the Airborne Visible/Infrared Imaging Spectrometer-Next Generation (AVIRIS-NG) sensor, and normalized ground spectra of gravel, birch, and spruce. We used the Uniform Pattern Decomposition Method (UPDM) for spectral unmixing, which is a sensor-independent method, where each pixel is expressed as the linear sum of standard reference spectra. The simulated hyperspectral data have spectral characteristics of AVIRIS-NG and the reflectance properties of Sentinel-2 data. We validated the simulated spectra by visually and statistically comparing it with real AVIRIS-NG data. We observed a high correlation between the spectra of tree classes collected from AVIRIS-NG and simulated hyperspectral data. Upon performing species level classification, we achieved a classification accuracy of 89% for the simulated hyperspectral data, which is better than the accuracy of Sentinel-2 data (77.8%). We generated a fuel map from the simulated hyperspectral image using the Random Forest classifier. Our study demonstrated that low-cost and high-quality hyperspectral data can be generated from Sentinel-2 data using UPDM for improved land cover and vegetation mapping in the boreal forest. 
    more » « less
  2. ; ; (, Nature Communications)
    Abstract Two-dimensional (2D) embedding methods are crucial for single-cell data visualization. Popular methods such as t-distributed stochastic neighbor embedding (t-SNE) and uniform manifold approximation and projection (UMAP) are commonly used for visualizing cell clusters; however, it is well known that t-SNE and UMAP’s 2D embeddings might not reliably inform the similarities among cell clusters. Motivated by this challenge, we present a statistical method, scDEED, for detecting dubious cell embeddings output by a 2D-embedding method. By calculating a reliability score for every cell embedding based on the similarity between the cell’s 2D-embedding neighbors and pre-embedding neighbors, scDEED identifies the cell embeddings with low reliability scores as dubious and those with high reliability scores as trustworthy. Moreover, by minimizing the number of dubious cell embeddings, scDEED provides intuitive guidance for optimizing the hyperparameters of an embedding method. We show the effectiveness of scDEED on multiple datasets for detecting dubious cell embeddings and optimizing the hyperparameters of t-SNE and UMAP. 
    more » « less
  3. ; ; ; ; ; ; (, Geosciences)
    Understanding the mineralogy and geochemistry of the subsurface is key when assessing and exploring for mineral deposits. To achieve this goal, rapid acquisition and accurate interpretation of drill core data are essential. Hyperspectral shortwave infrared imaging is a rapid and non-destructive analytical method widely used in the minerals industry to map minerals with diagnostic features in core samples. In this paper, we present an automated method to interpret hyperspectral shortwave infrared data on drill core to decipher major felsic rock-forming minerals using supervised machine learning techniques for processing, masking, and extracting mineralogical and textural information. This study utilizes a co-registered training dataset that integrates hyperspectral data with quantitative scanning electron microscopy data instead of spectrum matching using a spectral library. Our methodology overcomes previous limitations in hyperspectral data interpretation for the full mineralogy (i.e., quartz and feldspar) caused by the need to identify spectral features of minerals; in particular, it detects the presence of minerals that are considered invisible in traditional shortwave infrared hyperspectral analysis. 
    more » « less
  4. ; ; (, SPIE Proceedings International Conference on Space Optics)
    Minoglou, Kyriaki; Karafolas, Nikos; Cugny, Bruno (Ed.)
    This paper describes Flare Sentinel, a compact integral field spectrograph (IFS) for the study of the hydrogen Balmer series spectrum from 350 to 450 nm of solar flare from space. Flare Sentinel IFS is based on a new Machined Image Slicer Compact Spectrograph Array (MICS) design. MICS consists of an image slicer that divides a continuous 2D spatial field formed by an imaging system into multiple narrow slices, and an array of miniature spectrographs, each forming the spectra of one of the slices of the 2D field. The spectra formed by all the miniature spectrographs can be projected on a common 2D focal plane to be recorded by an image sensors. The spectra can also be distributed to multiple focal planes and recorded simultaneously by multiple sensors to increase the instantaneous hyperspectral field of view of the instrument. New image slicers with slit width of 36 um and 20 um have been successfully fabricated using Canon Inc.’s ultra-precision diamond-cutting CNC mill. This capability is enabling design and fabrication of IFSs with imaging format of 102 × 102 , and spectral resolution between 100 < R < 10, 000 in a very compact package. We will present the optical design and the optical hardware of a prototype IFS that has been fabricated. 
    more » « less
  5. ; ; ; ; ; ; ; ; ; ; et al (, Soil Science Society of America Journal)
    Abstract Understanding soil organic carbon (SOC) response to global change has been hindered by an inability to map SOC at horizon scales relevant to coupled hydrologic and biogeochemical processes. Standard SOC measurements rely on homogenized samples taken from distinct depth intervals. Such sampling prevents an examination of fine‐scale SOC distribution within a soil horizon. Visible near‐infrared hyperspectral imaging (HSI) has been applied to intact monoliths and split cores surfaces to overcome this limitation. However, the roughness of these surfaces can influence HSI spectra by scattering reflected light in different directions posing challenges to fine‐scale SOC mapping. Here, we examine the influence of prescribed surface orientation on reflected spectra, develop a method for correcting topographic effects, and calibrate a partial least squares regression (PLSR) model for SOC prediction. Two empirical models that account for surface slope, aspect, and wavelength and two theoretical models that account for the geometry of the spectrometer were compared using 681 homogenized soil samples from across the United States that were packed into sample wells and presented to the spectrometer at 91 orientations. The empirical approach outperformed the more complex geometric models in correcting spectra taken at non‐flat configurations. Topographically corrected spectra reduced bias and error in SOC predicted by PLSR, particularly at slope angles greater than 30°. Our approach clears the way for investigating the spatial distributions of multiple soil properties on rough intact soil samples. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email