This work focuses on the problem of fusing a hyperspectral image (HSI) and a multispectral image (MSI) to produce a super-resolution image that admits high spatial and spectral resolutions. Existing algorithms are mostly based on joint low-rank factorization of the ma-tricized HSI and MSI. This framework is effective to some extent, but several challenges remain. First, it is unclear whether or not the super-resolution image is identifiable in theory under this framework, while identifiability usually plays an essential role in such estimation problems. Second, most algorithms assume that the degradation operators from the super-resolution image to the HSI and MSI are known or can be easily estimated - which is hardly true in practice. In this work, we propose a novel coupled tensor decomposition method that can effectively circumvent these issues. The proposed approach guarantees the identifiability of the super-resolution image under realistic conditions. The method can work even without knowing the spatial degradation operator, which could be hard to accurately estimate in practice. Simulations using AVIRIS Cuprite data are employed to demonstrate the effectiveness of the proposed approach.
more »
« less
Hyperspectral Super-Resolution: Combining Low Rank Tensor and Matrix Structure
Hyperspectral super-resolution refers to the task of fusing a hyperspectral image (HSI) and a multispectral image (MSI) in order to produce a super-resolution image (SRI) that has high spatial and spectral resolution. Popular methods leverage matrix factorization that models each spectral pixel as a convex combination of spectral signatures belonging to a few endmembers. These methods are considered state-of-the-art, but several challenges remain. First, multiband images are naturally three dimensional (3-d) signals, while matrix methods usually ignore the 3-d structure, which is prone to information losses. Second, these methods do not provide identifiability guarantees under which the reconstruction task is feasible. Third, a tacit assumption is that the degradation operators from SRI to MSI and HSI are known - which is hardly the case in practice. Recently [1], [2] proposed a coupled tensor factorization approach to handle these issues. In this work we propose a hybrid model that combines the benefits of tensor and matrix factorization approaches. We also develop a new algorithm that is mathematically simple, enjoys identifiability under relaxed conditions and is completely agnostic of the spatial degradation operator. Experimental results with real hyperspectral data showcase the effectiveness of the proposed approach.
more »
« less
- Award ID(s):
- 1704074
- NSF-PAR ID:
- 10106326
- Date Published:
- Journal Name:
- 2018 25th IEEE International Conference on Image Processing (ICIP)
- Page Range / eLocation ID:
- 3318 to 3322
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The data deluge comes with high demands for data labeling. Crowdsourcing (or, more generally, ensemble learning) techniques aim to produce accurate labels via integrating noisy, non-expert labeling from annotators. The classic Dawid-Skene estimator and its accompanying expectation maximization (EM) algorithm have been widely used, but the theoretical properties are not fully understood. Tensor methods were proposed to guarantee identification of the Dawid-Skene model, but the sample complexity is a hurdle for applying such approaches---since the tensor methods hinge on the availability of third-order statistics that are hard to reliably estimate given limited data. In this paper, we propose a framework using pairwise co-occurrences of the annotator responses, which naturally admits lower sample complexity. We show that the approach can identify the Dawid-Skene model under realistic conditions. We propose an algebraic algorithm reminiscent of convex geometry-based structured matrix factorization to solve the model identification problem efficiently, and an identifiability-enhanced algorithm for handling more challenging and critical scenarios. Experiments show that the proposed algorithms outperform the state-of-art algorithms under a variety of scenarios.more » « less
-
Combining a hyperspectral (HS) image and a multi-spectral (MS) image---an example of image fusion---can result in a spatially and spectrally high-resolution image. Despite the plethora of fusion algorithms in remote sensing, a necessary prerequisite, namely registration, is mostly ignored. This limits their application to well-registered images from the same source. In this article, we propose and validate an integrated registration and fusion approach (code available at https://github.com/zhouyuanzxcv/Hyperspectral). The registration algorithm minimizes a least-squares (LSQ) objective function with the point spread function (PSF) incorporated together with a nonrigid freeform transformation applied to the HS image and a rigid transformation applied to the MS image. It can handle images with significant scale differences and spatial distortion. The fusion algorithm takes the full high-resolution HS image as an unknown in the objective function. Assuming that the pixels lie on a low-dimensional manifold invariant to local linear transformations from spectral degradation, the fusion optimization problem leads to a closed-form solution. The method was validated on the Pavia University, Salton Sea, and the Mississippi Gulfport datasets. When the proposed registration algorithm is compared to its rigid variant and two mutual information-based methods, it has the best accuracy for both the nonrigid simulated dataset and the real dataset, with an average error less than 0.15 pixels for nonrigid distortion of maximum 1 HS pixel. When the fusion algorithm is compared with current state-of-the-art algorithms, it has the best performance on images with registration errors as well as on simulations that do not consider registration effects.more » « less
-
Push-broom hyperspectral imaging (Pb-HSI) is a powerful technique for obtaining the spectral information along with the spatial information simultaneously for various applications, from remote sensing to chemical imaging. Spatial resolution improvement is beneficial in many instances; however, typical solutions suffer from the limitation of geometric extent, lowered light throughput, or reduced field-of-view (FOV). Sub-pixel shifting (SPS) acquires higher-resolution images, compared to typical imaging approaches, from the deconvolution of low-resolution images acquired with a higher sampling rate. Furthermore, SPS is particularly suited for Pb-HSI due to its scanning nature. In this study, an SPS approach is developed and implemented on a Pb-HSI system for plasma optical emission spectroscopy. The preliminary results showed that a periodic deconvolution error was generated in the final SPS Pb-HSI images. The periodic error was traced back to random noise present in the raw/convoluted SPS data and its frequency displays an inverse relationship with the number of sub-pixel samples acquired. Computer modelled data allows studying the effect of varying the relative standard deviation (RSD) in the raw/convoluted SPS data on the final reconstructed SPS images and optimization of noise filtering. The optimized SPS Pb-HSI technique was used to acquire the line-of-sight integrated optical emission maps from an atmospheric pressure micro-capillary dielectric barrier discharge (μDBD). The selected plasma species of interest (He, I, N 2 , N 2 + , and O) yield some insight into the underlying mechanisms. The SPS Pb-HSI technique developed here will allow implementing geometric super-resolution in many applications, for example, it will be used for extracting radially resolved information from Abel's inversion protocols, where improved fitting is expected due to the increase in resolution/data points.more » « less
-
Abstract Hyperspectral imaging has broad applications and impacts in areas including environmental science, weather, and geo/space exploration. The intrinsic spectral–spatial structures and potential multi-level features in different frequency bands make multilayer graph an intuitive model for hyperspectral images (HSI). To study the underlying characteristics of HSI and to take the advantage of graph signal processing (GSP) tools, this work proposes a multilayer graph spectral analysis for hyperspectral images based on multilayer graph signal processing (M-GSP). More specifically, we present multilayer graph (MLG) models and tensor representations for HSI. By exploring multilayer graph spectral space, we develop MLG-based methods for HSI applications, including unsupervised segmentation and supervised classification. Our experimental results demonstrate the strength of M-GSP in HSI processing and spectral–spatial information extraction.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/ICIP.2018.8451733
