skip to main content

Title: Semi-blind sparse affine spectral unmixing of autofluorescence-contaminated micrographs
Abstract Motivation

Spectral unmixing methods attempt to determine the concentrations of different fluorophores present at each pixel location in an image by analyzing a set of measured emission spectra. Unmixing algorithms have shown great promise for applications where samples contain many fluorescent labels; however, existing methods perform poorly when confronted with autofluorescence-contaminated images.


We propose an unmixing algorithm designed to separate fluorophores with overlapping emission spectra from contamination by autofluorescence and background fluorescence. First, we formally define a generalization of the linear mixing model, called the affine mixture model (AMM), that specifically accounts for background fluorescence. Second, we use the AMM to derive an affine nonnegative matrix factorization method for estimating fluorophore endmember spectra from reference images. Lastly, we propose a semi-blind sparse affine spectral unmixing (SSASU) algorithm that uses knowledge of the estimated endmembers to learn the autofluorescence and background fluorescence spectra on a per-image basis. When unmixing real-world spectral images contaminated by autofluorescence, SSASU greatly improved proportion indeterminacy as compared to existing methods for a given relative reconstruction error.

Availability and implementation

The source code used for this paper was written in Julia and is available with the test data at

more » « less
Award ID(s):
Author(s) / Creator(s):
 ;  ;  ;  ;  ;
Publisher / Repository:
Oxford University Press
Date Published:
Journal Name:
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Motivation

    Multispectral biological fluorescence microscopy has enabled the identification of multiple targets in complex samples. The accuracy in the unmixing result degrades (i) as the number of fluorophores used in any experiment increases and (ii) as the signal-to-noise ratio in the recorded images decreases. Further, the availability of prior knowledge regarding the expected spatial distributions of fluorophores in images of labeled cells provides an opportunity to improve the accuracy of fluorophore identification and abundance.


    We propose a regularized sparse and low-rank Poisson regression unmixing approach (SL-PRU) to deconvolve spectral images labeled with highly overlapping fluorophores which are recorded in low signal-to-noise regimes. First, SL-PRU implements multipenalty terms when pursuing sparseness and spatial correlation of the resulting abundances in small neighborhoods simultaneously. Second, SL-PRU makes use of Poisson regression for unmixing instead of least squares regression to better estimate photon abundance. Third, we propose a method to tune the SL-PRU parameters involved in the unmixing procedure in the absence of knowledge of the ground truth abundance information in a recorded image. By validating on simulated and real-world images, we show that our proposed method leads to improved accuracy in unmixing fluorophores with highly overlapping spectra.

    Availability and implementation

    The source code used for this article was written in MATLAB and is available with the test data at

    more » « less
  2. Autofluorescence has historically been considered a nuisance in medical imaging. Many endogenous fluorophores, specifically, collagen, elastin, NADH, and FAD, are found throughout the human body. Diagnostically, these signals can be prohibitive since they can outcompete signals introduced for diagnostic purposes. Recent advances in hyperspectral imaging have allowed the acquisition of significantly more data in a shorter time period by scanning the excitation spectra of fluorophores. The reduced acquisition time and increased signal-to-noise ratio allow for separation of significantly more fluorophores than previously possible. Here, we propose to utilize excitation-scanning of autofluorescence to examine tissues and diagnose pathologies. Spectra of autofluorescent molecules were obtained using a custom inverted microscope (TE-2000, Nikon Instruments) with a Xe arc lamp and thin film tunable filter array (VersaChrome, Semrock, Inc.) Scans utilized excitation wavelengths from 360 nm to 550 nm in 5 nm increments. The resultant spectra were used to examine hyperspectral image stacks from various collaborative studies, including an atherosclerotic rat model and a colon cancer study. Hyperspectral images were analyzed with ENVI and custom Matlab scripts including linear spectral unmixing (LSU) and principal component analysis (PCA). Initial results suggest the ability to separate the signals of endogenous fluorophores and measure the relative concentrations of fluorophores among healthy and diseased states of similar tissues. These results suggest pathology-specific changes to endogenous fluorophores can be detected using excitationscanning hyperspectral imaging. Future work will expand the library of pure molecules and will examine more defined disease states. 
    more » « less
  3. Summary

    Localization of mRNA and small RNAs (sRNAs) is important for understanding their function. Fluorescentin situhybridization (FISH) has been used extensively in animal systems to study the localization and expression of sRNAs. However, current methods for fluorescentin situdetection of sRNA in plant tissues are less developed. Here we report a protocol (sRNA‐FISH) for efficient fluorescent detection of sRNAs in plants. This protocol is suitable for application in diverse plant species and tissue types. The use of locked nucleic acid probes and antibodies conjugated with different fluorophores allows the detection of two sRNAs in the same sample. Using this method, we have successfully detected the co‐localization of miR2275 and a 24‐nucleotide phased small interfering RNA in maize anther tapetal and archesporial cells. We describe how to overcome the common problem of the wide range of autofluorescence in embedded plant tissue using linear spectral unmixing on a laser scanning confocal microscope. For highly autofluorescent samples, we show that multi‐photon fluorescence excitation microscopy can be used to separate the target sRNA‐FISH signal from background autofluorescence. In contrast to colorimetricin situhybridization, sRNA‐FISH signals can be imaged using super‐resolution microscopy to examine the subcellular localization of sRNAs. We detected maize miR2275 by super‐resolution structured illumination microscopy and direct stochastic optical reconstruction microscopy. In this study, we describe how we overcame the challenges of adapting FISH for imaging in plant tissue and provide a step‐by‐step sRNA‐FISH protocol for studying sRNAs at the cellular and even subcellular level.

    more » « less
  4. Abstract

    Fluorescence emission is common in plants. While fluorescence microscopy has been widely used to study living plants, its application in quantifying the fluorescence of fossil plants has been limited. Fossil plant fluorescence, from original fluorophores or formed during fossilization, can offer valuable insights into fluorescence in ancient plants and fossilization processes. In this work, we utilize two-photon fluorescence microspectroscopy to spatially and spectrally resolve the fluorescence emitted by amber-embedded plants, leaf compressions, and silicified wood. The advanced micro-spectroscope utilized, with its pixel-level spectral resolution and line-scan excitation capabilities, allows us to collect comprehensive excitation and emission spectra with high sensitivity and minimal laser damage to the specimens. By applying linear spectral unmixing to the spectrally resolved fluorescence images, we can differentiate between (a) the matrix and (b) the materials that comprise the fossil. Our analysis suggests that the latter correspond to durable tissues such as lignin and cellulose. Additionally, we observe potential signals from chlorophyll derivatives/tannins, although minerals may have contributed to this. This research opens doors to exploring ancient ecosystems and understanding the ecological roles of fluorescence in plants throughout time. Furthermore, the protocols developed herein can also be applied to analyze non-plant fossils and biological specimens.

    more » « less
  5. Abstract Background

    Spectral CT material decomposition provides quantitative information but is challenged by the instability of the inversion into basis materials. We have previously proposed the constrained One‐Step Spectral CT Image Reconstruction (cOSSCIR) algorithm to stabilize the material decomposition inversion by directly estimating basis material images from spectral CT data. cOSSCIR was previously investigated on phantom data.


    This study investigates the performance of cOSSCIR using head CT datasets acquired on a clinical photon‐counting CT (PCCT) prototype. This is the first investigation of cOSSCIR for large‐scale, anatomically complex, clinical PCCT data. The cOSSCIR decomposition is preceded by a spectrum estimation and nonlinear counts correction calibration step to address nonideal detector effects.


    Head CT data were acquired on an early prototype clinical PCCT system using an edge‐on silicon detector with eight energy bins. Calibration data of a step wedge phantom were also acquired and used to train a spectral model to account for the source spectrum and detector spectral response, and also to train a nonlinear counts correction model to account for pulse pileup effects. The cOSSCIR algorithm optimized the bone and adipose basis images directly from the photon counts data, while placing a grouped total variation (TV) constraint on the basis images. For comparison, basis images were also reconstructed by a two‐step projection‐domain approach of Maximum Likelihood Estimation (MLE) for decomposing basis sinograms, followed by filtered backprojection (MLE + FBP) or a TV minimization algorithm (MLE + TVmin) to reconstruct basis images. We hypothesize that the cOSSCIR approach will provide a more stable inversion into basis images compared to two‐step approaches. To investigate this hypothesis, the noise standard deviation in bone and soft‐tissue regions of interest (ROIs) in the reconstructed images were compared between cOSSCIR and the two‐step methods for a range of regularization constraint settings.


    cOSSCIR reduced the noise standard deviation in the basis images by a factor of two to six compared to that of MLE + TVmin, when both algorithms were constrained to produce images with the same TV. The cOSSCIR images demonstrated qualitatively improved spatial resolution and depiction of fine anatomical detail. The MLE + TVminalgorithm resulted in lower noise standard deviation than cOSSCIR for the virtual monoenergetic images (VMIs) at higher energy levels and constraint settings, while the cOSSCIR VMIs resulted in lower noise standard deviation at lower energy levels and overall higher qualitative spatial resolution. There were no statistically significant differences in the mean values within the bone region of images reconstructed by the studied algorithms. There were statistically significant differences in the mean values within the soft‐tissue region of the reconstructed images, with cOSSCIR producing mean values closer to the expected values.


    The cOSSCIR algorithm, combined with our previously proposed spectral model estimation and nonlinear counts correction method, successfully estimated bone and adipose basis images from high resolution, large‐scale patient data from a clinical PCCT prototype. The cOSSCIR basis images were able to depict fine anatomical details with a factor of two to six reduction in noise standard deviation compared to that of the MLE + TVmintwo‐step approach.

    more » « less