An official website of the United States government
Hyperspectral imaging technologies have shown great promise for biomedical applications. These techniques have been especially useful for detection of molecular events and characterization of cell, tissue, and biomaterial composition. Unfortunately, hyperspectral imaging technologies have been slow to translate to clinical devices – likely due to increased cost and complexity of the technology as well as long acquisition times often required to sample a spectral image. We have demonstrated that hyperspectral imaging approaches which scan the fluorescence excitation spectrum can provide increased signal strength and faster imaging, compared to traditional emission-scanning approaches. We have also demonstrated that excitation-scanning approaches may be able to detect spectral differences between colonic adenomas and adenocarcinomas and normal mucosa in flash-frozen tissues. Here, we report feasibility results from using excitation-scanning hyperspectral imaging to screen pairs of fresh tumoral and nontumoral colorectal tissues. Tissues were imaged using a novel hyperspectral imaging fluorescence excitation scanning microscope, sampling a wavelength range of 360-550 nm, at 5 nm increments. Image data were corrected to achieve a NIST-traceable flat spectral response. Image data were then analyzed using a range of supervised and unsupervised classification approaches within ENVI software (Harris Geospatial Solutions). Supervised classification resulted in >99% accuracy for single-patient image data, but only 64% accuracy for multi-patient classification (n=9 to date), with the drop in accuracy due to increased false-positive detection rates. Hence, initial data indicate that this approach may be a viable detection approach, but that larger patient sample sizes need to be evaluated and the effects of inter-patient variability studied.
more »
« less
This content will become publicly available on December 1, 2025
Fluorescence excitation-scanning hyperspectral imaging with scalable 2D–3D deep learning framework for colorectal cancer detection
Abstract Colorectal cancer is one of the top contributors to cancer-related deaths in the United States, with over 100,000 estimated cases in 2020 and over 50,000 deaths. The most common screening technique is minimally invasive colonoscopy using either reflected white light endoscopy or narrow-band imaging. However, current imaging modalities have only moderate sensitivity and specificity for lesion detection. We have developed a novel fluorescence excitation-scanning hyperspectral imaging (HSI) approach to sample image and spectroscopic data simultaneously on microscope and endoscope platforms for enhanced diagnostic potential. Unfortunately, fluorescence excitation-scanning HSI datasets pose major challenges for data processing, interpretability, and classification due to their high dimensionality. Here, we present an end-to-end scalable Artificial Intelligence (AI) framework built for classification of excitation-scanning HSI microscopy data that provides accurate image classification and interpretability of the AI decision-making process. The developed AI framework is able to perform real-time HSI classification with different speed/classification performance trade-offs by tailoring the dimensionality of the dataset, supporting different dimensions of deep learning models, and varying the architecture of deep learning models. We have also incorporated tools to visualize the exact location of the lesion detected by the AI decision-making process and to provide heatmap-based pixel-by-pixel interpretability. In addition, our deep learning framework provides wavelength-dependent impact as a heatmap, which allows visualization of the contributions of HSI wavelength bands during the AI decision-making process. This framework is well-suited for HSI microscope and endoscope platforms, where real-time analysis and visualization of classification results are required by clinicians.
more »
« less
- Award ID(s):
- 1953544
- PAR ID:
- 10567346
- Publisher / Repository:
- Scientific Reports - Nature
- Date Published:
- Journal Name:
- Scientific Reports
- Volume:
- 14
- Issue:
- 1
- ISSN:
- 2045-2322
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
In the past two decades, spectral imaging technologies have expanded the capacity of fluorescence microscopy for accurate detection of multiple labels, separation of labels from cellular and tissue autofluorescence, and analysis of autofluorescence signatures. These technologies have been implemented using a range of optical techniques, such as tunable filters, diffraction gratings, prisms, interferometry, and custom Bayer filters. Each of these techniques has associated strengths and weaknesses with regard to spectral resolution, spatial resolution, temporal resolution, and signal-to-noise characteristics. We have previously shown that spectral scanning of the fluorescence excitation spectrum can provide greatly increased signal strength compared to traditional emission-scanning approaches. Here, we present results from utilizing a Hyperspectral Imaging Fluorescence Excitation Scanning (HIFEX) microscope system for live cell imaging. Live cell signaling studies were performed using HEK 293 and rat pulmonary microvascular endothelial cells (PMVECs), transfected with either a cAMP FRET reporter or a Ca2+ reporter. Cells were further labeled to visualize subcellular structures (nuclei, membrane, mitochondria, etc.). Spectral images were acquired using a custom inverted microscope (TE2000, Nikon Instruments) equipped with a 300W Xe arc lamp and tunable excitation filter (VF- 5, Sutter Instrument Co., equipped with VersaChrome filters, Semrock), and run through MicroManager. Timelapse spectral images were acquired from 350-550 nm, in 5 nm increments. Spectral image data were linearly unmixed using custom MATLAB scripts. Results indicate that the HIFEX microscope system can acquire live cell image data at acquisition speeds of 8 ms/wavelength band with minimal photobleaching, sufficient for studying moderate speed cAMP and Ca2+ events.more » « less
-
Spectroscopic image data has provided molecular discrimination for numerous fields including: remote sensing, food safety and biomedical imaging. Despite the various technologies for acquiring spectral data, there remains a trade-off when acquiring data. Typically, spectral imaging either requires long acquisition times to collect an image stack with high spectral specificity or acquisition times are shortened at the expense of fewer spectral bands or reduced spatial sampling. Hence, new spectral imaging microscope platforms are needed to help mitigate these limitations. Fluorescence excitation-scanning spectral imaging is one such new technology, which allows more of the emitted signal to be detected than comparable emission-scanning spectral imaging systems. Here, we have developed a new optical geometry that provides spectral illumination for use in excitation-scanning spectral imaging microscope systems. This was accomplished using a wavelength-specific LED array to acquire spectral image data. Feasibility of the LED-based spectral illuminator was evaluated through simulation and benchtop testing and assessment of imaging performance when integrated with a widefield fluorescence microscope. Ray tracing simulations (TracePro) were used to determine optimal optical component selection and geometry. Spectral imaging feasibility was evaluated using a series of 6-label fluorescent slides. The LED-based system response was compared to a previously tested thin-film tunable filter (TFTF)-based system. Spectral unmixing successfully discriminated all fluorescent components in spectral image data acquired from both the LED and TFTF systems. Therefore, the LED-based spectral illuminator provided spectral image data sets with comparable information content so as to allow identification of each fluorescent component. These results provide proof-of-principle demonstration of the ability to combine output from many discrete wavelength LED sources using a double-mirror (Cassegrain style) optical configuration that can be further modified to allow for high speed, video-rate spectral image acquisition. Real-time spectral fluorescence microscopy would allow monitoring of rapid cell signaling processes (i.e., Ca2+and other second messenger signaling) and has potential to be translated to clinical imaging platforms.more » « less
-
Positive outcomes for colorectal cancer treatment have been linked to early detection. The difficulty in detecting early lesions is the limited contrast with surrounding mucosa and minimal definitive markers to distinguish between hyperplastic and carcinoma lesions. Colorectal cancer is the 3rd leading cancer for incidence and mortality rates which is potentially linked to missed early lesions which allow for increased growth and metastatic potential. One potential technology for early-stage lesion detection is hyperspectral imaging. Traditionally, hyperspectral imaging uses reflectance spectroscopic data to provide a component analysis, per pixel, of an image in fields such as remote sensing, agriculture, food processing and archaeology. This work aims to acquire higher signal-to-noise fluorescence spectroscopic data, harnessing the autofluorescence of tissue, adding a hyperspectral contrast to colorectal cancer detection while maintaining spatial resolution at video-rate speeds. We have previously designed a multi-furcated LED-based spectral light source to prove this concept. Our results demonstrated that the technique is feasible, but the initial prototype has a high light transmission loss (~98%) minimizing spatial resolution and slowing video acquisition. Here, we present updated results in developing an optical ray-tracing model of light source geometries to maximize irradiance throughput for excitation-scanning hyperspectral imaging. Results show combining solid light guide branches have a compounding light loss effect, however, there is potential to minimize light loss through the use of optical claddings. This simulation data will provide the necessary metrics to verify and validate future physical optical components within the hyperspectral endoscopic system for detecting colorectal cancer.more » « less
-
Abstract We present a two-photon fluorescence microscope designed for high-speed imaging of neural activity in cellular resolution. Our microscope uses a new adaptive sampling scheme with line illumination. Instead of building images pixel by pixel via scanning a diffraction-limited spot across the sample, our scheme only illuminates the regions of interest (i.e., neuronal cell bodies), and samples a large area of them in a single measurement. Such a scheme significantly increases the imaging speed and reduces the overall laser power on the brain tissue. Using this approach, we performed high-speed imaging of the neural activity of mouse cortexin vivo. Our method provides a new sampling strategy in laser-scanning two-photon microscopy, and will be powerful for high-throughput imaging of neural activity.more » « less
