skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Deep Learning-Based Point-Scanning Super-Resolution Imaging
Point scanning imaging systems (e.g. scanning electron or laser scanning confocal microscopes) are perhaps the most widely used tools for high resolution cellular and tissue imaging. Like all other imaging modalities, the resolution, speed, sample preservation, and signal-to-noise ratio (SNR) of point scanning systems are difficult to optimize simultaneously. In particular, point scanning systems are uniquely constrained by an inverse relationship between imaging speed and pixel resolution. Here we show these limitations can be miti gated via the use of deep learning-based super-sampling of undersampled images acquired on a point-scanning system, which we termed point -scanning super-resolution (PSSR) imaging. Oversampled ground truth images acquired on scanning electron or Airyscan laser scanning confocal microscopes were used to generate semi-synthetictrain ing data for PSSR models that were then used to restore undersampled images. Remarkably, our EM PSSR model was able to restore undersampled images acquired with different optics, detectors, samples, or sample preparation methods in other labs . PSSR enabled previously unattainable xy resolution images with our serial block face scanning electron microscope system. For fluorescence, we show that undersampled confocal images combined with a multiframe PSSR model trained on Airyscan timelapses facilitates Airyscan-equivalent spati al resolution and SNR with ~100x lower laser dose and 16x higher frame rates than corresponding high-resolution acquisitions. In conclusion, PSSR facilitates point-scanning image acquisition with otherwise unattainable resolution, speed, and sensitivity.  more » « less
Award ID(s):
1707356
PAR ID:
10171019
Author(s) / Creator(s):
Date Published:
Journal Name:
PloS one
ISSN:
1932-6203
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; (, Biomedical Optics Express)
    Confocal microscopy is a standard approach for obtaining volumetric images of a sample with high axial and lateral resolution, especially when dealing with scattering samples. Unfortunately, a confocal microscope is quite expensive compared to traditional microscopes. In addition, the point scanning in confocal microscopy leads to slow imaging speed and photobleaching due to the high dose of laser energy. In this paper, we demonstrate how the advances in machine learning can be exploited to teach a traditional wide-field microscope, one that’s available in every lab, into producing 3D volumetric images like a confocal microscope. The key idea is to obtain multiple images with different focus settings using a wide-field microscope and use a 3D generative adversarial network (GAN) based neural network to learn the mapping between the blurry low-contrast image stacks obtained using a wide-field microscope and the sharp, high-contrast image stacks obtained using a confocal microscope. After training the network with widefield-confocal stack pairs, the network can reliably and accurately reconstruct 3D volumetric images that rival confocal images in terms of its lateral resolution, z-sectioning and image contrast. Our experimental results demonstrate generalization ability to handle unseen data, stability in the reconstruction results, high spatial resolution even when imaging thick (∼40 microns) highly-scattering samples. We believe that such learning-based microscopes have the potential to bring confocal imaging quality to every lab that has a wide-field microscope. 
    more » « less
  2. ; ; ; ; ; ; (, PLOS One)
    Pesce, Luca (Ed.)
    Expansion microscopy (ExM) enables sub-diffraction imaging by physically expanding labeled tissue samples. This increases the tissue volume relative to the instrument point spread function (PSF), thereby improving the effective resolution by reported factors of 4 - 20X. However, this volume increase dilutes the fluorescence signal, reducing both signal-to-noise ratio (SNR) and acquisition speed. This paper proposes and validates a method for mitigating these challenges. We overcame the limitations of ExM by developing a fast photo-stable protocol to enable scalable widefield three-dimensional imaging with ExM. We combined widefield imaging with quantum dots (QDots). Widefield imaging provides a significantly faster acquisition of a single field-of-view (FOV). However, the uncontrolled incoherent illumination induces photobleaching. We mitigated this challenge using QDots, which exhibit a long fluorescence lifetime and improved photostability. First, we developed a protocol for QDot labeling. Next, we utilized widefield imaging to obtain 3D image stacks and applied deconvolution, which is feasible due to reduced scattering in ExM samples. We show that increased transparency, which is a side-effect of ExM, enables widefield deconvolution, dramatically reducing the acquisition time for three-dimensional images compared to laser scanning microscopy. The proposed QDot labeling protocol is compatible with ExM and provides enhanced photostability compared to traditional fluorescent dyes. Widefield imaging significantly improves SNR and acquisition speed compared to conventional confocal microscopy. Combining widefield imaging with QDot labeling and deconvolution has the potential to be applied to ExM for faster imaging of large three-dimensional samples with improved SNR. 
    more » « less
  3. ; (, Optics Letters)
    We demonstrate hyperspectral confocal microscopy in the short-wave infrared (SWIR) range of 1100–1600 nm using a wavelength-scanning laser in tandem with laser scanning confocal microscopy. Confocal microscopy in the SWIR range allows for high-resolution inspection of an integrated circuit (IC) chip, while hyperspectral imaging, together with a chemometric analysis, enables us to identify functional circuit block groups in the acquired image. With the extended capability, the developed instrument can be potentially used for inline inspection and non-invasive failure analysis of IC chips. 
    more » « less
  4. ; ; ; ; ; ; ; ; ; (, Journal of Chemical Education)
    Scanning probe-based microscopes (SPMs) are widely used in biology, chemistry, materials science, and physics to image and manipulate matter on the nanoscale. Unfortunately, high school and university departments lack expensive SPM tools and materials microscopy activities to educate a large number of students in this vital SPM imaging technique. As a result, students face challenges participating in and contributing value to the nanotechnology revolution driving modern scientific innovations. Here we demonstrate an affordable scanning laser-based imaging system (approximately $400, excluding the computer) to introduce students to the point-by-point image formation process underlying SPM methods. In this laboratory activity, students learn how to construct and optimize images of a working solar panel using a laser beam-induced current (LBIC) imaging system. We envision undergraduate and graduate students should be able to use this LBIC system for independent solar energy research projects as well as apply fundamental knowledge and measurement skills to understand other SPM techniques. 
    more » « less
  5. ; ; ; ; ; (, Journal of the Optical Society of America A)
    Imaging beyond the diffraction limit barrier has attracted wide attention due to the ability to resolve previously hidden image features. Of the various super-resolution microscopy techniques available, a particularly simple method called saturated excitation microscopy (SAX) requires only simple modification of a laser scanning microscope: The illumination beam power is sinusoidally modulated and driven into saturation. SAX images are extracted from the harmonics of the modulation frequency and exhibit improved spatial resolution. Unfortunately, this elegant strategy is hindered by the incursion of shot noise that prevents high-resolution imaging in many realistic scenarios. Here, we demonstrate a technique for super-resolution imaging that we call computational saturated absorption (CSA) in which a joint deconvolution is applied to a set of images with diversity in spatial frequency support among the point spread functions (PSFs) used in the image formation with saturated laser scanning fluorescence microscopy. CSA microscopy allows access to the high spatial frequency diversity in a set of saturated effective PSFs, while avoiding image degradation from shot noise. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email