skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Attention:

The NSF Public Access Repository (PAR) system and access will be unavailable from 10:00 PM ET on Friday, February 6 until 10:00 AM ET on Saturday, February 7 due to maintenance. We apologize for the inconvenience.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Deep Learning-Based Point-Scanning Super-Resolution Imaging
Point scanning imaging systems (e.g. scanning electron or laser scanning confocal microscopes) are perhaps the most widely used tools for high resolution cellular and tissue imaging. Like all other imaging modalities, the resolution, speed, sample preservation, and signal-to-noise ratio (SNR) of point scanning systems are difficult to optimize simultaneously. In particular, point scanning systems are uniquely constrained by an inverse relationship between imaging speed and pixel resolution. Here we show these limitations can be miti gated via the use of deep learning-based super-sampling of undersampled images acquired on a point-scanning system, which we termed point -scanning super-resolution (PSSR) imaging. Oversampled ground truth images acquired on scanning electron or Airyscan laser scanning confocal microscopes were used to generate semi-synthetictrain ing data for PSSR models that were then used to restore undersampled images. Remarkably, our EM PSSR model was able to restore undersampled images acquired with different optics, detectors, samples, or sample preparation methods in other labs . PSSR enabled previously unattainable xy resolution images with our serial block face scanning electron microscope system. For fluorescence, we show that undersampled confocal images combined with a multiframe PSSR model trained on Airyscan timelapses facilitates Airyscan-equivalent spati al resolution and SNR with ~100x lower laser dose and 16x higher frame rates than corresponding high-resolution acquisitions. In conclusion, PSSR facilitates point-scanning image acquisition with otherwise unattainable resolution, speed, and sensitivity.  more » « less
Award ID(s):
1707356
PAR ID:
10171019
Author(s) / Creator(s):
Date Published:
Journal Name:
PloS one
ISSN:
1932-6203
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; (, PLOS One)
    Pesce, Luca (Ed.)
    Expansion microscopy (ExM) enables sub-diffraction imaging by physically expanding labeled tissue samples. This increases the tissue volume relative to the instrument point spread function (PSF), thereby improving the effective resolution by reported factors of 4 - 20X. However, this volume increase dilutes the fluorescence signal, reducing both signal-to-noise ratio (SNR) and acquisition speed. This paper proposes and validates a method for mitigating these challenges. We overcame the limitations of ExM by developing a fast photo-stable protocol to enable scalable widefield three-dimensional imaging with ExM. We combined widefield imaging with quantum dots (QDots). Widefield imaging provides a significantly faster acquisition of a single field-of-view (FOV). However, the uncontrolled incoherent illumination induces photobleaching. We mitigated this challenge using QDots, which exhibit a long fluorescence lifetime and improved photostability. First, we developed a protocol for QDot labeling. Next, we utilized widefield imaging to obtain 3D image stacks and applied deconvolution, which is feasible due to reduced scattering in ExM samples. We show that increased transparency, which is a side-effect of ExM, enables widefield deconvolution, dramatically reducing the acquisition time for three-dimensional images compared to laser scanning microscopy. The proposed QDot labeling protocol is compatible with ExM and provides enhanced photostability compared to traditional fluorescent dyes. Widefield imaging significantly improves SNR and acquisition speed compared to conventional confocal microscopy. Combining widefield imaging with QDot labeling and deconvolution has the potential to be applied to ExM for faster imaging of large three-dimensional samples with improved SNR. 
    more » « less
  2. ; (, Optics Letters)
    We demonstrate hyperspectral confocal microscopy in the short-wave infrared (SWIR) range of 1100–1600 nm using a wavelength-scanning laser in tandem with laser scanning confocal microscopy. Confocal microscopy in the SWIR range allows for high-resolution inspection of an integrated circuit (IC) chip, while hyperspectral imaging, together with a chemometric analysis, enables us to identify functional circuit block groups in the acquired image. With the extended capability, the developed instrument can be potentially used for inline inspection and non-invasive failure analysis of IC chips. 
    more » « less
  3. ; ; ; ; ; ; ; ; ; (, Journal of Chemical Education)
    Scanning probe-based microscopes (SPMs) are widely used in biology, chemistry, materials science, and physics to image and manipulate matter on the nanoscale. Unfortunately, high school and university departments lack expensive SPM tools and materials microscopy activities to educate a large number of students in this vital SPM imaging technique. As a result, students face challenges participating in and contributing value to the nanotechnology revolution driving modern scientific innovations. Here we demonstrate an affordable scanning laser-based imaging system (approximately $400, excluding the computer) to introduce students to the point-by-point image formation process underlying SPM methods. In this laboratory activity, students learn how to construct and optimize images of a working solar panel using a laser beam-induced current (LBIC) imaging system. We envision undergraduate and graduate students should be able to use this LBIC system for independent solar energy research projects as well as apply fundamental knowledge and measurement skills to understand other SPM techniques. 
    more » « less
  4. ; (, BMC Methods)
    Abstract BackgroundTo address the limitations of large-scale high quality microscopy image acquisition, PSSR (Point-Scanning Super-Resolution) was introduced to enhance easily acquired low quality microscopy data to a higher quality using deep learning-based methods. However, while PSSR was released as open-source, it was difficult for users to implement into their workflows due to an outdated codebase, limiting its usage by prospective users. Additionally, while the data enhancements provided by PSSR were significant, there was still potential for further improvement. MethodsTo overcome this, we introduce PSSR2, a redesigned implementation of PSSR workflows and methods built to put state-of-the-art technology into the hands of the general microscopy and biology research community. PSSR2 enables user-friendly implementation of super-resolution workflows for simultaneous super-resolution and denoising of undersampled microscopy data, especially through its integrated Command Line Interface and Napari plugin. PSSR2 improves and expands upon previously established PSSR algorithms, mainly through improvements in the semi-synthetic data generation (“crappification”) and training processes. ResultsIn benchmarking PSSR2 on a test dataset of paired high and low resolution electron microscopy images, PSSR2 super-resolves high-resolution images from low-resolution images to a significantly higher accuracy than PSSR. The super-resolved images are also more visually representative of real-world high-resolution images. DiscussionThe improvements in PSSR2, in providing higher quality images, should improve the performance of downstream analyses. We note that for accurate super-resolution, PSSR2 models should only be applied to super-resolve data sufficiently similar to training data and should be validated against real-world ground truth data. 
    more » « less
  5. ; ; ; ; ; (, Journal of the Optical Society of America A)
    Imaging beyond the diffraction limit barrier has attracted wide attention due to the ability to resolve previously hidden image features. Of the various super-resolution microscopy techniques available, a particularly simple method called saturated excitation microscopy (SAX) requires only simple modification of a laser scanning microscope: The illumination beam power is sinusoidally modulated and driven into saturation. SAX images are extracted from the harmonics of the modulation frequency and exhibit improved spatial resolution. Unfortunately, this elegant strategy is hindered by the incursion of shot noise that prevents high-resolution imaging in many realistic scenarios. Here, we demonstrate a technique for super-resolution imaging that we call computational saturated absorption (CSA) in which a joint deconvolution is applied to a set of images with diversity in spatial frequency support among the point spread functions (PSFs) used in the image formation with saturated laser scanning fluorescence microscopy. CSA microscopy allows access to the high spatial frequency diversity in a set of saturated effective PSFs, while avoiding image degradation from shot noise. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Text Encoded Conversion
  • XML
  • Save / Share this Record
  • Send to Email