skip to main content

Title: Deep Learning-Based Point-Scanning Super-Resolution Imaging
Point scanning imaging systems (e.g. scanning electron or laser scanning confocal microscopes) are perhaps the most widely used tools for high resolution cellular and tissue imaging. Like all other imaging modalities, the resolution, speed, sample preservation, and signal-to-noise ratio (SNR) of point scanning systems are difficult to optimize simultaneously. In particular, point scanning systems are uniquely constrained by an inverse relationship between imaging speed and pixel resolution. Here we show these limitations can be miti gated via the use of deep learning-based super-sampling of undersampled images acquired on a point-scanning system, which we termed point -scanning super-resolution (PSSR) imaging. Oversampled ground truth images acquired on scanning electron or Airyscan laser scanning confocal microscopes were used to generate semi-synthetictrain ing data for PSSR models that were then used to restore undersampled images. Remarkably, our EM PSSR model was able to restore undersampled images acquired with different optics, detectors, samples, or sample preparation methods in other labs . PSSR enabled previously unattainable xy resolution images with our serial block face scanning electron microscope system. For fluorescence, we show that undersampled confocal images combined with a multiframe PSSR model trained on Airyscan timelapses facilitates Airyscan-equivalent spati al resolution and SNR with ~100x lower laser dose and 16x higher frame rates than corresponding high-resolution acquisitions. In conclusion, PSSR facilitates point-scanning image acquisition with otherwise unattainable resolution, speed, and sensitivity.  more » « less
Award ID(s):
Author(s) / Creator(s):
Date Published:
Journal Name:
PloS one
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Confocal microscopy is a standard approach for obtaining volumetric images of a sample with high axial and lateral resolution, especially when dealing with scattering samples. Unfortunately, a confocal microscope is quite expensive compared to traditional microscopes. In addition, the point scanning in confocal microscopy leads to slow imaging speed and photobleaching due to the high dose of laser energy. In this paper, we demonstrate how the advances in machine learning can be exploited to teach a traditional wide-field microscope, one that’s available in every lab, into producing 3D volumetric images like a confocal microscope. The key idea is to obtain multiple images with different focus settings using a wide-field microscope and use a 3D generative adversarial network (GAN) based neural network to learn the mapping between the blurry low-contrast image stacks obtained using a wide-field microscope and the sharp, high-contrast image stacks obtained using a confocal microscope. After training the network with widefield-confocal stack pairs, the network can reliably and accurately reconstruct 3D volumetric images that rival confocal images in terms of its lateral resolution, z-sectioning and image contrast. Our experimental results demonstrate generalization ability to handle unseen data, stability in the reconstruction results, high spatial resolution even when imaging thick (∼40 microns) highly-scattering samples. We believe that such learning-based microscopes have the potential to bring confocal imaging quality to every lab that has a wide-field microscope.

    more » « less
  2. Recent advancements in image-scanning microscopy have significantly enriched super-resolution biological research, providing deeper insights into cellular structures and processes. However, current image-scanning techniques often require complex instrumentation and alignment, constraining their broader applicability in cell biological discovery and convenient, cost-effective integration into commonly used frameworks like epi-fluorescence microscopes. Here, we introduce three-dimensional multifocal scanning microscopy (3D-MSM) for super-resolution imaging of cells and tissue with substantially reduced instrumental complexity. This method harnesses the inherent 3D movement of specimens to achieve stationary, multi-focal excitation and super-resolution microscopy through a standard epi-fluorescence platform. We validated the system using a range of phantom, single-cell, and tissue specimens. The combined strengths of structured illumination, confocal detection, and epi-fluorescence setup result in two-fold resolution improvement in all three dimensions, effective optical sectioning, scalable volume acquisition, and compatibility with general imaging and sample protocols. We anticipate that 3D-MSM will pave a promising path for future super-resolution investigations in cell and tissue biology.

    more » « less
  3. Confocal microscopy provides optical sectioning that is invaluable for many applications, most notably imaging into thick samples. However, the high cost of commercial confocal microscopes limits uses to specialized research and clinical settings. We present a minimalistic line-scanning confocal microscope costing less than $6,500 with optical performance comparable to a commercial laser scanning system. The optical sectioning and imaging performance are shown through measurement of the axial line-spread function and imaging of biological samples of varying thickness. Comparison is made to commercial widefield and confocal microscopes. The low cost of goods and optical sectioning capability of this microscope will allow the use of confocal microscopy in additional research and educational settings.

    more » « less
  4. We demonstrate hyperspectral confocal microscopy in the short-wave infrared (SWIR) range of 1100–1600 nm using a wavelength-scanning laser in tandem with laser scanning confocal microscopy. Confocal microscopy in the SWIR range allows for high-resolution inspection of an integrated circuit (IC) chip, while hyperspectral imaging, together with a chemometric analysis, enables us to identify functional circuit block groups in the acquired image. With the extended capability, the developed instrument can be potentially used for inline inspection and non-invasive failure analysis of IC chips.

    more » « less
  5. Confocal laser scanning microscopy (CLSM) is a preferred method for obtaining optical images with submicrometer resolution. Replacing the pinhole and detector of a CLSM with a digital camera [charge-coupled device (CCD) or complementary metal oxide semiconductor (CMOS)] has the potential to simplify the design and reduce cost. However, the relatively slow speed of a typical camera results in long scans. To address this issue, in the present investigation a microlens array was used to split the laser beam into 48 beamlets that are focused onto the sample. In essence, 48 pinhole-detector measurements were performed in parallel. Images obtained from the 48 laser spots were stitched together into a final image.

    more » « less