Point scanning imaging systems (e.g. scanning electron or laser scanning confocal microscopes) are perhaps the most widely used tools for high resolution cellular and tissue imaging. Like all other imaging modalities, the resolution, speed, sample preservation, and signal-to-noise ratio (SNR) of point scanning systems are difficult to optimize simultaneously. In particular, point scanning systems are uniquely constrained by an inverse relationship between imaging speed and pixel resolution. Here we show these limitations can be miti gated via the use of deep learning-based super-sampling of undersampled images acquired on a point-scanning system, which we termed point -scanning super-resolution (PSSR) imaging. Oversampled ground truth images acquired on scanning electron or Airyscan laser scanning confocal microscopes were used to generate semi-synthetictrain ing data for PSSR models that were then used to restore undersampled images. Remarkably, our EM PSSR model was able to restore undersampled images acquired with different optics, detectors, samples, or sample preparation methods in other labs . PSSR enabled previously unattainable xy resolution images with our serial block face scanning electron microscope system. For fluorescence, we show that undersampled confocal images combined with a multiframe PSSR model trained on Airyscan timelapses facilitates Airyscan-equivalent spati al resolution and SNR with ~100x lower laser dose and 16x higher frame rates than corresponding high-resolution acquisitions. In conclusion, PSSR facilitates point-scanning image acquisition with otherwise unattainable resolution, speed, and sensitivity.
more »
« less
This content will become publicly available on June 23, 2024
Super-resolution computational saturated absorption microscopy
Imaging beyond the diffraction limit barrier has attracted wide attention due to the ability to resolve previously hidden image features. Of the various super-resolution microscopy techniques available, a particularly simple method called saturated excitation microscopy (SAX) requires only simple modification of a laser scanning microscope: The illumination beam power is sinusoidally modulated and driven into saturation. SAX images are extracted from the harmonics of the modulation frequency and exhibit improved spatial resolution. Unfortunately, this elegant strategy is hindered by the incursion of shot noise that prevents high-resolution imaging in many realistic scenarios. Here, we demonstrate a technique for super-resolution imaging that we call computational saturated absorption (CSA) in which a joint deconvolution is applied to a set of images with diversity in spatial frequency support among the point spread functions (PSFs) used in the image formation with saturated laser scanning fluorescence microscopy. CSA microscopy allows access to the high spatial frequency diversity in a set of saturated effective PSFs, while avoiding image degradation from shot noise.
more » « less- NSF-PAR ID:
- 10425217
- Publisher / Repository:
- Optical Society of America
- Date Published:
- Journal Name:
- Journal of the Optical Society of America A
- Volume:
- 40
- Issue:
- 7
- ISSN:
- 1084-7529; JOAOD6
- Page Range / eLocation ID:
- Article No. 1409
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
Abstract The photoluminescence (PL) saturation of CdSe/ZnS core/shell inorganic semiconductor quantum dots (QDs) and its utility as a probe for saturated excitation (SAX) microscopy are reported. Under saturating excitation power densities, the PL signal was demodulated and recorded at harmonics of the fundamental frequency. For commercially available Qdot® 655 ITK™ QDs, the power density required to achieve saturation was dependent upon the local environment of the QDs. For QDs deposited and dried on a glass substrate, the excitation power density required for PL saturation was less than 1 kW/cm2. Compared to this, saturation of PL for QDs dispersed in water required an excitation power density greater than 200 kW/cm2. This observation is manifested as a limitation in the imaging of hydrated samples, as demonstrated for HeLa cells labelled with biotinylated‐phalloidin followed by labelling with streptavidin‐coated QDs. As saturation affects the obtained spatial resolution in several imaging formats, including confocal imaging, the provided data will aid in obtaining the optimal spatial resolution when using QD probes to image biological samples.
-
more » « less
Confocal microscopy is a standard approach for obtaining volumetric images of a sample with high axial and lateral resolution, especially when dealing with scattering samples. Unfortunately, a confocal microscope is quite expensive compared to traditional microscopes. In addition, the point scanning in confocal microscopy leads to slow imaging speed and photobleaching due to the high dose of laser energy. In this paper, we demonstrate how the advances in machine learning can be exploited to teach a traditional wide-field microscope, one that’s available in every lab, into producing 3D volumetric images like a confocal microscope. The key idea is to obtain multiple images with different focus settings using a wide-field microscope and use a 3D generative adversarial network (GAN) based neural network to learn the mapping between the blurry low-contrast image stacks obtained using a wide-field microscope and the sharp, high-contrast image stacks obtained using a confocal microscope. After training the network with widefield-confocal stack pairs, the network can reliably and accurately reconstruct 3D volumetric images that rival confocal images in terms of its lateral resolution, z-sectioning and image contrast. Our experimental results demonstrate generalization ability to handle unseen data, stability in the reconstruction results, high spatial resolution even when imaging thick (∼40 microns) highly-scattering samples. We believe that such learning-based microscopes have the potential to bring confocal imaging quality to every lab that has a wide-field microscope.
-
more » « less
Abstract Chemical imaging, especially mid-infrared spectroscopic microscopy, enables label-free biomedical analyses while achieving expansive molecular sensitivity. However, its slow speed and poor image quality impede widespread adoption. We present a microscope that provides high-throughput recording, low noise, and high spatial resolution where the bottom-up design of its optical train facilitates dual-axis galvo laser scanning of a diffraction-limited focal point over large areas using custom, compound, infinity-corrected refractive objectives. We demonstrate whole-slide, speckle-free imaging in ~3 min per discrete wavelength at 10× magnification (2 μm/pixel) and high-resolution capability with its 20× counterpart (1 μm/pixel), both offering spatial quality at theoretical limits while maintaining high signal-to-noise ratios (>100:1). The data quality enables applications of modern machine learning and capabilities not previously feasible – 3D reconstructions using serial sections, comprehensive assessments of whole model organisms, and histological assessments of disease in time comparable to clinical workflows. Distinct from conventional approaches that focus on morphological investigations or immunostaining techniques, this development makes label-free imaging of minimally processed tissue practical.
-
Introduction Multi-series CT (MSCT) scans, including non-contrast CT (NCCT), CT Perfusion (CTP), and CT Angiography (CTA), are widely used in acute stroke imaging. While each scan has its advantage in disease diagnosis, the varying image resolution of different series hinders the ability of the radiologist to discern subtle suspicious findings. Besides, higher image quality requires high radiation doses, leading to increases in health risks such as cataract formation and cancer induction. Thus, it is highly crucial to develop an approach to improve MSCT resolution and to lower radiation exposure. Hypothesis MSCT imaging of the same patient is highly correlated in structural features, the transferring and integration of the shared and complementary information from different series are beneficial for achieving high image quality. Methods We propose TL-GAN, a learning-based method by using Transfer Learning (TL) and Generative Adversarial Network (GAN) to reconstruct high-quality diagnostic images. Our TL-GAN method is evaluated on 4,382 images collected from nine patients’ MSCT scans, including 415 NCCT slices, 3,696 CTP slices, and 271 CTA slices. We randomly split the nine patients into a training set (4 patients), a validation set (2 patients), and a testing set (3 patients). In preprocessing, we remove the background and skull and visualize in brain window. The low-resolution images (1/4 of the original spatial size) are simulated by bicubic down-sampling. For training without TL, we train different series individually, and for with TL, we follow the scanning sequence (NCCT, CTP, and CTA) by finetuning. Results The performance of TL-GAN is evaluated by the peak-signal-to-noise ratio (PSNR) and structural similarity (SSIM) index on 184 NCCT, 882 CTP, and 107 CTA test images. Figure 1 provides both visual (a-c) and quantity (d-f) comparisons. Through TL-GAN, there is a significant improvement with TL than without TL (training from scratch) for NCCT, CTP, and CTA images, respectively. These significances of performance improvement are evaluated by one-tailed paired t-tests (p < 0.05). We enlarge the regions of interest for detail visual comparisons. Further, we evaluate the CTP performance by calculating the perfusion maps, including cerebral blood flow (CBF) and cerebral blood volume (CBV). The visual comparison of the perfusion maps in Figure 2 demonstrate that TL-GAN is beneficial for achieving high diagnostic image quality, which are comparable to the ground truth images for both CBF and CBV maps. Conclusion Utilizing TL-GAN can effectively improve the image resolution for MSCT, provides radiologists more image details for suspicious findings, which is a practical solution for MSCT image quality enhancement.more » « less
