An official website of the United States government
Abstract Since its first demonstration over 100 years ago, scattering‐based light‐sheet microscopy has recently re‐emerged as a key modality in label‐free tissue imaging and cellular morphometry; however, scattering‐based light‐sheet imaging with subcellular resolution remains an unmet target. This is because related approaches inevitably superimpose speckle or granular intensity modulation on to the native subcellular features. Here, we addressed this challenge by deploying a time‐averaged pseudo‐thermalized light‐sheet illumination. While this approach increased the lateral dimensions of the illumination sheet, we achieved subcellular resolving power after image deconvolution. We validated this approach by imaging cytosolic carbon depots in yeast and bacteria with increased specificity, no staining, and ultralow irradiance levels. Overall, we expect this scattering‐based light‐sheet microscopy approach will advance single, live cell imaging by conferring low‐irradiance and label‐free operation towards eradicating phototoxicity.
more »
« less
Content-aware frame interpolation (CAFI): deep learning-based temporal super-resolution for fast bioimaging
Abstract The development of high-resolution microscopes has made it possible to investigate cellular processes in 3D and over time. However, observing fast cellular dynamics remains challenging because of photobleaching and phototoxicity. Here we report the implementation of two content-aware frame interpolation (CAFI) deep learning networks, Zooming SlowMo and Depth-Aware Video Frame Interpolation, that are highly suited for accurately predicting images in between image pairs, therefore improving the temporal resolution of image series post-acquisition. We show that CAFI is capable of understanding the motion context of biological structures and can perform better than standard interpolation methods. We benchmark CAFI’s performance on 12 different datasets, obtained from four different microscopy modalities, and demonstrate its capabilities for single-particle tracking and nuclear segmentation. CAFI potentially allows for reduced light exposure and phototoxicity on the sample for improved long-term live-cell imaging. The models and the training and testing data are available via the ZeroCostDL4Mic platform.
more »
« less
- Award ID(s):
- 2014862
- PAR ID:
- 10523515
- Publisher / Repository:
- Nature methods
- Date Published:
- Journal Name:
- Nature Methods
- Volume:
- 21
- Issue:
- 2
- ISSN:
- 1548-7091
- Page Range / eLocation ID:
- 322 to 330
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract There is a significant gap in cost-effective quantitative phase microscopy (QPM) systems for studying dynamic cellular processes while maintaining accuracy for long-term cellular monitoring. Current QPM systems often rely on complex and expensive voltage-controllable components like Spatial Light Modulators or two-beam interferometry. To address this, we introduce a QPM system optimized for time-varying phase samples using azobenzene liquid crystal as a Zernike filter with a polarization-sensing camera. This system operates without input voltage or moving components, reducing complexity and cost. Optimized for gentle illumination to minimize phototoxicity, it achieves a 1 Hz frame rate for prolonged monitoring. The system demonstrated accuracy with a maximum standard deviation of ±42 nm and low noise fluctuations of ±2.5 nm. Designed for simplicity and single-shot operations, our QPM system is efficient, robust, and precisely calibrated for reliable measurements. Using inexpensive optical components, it offers an economical solution for long-term, noninvasive biological monitoring and research applications.more » « less
-
Though recent years have witnessed remarkable progress in single image super-resolution (SISR) tasks with the prosperous development of deep neural networks (DNNs), the deep learning methods are confronted with the computation and memory consumption issues in practice, especially for resource-limited platforms such as mobile devices. To overcome the challenge and facilitate the real-time deployment of SISR tasks on mobile, we combine neural architecture search with pruning search and propose an automatic search framework that derives sparse super-resolution (SR) models with high image quality while satisfying the real-time inference requirement. To decrease the search cost, we leverage the weight sharing strategy by introducing a supernet and decouple the search problem into three stages, including supernet construction, compiler-aware architecture and pruning search, and compiler-aware pruning ratio search. With the proposed framework, we are the first to achieve real-time SR inference (with only tens of milliseconds per frame) for implementing 720p resolution with competitive image quality (in terms of PSNR and SSIM) on mobile platforms (Samsung Galaxy S20).more » « less
-
Superresolution of GOES-16 ABI Bands to a Common High Resolution with a Convolutional Neural NetworkAbstract Superresolution is the general task of artificially increasing the spatial resolution of an image. The recent surge in machine learning (ML) research has yielded many promising ML-based approaches for performing single-image superresolution including applications to satellite remote sensing. We develop a convolutional neural network (CNN) to superresolve the 1- and 2-km bands on the GOES-R series Advanced Baseline Imager (ABI) to a common high resolution of 0.5 km. Access to 0.5-km imagery from ABI band 2 enables the CNN to realistically sharpen lower-resolution bands without significant blurring. We first train the CNN on a proxy task, which allows us to only use ABI imagery, namely, degrading the resolution of ABI bands and training the CNN to restore the original imagery. Comparisons at reduced resolution and at full resolution withLandsat-8/Landsat-9observations illustrate that the CNN produces images with realistic high-frequency detail that is not present in a bicubic interpolation baseline. Estimating all ABI bands at 0.5-km resolution allows for more easily combining information across bands without reconciling differences in spatial resolution. However, more analysis is needed to determine impacts on derived products or multispectral imagery that use superresolved bands. This approach is extensible to other remote sensing instruments that have bands with different spatial resolutions and requires only a small amount of data and knowledge of each channel’s modulation transfer function. Significance StatementSatellite remote sensing instruments often have bands with different spatial resolutions. This work shows that we can artificially increase the resolution of some lower-resolution bands by taking advantage of the texture of higher-resolution bands on theGOES-16ABI instrument using a convolutional neural network. This may help reconcile differences in spatial resolution when combining information across bands, but future analysis is needed to precisely determine impacts on derived products that might use superresolved bands.more » « less
-
Point scanning imaging systems (e.g. scanning electron or laser scanning confocal microscopes) are perhaps the most widely used tools for high resolution cellular and tissue imaging. Like all other imaging modalities, the resolution, speed, sample preservation, and signal-to-noise ratio (SNR) of point scanning systems are difficult to optimize simultaneously. In particular, point scanning systems are uniquely constrained by an inverse relationship between imaging speed and pixel resolution. Here we show these limitations can be miti gated via the use of deep learning-based super-sampling of undersampled images acquired on a point-scanning system, which we termed point -scanning super-resolution (PSSR) imaging. Oversampled ground truth images acquired on scanning electron or Airyscan laser scanning confocal microscopes were used to generate semi-synthetictrain ing data for PSSR models that were then used to restore undersampled images. Remarkably, our EM PSSR model was able to restore undersampled images acquired with different optics, detectors, samples, or sample preparation methods in other labs . PSSR enabled previously unattainable xy resolution images with our serial block face scanning electron microscope system. For fluorescence, we show that undersampled confocal images combined with a multiframe PSSR model trained on Airyscan timelapses facilitates Airyscan-equivalent spati al resolution and SNR with ~100x lower laser dose and 16x higher frame rates than corresponding high-resolution acquisitions. In conclusion, PSSR facilitates point-scanning image acquisition with otherwise unattainable resolution, speed, and sensitivity.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1038/s41592-023-02138-w
