More Like this

1. Deep-3D microscope: 3D volumetric microscopy of thick scattering samples using a wide-field microscope and machine learning

   https://doi.org/10.1364/BOE.444488  

   Li, Bowen ; Tan, Shiyu ; Dong, Jiuyang ; Lian, Xiaocong ; Zhang, Yongbing ; Ji, Xiangyang ; Veeraraghavan, Ashok ( December 2021 , Biomedical Optics Express)

   Confocal microscopy is a standard approach for obtaining volumetric images of a sample with high axial and lateral resolution, especially when dealing with scattering samples. Unfortunately, a confocal microscope is quite expensive compared to traditional microscopes. In addition, the point scanning in confocal microscopy leads to slow imaging speed and photobleaching due to the high dose of laser energy. In this paper, we demonstrate how the advances in machine learning can be exploited to teach a traditional wide-field microscope, one that’s available in every lab, into producing 3D volumetric images like a confocal microscope. The key idea is to obtain multiple images with different focus settings using a wide-field microscope and use a 3D generative adversarial network (GAN) based neural network to learn the mapping between the blurry low-contrast image stacks obtained using a wide-field microscope and the sharp, high-contrast image stacks obtained using a confocal microscope. After training the network with widefield-confocal stack pairs, the network can reliably and accurately reconstruct 3D volumetric images that rival confocal images in terms of its lateral resolution, z-sectioning and image contrast. Our experimental results demonstrate generalization ability to handle unseen data, stability in the reconstruction results, high spatial resolution even when imaging thick (∼40 microns) highly-scattering samples. We believe that such learning-based microscopes have the potential to bring confocal imaging quality to every lab that has a wide-field microscope.

    

   more » « less
2. Deep Learning-Based Point-Scanning Super-Resolution Imaging

   https://doi.org/10.1101/740548  

   Fang L, Monroe F ( January 2019 , PloS one)

   Point scanning imaging systems (e.g. scanning electron or laser scanning confocal microscopes) are perhaps the most widely used tools for high resolution cellular and tissue imaging. Like all other imaging modalities, the resolution, speed, sample preservation, and signal-to-noise ratio (SNR) of point scanning systems are difficult to optimize simultaneously. In particular, point scanning systems are uniquely constrained by an inverse relationship between imaging speed and pixel resolution. Here we show these limitations can be miti gated via the use of deep learning-based super-sampling of undersampled images acquired on a point-scanning system, which we termed point -scanning super-resolution (PSSR) imaging. Oversampled ground truth images acquired on scanning electron or Airyscan laser scanning confocal microscopes were used to generate semi-synthetictrain ing data for PSSR models that were then used to restore undersampled images. Remarkably, our EM PSSR model was able to restore undersampled images acquired with different optics, detectors, samples, or sample preparation methods in other labs . PSSR enabled previously unattainable xy resolution images with our serial block face scanning electron microscope system. For fluorescence, we show that undersampled confocal images combined with a multiframe PSSR model trained on Airyscan timelapses facilitates Airyscan-equivalent spati al resolution and SNR with ~100x lower laser dose and 16x higher frame rates than corresponding high-resolution acquisitions. In conclusion, PSSR facilitates point-scanning image acquisition with otherwise unattainable resolution, speed, and sensitivity. 

   more » « less
3. Deep learning extended depth-of-field microscope for fast and slide-free histology

   https://doi.org/10.1073/pnas.2013571117  

   Jin, Lingbo ; Tang, Yubo ; Wu, Yicheng ; Coole, Jackson B. ; Tan, Melody T. ; Zhao, Xuan ; Badaoui, Hawraa ; Robinson, Jacob T. ; Williams, Michelle D. ; Gillenwater, Ann M. ; et al ( December 2020 , Proceedings of the National Academy of Sciences)

   null (Ed.)

   Microscopic evaluation of resected tissue plays a central role in the surgical management of cancer. Because optical microscopes have a limited depth-of-field (DOF), resected tissue is either frozen or preserved with chemical fixatives, sliced into thin sections placed on microscope slides, stained, and imaged to determine whether surgical margins are free of tumor cells—a costly and time- and labor-intensive procedure. Here, we introduce a deep-learning extended DOF (DeepDOF) microscope to quickly image large areas of freshly resected tissue to provide histologic-quality images of surgical margins without physical sectioning. The DeepDOF microscope consists of a conventional fluorescence microscope with the simple addition of an inexpensive (less than $10) phase mask inserted in the pupil plane to encode the light field and enhance the depth-invariance of the point-spread function. When used with a jointly optimized image-reconstruction algorithm, diffraction-limited optical performance to resolve subcellular features can be maintained while significantly extending the DOF (200 µm). Data from resected oral surgical specimens show that the DeepDOF microscope can consistently visualize nuclear morphology and other important diagnostic features across highly irregular resected tissue surfaces without serial refocusing. With the capability to quickly scan intact samples with subcellular detail, the DeepDOF microscope can improve tissue sampling during intraoperative tumor-margin assessment, while offering an affordable tool to provide histological information from resected tissue specimens in resource-limited settings. 

   more » « less
4. Intelligent nanoscope for rapid nanomaterial identification and classification

   https://doi.org/10.1039/D2LC00206J  

   Jin, Geonsoo ; Hong, Seongwoo ; Rich, Joseph ; Xia, Jianping ; Kim, Kyeri ; You, Lingchong ; Zhao, Chenglong ; Huang, Tony Jun ( June 2022 , Lab on a Chip)

   Machine learning image recognition and classification of particles and materials is a rapidly expanding field. However, nanomaterial identification and classification are dependent on the image resolution, the image field of view, and the processing time. Optical microscopes are one of the most widely utilized technologies in laboratories across the world, due to their nondestructive abilities to identify and classify critical micro-sized objects and processes, but identifying and classifying critical nano-sized objects and processes with a conventional microscope are outside of its capabilities, due to the diffraction limit of the optics and small field of view. To overcome these challenges of nanomaterial identification and classification, we developed an intelligent nanoscope that combines machine learning and microsphere array-based imaging to: (1) surpass the diffraction limit of the microscope objective with microsphere imaging to provide high-resolution images; (2) provide large field-of-view imaging without the sacrifice of resolution by utilizing a microsphere array; and (3) rapidly classify nanomaterials using a deep convolution neural network. The intelligent nanoscope delivers more than 46 magnified images from a single image frame so that we collected more than 1000 images within 2 seconds. Moreover, the intelligent nanoscope achieves a 95% nanomaterial classification accuracy using 1000 images of training sets, which is 45% more accurate than without the microsphere array. The intelligent nanoscope also achieves a 92% bacteria classification accuracy using 50 000 images of training sets, which is 35% more accurate than without the microsphere array. This platform accomplished rapid, accurate detection and classification of nanomaterials with miniscule size differences. The capabilities of this device wield the potential to further detect and classify smaller biological nanomaterial, such as viruses or extracellular vesicles. 

   more » « less
5. Lensfree time-gated photoluminescent imaging

   https://doi.org/10.1063/5.0148217  

   Baker, Maryam ; McLeod, Euan ( June 2023 , APL Photonics)

   Fluorescence and, more generally, photoluminescence enable high contrast imaging of targeted regions of interest through the use of photoluminescent probes with high specificity for different targets. Fluorescence can be used for rare cell imaging; however, this often requires a high space-bandwidth product: simultaneous high resolution and large field of view. With bulky traditional microscopes, high space-bandwidth product images require time-consuming mechanical scanning and stitching. Lensfree imaging can compactly and cost-effectively achieve a high space-bandwidth product in a single image through computational reconstruction of images from diffraction patterns recorded over the full field of view of standard image sensors. Many methods of lensfree photoluminescent imaging exist, where the excitation light is filtered before the image sensor, often by placing spectral filters between the sample and sensor. However, the sample-to-sensor distance is one of the limiting factors on resolution in lensfree systems and so more competitive performance can be obtained if this distance is reduced. Here, we show a time-gated lensfree photoluminescent imaging system that can achieve a resolution of 8.77 µm. We use europium chelate fluorophores because of their long lifetime (642 µs) and trigger camera exposure ∼50 µs after excitation. Because the excitation light is filtered temporally, there is no need for physical filters, enabling reduced sample-to-sensor distances and higher resolutions. 

   more » « less