More Like this

1. High-speed two-photon microscopy with adaptive line-excitation

   https://doi.org/10.1101/2024.01.24.577149  

   Li, Y; Guo, S; Mattison, B; Yang, W (January 2024, bioRxiv)

   Abstract We present a two-photon fluorescence microscope designed for high-speed imaging of neural activity in cellular resolution. Our microscope uses a new adaptive sampling scheme with line illumination. Instead of building images pixel by pixel via scanning a diffraction-limited spot across the sample, our scheme only illuminates the regions of interest (i.e., neuronal cell bodies), and samples a large area of them in a single measurement. Such a scheme significantly increases the imaging speed and reduces the overall laser power on the brain tissue. Using this approach, we performed high-speed imaging of the neural activity of mouse cortexin vivo. Our method provides a new sampling strategy in laser-scanning two-photon microscopy, and will be powerful for high-throughput imaging of neural activity. 

   more » « less
2. High-speed two-photon microscopy with adaptive line-excitation

   https://doi.org/10.1364/OPTICA.529930  

   Li, Yunyang; Guo, Shu; Mattison, Ben; Hu, Junjie; Man, Kwun_Nok_Mimi; Yang, Weijian (August 2024, Optica)

   We present a two-photon fluorescence microscope designed for high-speed imaging of neural activity at cellular resolution. Our microscope uses an adaptive sampling scheme with line illumination. Instead of building images pixel by pixel via scanning a diffraction-limited spot across the sample, our scheme only illuminates the regions of interest (i.e., neuronal cell bodies) and samples a large area of them in a single measurement. Such a scheme significantly increases the imaging speed and reduces the overall laser power on the brain tissue. Using this approach, we performed high-speed imaging of the neuronal activity in mouse cortexin vivo. Our method provides a sampling strategy in laser-scanning two-photon microscopy and will be powerful for high-throughput imaging of neural activity. 

   more » « less
3. Deep Learning-Based Point-Scanning Super-Resolution Imaging

   https://doi.org/10.1101/740548  

   Fang L, Monroe F (January 2019, PloS one)

   Point scanning imaging systems (e.g. scanning electron or laser scanning confocal microscopes) are perhaps the most widely used tools for high resolution cellular and tissue imaging. Like all other imaging modalities, the resolution, speed, sample preservation, and signal-to-noise ratio (SNR) of point scanning systems are difficult to optimize simultaneously. In particular, point scanning systems are uniquely constrained by an inverse relationship between imaging speed and pixel resolution. Here we show these limitations can be miti gated via the use of deep learning-based super-sampling of undersampled images acquired on a point-scanning system, which we termed point -scanning super-resolution (PSSR) imaging. Oversampled ground truth images acquired on scanning electron or Airyscan laser scanning confocal microscopes were used to generate semi-synthetictrain ing data for PSSR models that were then used to restore undersampled images. Remarkably, our EM PSSR model was able to restore undersampled images acquired with different optics, detectors, samples, or sample preparation methods in other labs . PSSR enabled previously unattainable xy resolution images with our serial block face scanning electron microscope system. For fluorescence, we show that undersampled confocal images combined with a multiframe PSSR model trained on Airyscan timelapses facilitates Airyscan-equivalent spati al resolution and SNR with ~100x lower laser dose and 16x higher frame rates than corresponding high-resolution acquisitions. In conclusion, PSSR facilitates point-scanning image acquisition with otherwise unattainable resolution, speed, and sensitivity. 

   more » « less
4. Pix2HDR - A pixel-wise acquisition and deep learning-based synthesis approach for high-speed HDR videos

   https://doi.org/10.1109/TPAMI.2024.3410140  

   Wang, Caixin; Zhang, Jie; Wilson, Matthew A; Etienne-Cummings, Ralph (January 2024, IEEE Transactions on Pattern Analysis and Machine Intelligence)

   Abstract—Accurately capturing dynamic scenes with wideranging motion and light intensity is crucial for many vision applications. However, acquiring high-speed high dynamic range (HDR) video is challenging because the camera’s frame rate restricts its dynamic range. Existing methods sacrifice speed to acquire multi-exposure frames. Yet, misaligned motion in these frames can still pose complications for HDR fusion algorithms, resulting in artifacts. Instead of frame-based exposures, we sample the videos using individual pixels at varying exposures and phase offsets. Implemented on a monochrome pixel-wise programmable image sensor, our sampling pattern captures fast motion at a high dynamic range. We then transform pixel-wise outputs into an HDR video using end-to-end learned weights from deep neural networks, achieving high spatiotemporal resolution with minimized motion blurring. We demonstrate aliasing-free HDR video acquisition at 1000 FPS, resolving fast motion under low-light conditions and against bright backgrounds — both challenging conditions for conventional cameras. By combining the versatility of pixel-wise sampling patterns with the strength of deep neural networks at decoding complex scenes, our method greatly enhances the vision system’s adaptability and performance in dynamic conditions. Index Terms—High-dynamic-range video, high-speed imaging, CMOS image sensors, programmable sensors, deep learning, convolutional neural networks. 

   more » « less
5. Deep-3D microscope: 3D volumetric microscopy of thick scattering samples using a wide-field microscope and machine learning

   https://doi.org/10.1364/BOE.444488  

   Li, Bowen; Tan, Shiyu; Dong, Jiuyang; Lian, Xiaocong; Zhang, Yongbing; Ji, Xiangyang; Veeraraghavan, Ashok (December 2021, Biomedical Optics Express)

   Confocal microscopy is a standard approach for obtaining volumetric images of a sample with high axial and lateral resolution, especially when dealing with scattering samples. Unfortunately, a confocal microscope is quite expensive compared to traditional microscopes. In addition, the point scanning in confocal microscopy leads to slow imaging speed and photobleaching due to the high dose of laser energy. In this paper, we demonstrate how the advances in machine learning can be exploited to teach a traditional wide-field microscope, one that’s available in every lab, into producing 3D volumetric images like a confocal microscope. The key idea is to obtain multiple images with different focus settings using a wide-field microscope and use a 3D generative adversarial network (GAN) based neural network to learn the mapping between the blurry low-contrast image stacks obtained using a wide-field microscope and the sharp, high-contrast image stacks obtained using a confocal microscope. After training the network with widefield-confocal stack pairs, the network can reliably and accurately reconstruct 3D volumetric images that rival confocal images in terms of its lateral resolution, z-sectioning and image contrast. Our experimental results demonstrate generalization ability to handle unseen data, stability in the reconstruction results, high spatial resolution even when imaging thick (∼40 microns) highly-scattering samples. We believe that such learning-based microscopes have the potential to bring confocal imaging quality to every lab that has a wide-field microscope. 

   more » « less