An official website of the United States government
Diffraction-limited optical imaging through scattering media has the potential to transform many applications such as airborne and space-based imaging (through the atmosphere), bioimaging (through skin and human tissue), and fiber-based imaging (through fiber bundles). Existing wavefront shaping methods can image through scattering media and other obscurants by optically correcting wavefront aberrations using high-resolution spatial light modulators—but these methods generally require (i) guidestars, (ii) controlled illumination, (iii) point scanning, and/or (iv) statics scenes and aberrations. We propose neural wavefront shaping (NeuWS), a scanning-free wavefront shaping technique that integrates maximum likelihood estimation, measurement modulation, and neural signal representations to reconstruct diffraction-limited images through strong static and dynamic scattering media without guidestars, sparse targets, controlled illumination, nor specialized image sensors. We experimentally demonstrate guidestar-free, wide field-of-view, high-resolution, diffraction-limited imaging of extended, nonsparse, and static/dynamic scenes captured through static/dynamic aberrations.
more »
« less
Non-parametric Vignetting Correction for Sparse Spatial Transcriptomics Images
Spatial transcriptomics techniques such as STARmap [15] enable the subcellular detection of RNA transcripts within complex tissue sections. The data from these techniques are impacted by optical microscopy limitations, such as shading or vignetting effects from uneven illumination during image capture. Downstream analysis of these sparse spatially resolved transcripts is dependent upon the correction of these artefacts. This paper introduces a novel non-parametric vignetting correction tool for spatial transcriptomic images, which estimates the illumination field and background using an efficient iterative sliced histogram normalization routine. We show that our method outperforms the state-of-the-art shading correction techniques both in terms of illumination and background field estimation and requires fewer input images to perform the estimation adequately. We further demonstrate an important downstream application of our technique, showing that spatial transcriptomic volumes corrected by our method yield a higher and more uniform gene expression spot-calling in the rodent hippocampus. Python code and a demo file to reproduce our results are provided in the supplementary material and at this github page: https://github.com/BoveyRao/Non-parametric-vc-for-sparse-st.
more »
« less
- Award ID(s):
- 1912194
- PAR ID:
- 10297250
- Editor(s):
- de Bruijne, M.
- Date Published:
- Journal Name:
- Medical Image Computing and Computer Assisted Intervention – MICCAI 2021
- Page Range / eLocation ID:
- 466-475
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract. In humid tropical regions, irregular illumination and cloud shadows can complicate near-surface optical remote sensing. This could lead to costly and repetitive surveys to maintain geographical and spectral consistency. This could have a significant impact on the regular monitoring of forest ecosystems. A novel correction method using deep learning is presented here to address the issue in high-resolution canopy images. Our method involves training a deep learning model on one or a few well-illuminated/homogeneous reference images augmented with artificially generated cloud shadows. This training enables the model to predict illumination and cloud shadow patterns in any image and ultimately mitigate these effects. Using images captured by multispectral and RGB cameras, we evaluated the method across multiple sensors and conditions. These included nadir-view images from two sensors mounted on a drone and tower-mounted RGB Phenocams. The technique effectively corrects uneven illumination in near-infrared and true-color RGB images, including non-forested areas. This improvement was also evident in more consistent normalized difference vegetation index (NDVI) patterns in areas affected by uneven illumination. By comparing corrected RGB images to the original in a binary classification task, we evaluated the method's accuracy and Kappa values. Our goal was to detect non-photosynthetic vegetation (NPV) in a mosaic. The overall accuracy and Kappa were both significantly improved in corrected images, with a 2.5% and 1.1% increase, respectively. Moreover, the method can be generalized across sensors and conditions. Further work should focus on refining the technique and exploring its applicability to satellite imagery and beyond.more » « less
-
Abstract Tissue development and disease lead to changes in cellular organization, nuclear morphology, and gene expression, which can be jointly measured by spatial transcriptomic technologies. However, methods for jointly analyzing the different spatial data modalities in 3D are still lacking. We present a computational framework to integrate Spatial Transcriptomic data using over-parameterized graph-based Autoencoders with Chromatin Imaging data (STACI) to identify molecular and functional alterations in tissues. STACI incorporates multiple modalities in a single representation for downstream tasks, enables the prediction of spatial transcriptomic data from nuclear images in unseen tissue sections, and provides built-in batch correction of gene expression and tissue morphology through over-parameterization. We apply STACI to analyze the spatio-temporal progression of Alzheimer’s disease and identify the associated nuclear morphometric and coupled gene expression features. Collectively, we demonstrate the importance of characterizing disease progression by integrating multiple data modalities and its potential for the discovery of disease biomarkers.more » « less
-
Significance: Laparoscopic surgery presents challenges in localizing oncological margins due to poor contrast between healthy and malignant tissues. Optical properties can uniquely identify various tissue types and disease states with high sensitivity and specificity, making it a promising tool for surgical guidance. Although spatial frequency domain imaging (SFDI) effectively measures quantitative optical properties, its deployment in laparoscopy is challenging due to the constrained imaging environment. Thus, there is a need for compact structured illumination techniques to enable accurate, quantitative endogenous contrast in minimally invasive surgery. Aim: We introduce a compact, two-camera laparoscope that incorporates both active stereo depth estimation and speckle-illumination SFDI (si-SFDI) to map profile-corrected, pixel-level absorption (μa), and reduced scattering (μ′s) optical properties in images of tissues with complex geometries. Approach: We used a multimode fiber-coupled 639-nm laser illumination to generate high-contrast speckle patterns on the object. These patterns were imaged through a modified commercial stereo laparoscope for optical property estimation via si-SFDI. Compared with the original si-SFDI work, which required ≥10 images of randomized speckle patterns for accurate optical property estimations, our approach approximates the DC response using a laser speckle reducer (LSR) and consequently requires only two images. In addition, we demonstrate 3D profilometry using active stereo from low-coherence RGB laser flood illumination. Sample topography was then used to correct for measured intensity variations caused by object height and surface angle differences with respect to a calibration phantom. The low-contrast RGB speckle pattern was blurred using an LSR to approximate incoherent white light illumination. We validated profile-corrected si-SFDI against conventional SFDI in phantoms with simple and complex geometries, as well as in a human finger in vivo time-series constriction study. Results: Laparoscopic si-SFDI optical property measurements agreed with conventional SFDI measurements when measuring flat tissue phantoms, exhibiting an error of 6.4% for absorption and 5.8% for reduced scattering. Profile-correction improved the accuracy for measurements of phantoms with complex geometries, particularly for absorption, where it reduced the error by 23.7%. An in vivo finger constriction study further validated laparoscopic si-SFDI, demonstrating an error of 8.2% for absorption and 5.8% for reduced scattering compared with conventional SFDI. Moreover, the observed trends in optical properties due to physiological changes were consistent with previous studies. Conclusions: Our stereo-laparoscopic implementation of si-SFDI provides a simple method to obtain accurate optical property maps through a laparoscope for flat and complex geometries. This has the potential to provide quantitative endogenous contrast for minimally invasive surgical guidance.more » « less
-
Light-sheet fluorescence microscopy (LSFM) provides access to multi-dimensional and multi-scale in vivo imaging of animal models with highly coherent volumetric reconstruction of the tissue morphology, via a focused laser light sheet. The orthogonal illumination and detection LSFM pathways account for minimal photobleaching and deep tissue optical sectioning through different perspective views. Although rotation of the sample and deep tissue scanning constitutes major advantages of LSFM, images may suffer from intrinsic problems within the modality, such as light mismatch of refractive indices between the sample and mounting media and varying quantum efficiency across different depths. To overcome these challenges, we hereby introduce an illumination correction technique integrated with depth detail amelioration to achieve symmetric contrast in large field-of-view images acquired using a low power objective lens. Due to an increase in angular dispersion of emitted light flux with the depth, we combined the dehazing algorithm with morphological operations to enhance poorly separated overlapping structures with subdued intensity. The proposed method was tested on different LSFM modalities to illustrate its applicability on correcting anisotropic illumination affecting the volumetric reconstruction of the fluorescently tagged region of interest.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
