More Like this

1. 3D PHASE RETRIEVAL AT NANO-SCALE VIA ACCELERATED WIRTINGER FLOW

   Fabian, Zalan; Haldar, Justin; Leahy, Richard; Soltanolkotabi, Mahdi (January 2020, EUSIPCO)

   Imaging 3D nano-structures at very high resolution is crucial in a variety of scientific fields. However, due to fundamental limitations of light propagation we can only measure the object indirectly via 2D intensity measurements of the 3D specimen through highly nonlinear projection mappings where a variety of information (including phase) is lost. Reconstruction therefore involves inverting highly nonlinear and seemingly non-invertible mappings. In this paper, we introduce a novel technique where the 3D object is directly reconstructed from an accurate non-linear propagation model. Furthermore, we characterize the ambiguities of this model and leverage a priori knowledge to mitigate their effect and also significantly reduce the required number of measurements and hence the acquisition time. We demonstrate the performance of our algorithm via numerical experiments aimed at nano-scale reconstruction of 3D integrated circuits. Moreover, we provide rigorous theoretical guarantees for convergence to stationarity. 

   more » « less
2. DeepInMiniscope: Deep learning–powered physics-informed integrated miniscope

   https://doi.org/10.1126/sciadv.adr6687  

   Tian, Feng; Mattison, Ben; Yang, Weijian (September 2025, Science Advances)

   Mask-based integrated fluorescence microscopy is a compact imaging technique for biomedical research. It can perform snapshot 3D imaging through a thin optical mask with a scalable field of view (FOV). Integrated microscopy uses computational algorithms for object reconstruction, but efficient reconstruction algorithms for large-scale data have been lacking. Here, we developed DeepInMiniscope, a miniaturized integrated microscope featuring a custom-designed optical mask and an efficient physics-informed deep learning model that markedly reduces computational demand. Parts of the 3D object can be individually reconstructed and combined. Our deep learning algorithm can reconstruct object volumes over 4 millimeters by 6 millimeters by 0.6 millimeters. We demonstrated substantial improvement in both reconstruction quality and speed compared to traditional methods for large-scale data. Notably, we imaged neuronal activity with near-cellular resolution in awake mouse cortex, representing a substantial leap over existing integrated microscopes. DeepInMiniscope holds great promise for scalable, large-FOV, high-speed, 3D imaging applications with compact device footprint. 

   more » « less
3. A single camera unit-based three-dimensional surface imaging technique

   https://doi.org/10.1007/s00170-023-11866-4  

   Wang, Yinhe; Guo, Xiangyu; Kim, Jungsub; Lin, Pengfei; Lu, Kuan; Lee, Hyunjae; Lee, ChaBum (August 2023, The International Journal of Advanced Manufacturing Technology)

   This paper introduces a simple three-dimensional (3D) stereoscopic method using a single unit of an imaging device consisting of a charge-coupled device (CCD) and a zoom lens. Unlike conventional stereoscopy, which requires a pair of imaging devices, 3D surface imaging is achieved by 3D image reconstruction of two images obtained from two different camera positions by scanning. The experiments were performed by obtaining two images of the measurement target in two different ways: (1) by moving the object while the imaging device is stationary, and (2) by moving the imaging device while the object is stationary. Conventional stereoscopy is limited by disparity errors in 3D image reconstruction because a pair of imaging devices is not ideally identical and alignment errors are always present in the imaging system setup. The proposed method significantly reduced the disparity error in 3D image reconstruction, and the calibration process of the imaging system became simple and convenient. The proposed imaging system showed a disparity error of 0.26 in the camera pixel. 

   more » « less
4. 3D object tracking using integral imaging with mutual information and Bayesian optimization

   https://doi.org/10.1364/OE.517312  

   Wani, Pranav; Usmani, Kashif; Krishnan, Gokul; Javidi, Bahram (February 2024, Optics Express)

   Integral imaging has proven useful for three-dimensional (3D) object visualization in adverse environmental conditions such as partial occlusion and low light. This paper considers the problem of 3D object tracking. Two-dimensional (2D) object tracking within a scene is an active research area. Several recent algorithms use object detection methods to obtain 2D bounding boxes around objects of interest in each frame. Then, one bounding box can be selected out of many for each object of interest using motion prediction algorithms. Many of these algorithms rely on images obtained using traditional 2D imaging systems. A growing literature demonstrates the advantage of using 3D integral imaging instead of traditional 2D imaging for object detection and visualization in adverse environmental conditions. Integral imaging’s depth sectioning ability has also proven beneficial for object detection and visualization. Integral imaging captures an object’s depth in addition to its 2D spatial position in each frame. A recent study uses integral imaging for the 3D reconstruction of the scene for object classification and utilizes the mutual information between the object’s bounding box in this 3D reconstructed scene and the 2D central perspective to achieve passive depth estimation. We build over this method by using Bayesian optimization to track the object’s depth in as few 3D reconstructions as possible. We study the performance of our approach on laboratory scenes with occluded objects moving in 3D and show that the proposed approach outperforms 2D object tracking. In our experimental setup, mutual information-based depth estimation with Bayesian optimization achieves depth tracking with as few as two 3D reconstructions per frame which corresponds to the theoretical minimum number of 3D reconstructions required for depth estimation. To the best of our knowledge, this is the first report on 3D object tracking using the proposed approach. 

   more » « less
5. NeurEPDiff: Neural Operators to Predict Geodesics in Deformation Spaces

   https://doi.org/10.1007/978-3-031-34048-2_45  

   Wu, N; Zhang, M (June 2023, Information processing in medical imaging)

   This paper presents NeurEPDiff, a novel network to fast predict the geodesics in deformation spaces generated by a well known Euler-Poincaré differential equation (EPDiff). To achieve this, we develop a neural operator that for the first time learns the evolving trajectory of geodesic deformations parameterized in the tangent space of diffeomorphisms (a.k.a velocity fields). In contrast to previous methods that purely fit the training images, our proposed NeurEPDiff learns a nonlinear mapping function between the time-dependent velocity fields. A composition of integral operators and smooth activation functions is formulated in each layer of NeurEPDiff to effectively approximate such mappings. The fact that NeurEPDiff is able to rapidly provide the numerical solution of EPDiff (given any initial condition) results in a significantly reduced computational cost of geodesic shooting of diffeomorphisms in a high-dimensional image space. Additionally, the properties of discretization/resolution-invariant of NeurEPDiff make its performance generalizable to multiple image resolutions after being trained offline. We demonstrate the effectiveness of NeurEPDiff in registering two image datasets: 2D synthetic data and 3D brain resonance imaging (MRI). The registration accuracy and computational efficiency are compared with the state-of-the-art diffeomophic registration algorithms with geodesic shooting. 

   more » « less