More Like this

1. An Open-Source Photogrammetry Workflow for Reconstructing 3D Models

   https://doi.org/10.1093/iob/obad024  

   Zhang, C; Maga, A M (January 2023, Integrative Organismal Biology)

   Synopsis Acquiring accurate 3D biological models efficiently and economically is important for morphological data collection and analysis in organismal biology. In recent years, structure-from-motion (SFM) photogrammetry has become increasingly popular in biological research due to its flexibility and being relatively low cost. SFM photogrammetry registers 2D images for reconstructing camera positions as the basis for 3D modeling and texturing. However, most studies of organismal biology still relied on commercial software to reconstruct the 3D model from photographs, which impeded the adoption of this workflow in our field due the blocking issues such as cost and affordability. Also, prior investigations in photogrammetry did not sufficiently assess the geometric accuracy of the models reconstructed. Consequently, this study has two goals. First, we presented an affordable and highly flexible SFM photogrammetry pipeline based on the open-source package OpenDroneMap (ODM) and its user interface WebODM. Second, we assessed the geometric accuracy of the photogrammetric models acquired from the ODM pipeline by comparing them to the models acquired via microCT scanning, the de facto method to image skeleton. Our sample comprised 15 Aplodontia rufa (mountain beaver) skulls. Using models derived from microCT scans of the samples as reference, our results showed that the geometry of the models derived from ODM was sufficiently accurate for gross metric and morphometric analysis as the measurement errors are usually around or below 2%, and morphometric analysis captured consistent patterns of shape variations in both modalities. However, subtle but distinct differences between the photogrammetric and microCT-derived 3D models could affect the landmark placement, which in return affected the downstream shape analysis, especially when the variance within a sample is relatively small. At the minimum, we strongly advise not combining 3D models derived from these two modalities for geometric morphometric analysis. Our findings can be indictive of similar issues in other SFM photogrammetry tools since the underlying pipelines are similar. We recommend that users run a pilot test of geometric accuracy before using photogrammetric models for morphometric analysis. For the research community, we provide detailed guidance on using our pipeline for building 3D models from photographs. 

   more » « less
2. The radial component of the local Galactic magnetic field in 3D

   https://doi.org/10.1093/mnras/stag640  

   McCallum, Lewis; Frank, Philipp; Hutschenreuter, Sebastian; Benjamin, Robert; Booth, Rebecca A; Clark, Susan E; Haverkorn, Marijke; Hill, Alex S; Mertsch, Philipp; Ordog, Anna; et al (April 2026, Monthly Notices of the Royal Astronomical Society)

   ABSTRACT We present a distance-resolved reconstruction of the local line-of-sight Galactic magnetic field, $$B_{||}$$, by combining a 3D electron density ($$n_{\mathrm{ e}}$$) map derived from dust map-informed simulations and a full-sky map of Faraday rotation measure (RM). The forward model evaluates RM on the same 3D grid as the $$n_{\mathrm{ e}}$$ map and compares to the Galactic Faraday rotation sky. We infer $$B_{||}$$ with a Gaussian-process prior whose power spectrum is inferred from the data using geometric variational inference. The result is a local (within 1.25 kpc where $$|b| \gt 5^{\circ }$$) map of $$B_{||}$$ with uncertainties. The reconstructed RM sky reproduces prominent features of Faraday rotation sky, with a root mean square average strength of $$B_{||}$$ of $$1.63\pm 0.16\,\rm \mu G$$. In face-on views, the magnetic field exhibits coherent patches with alternating sign and hints of kpc-scale modulations, but with significant structure seen on scales of order 100 pc. The $$B_{||}$$ field is seen to exhibit a 3D power spectrum with an average slope of $$-2.73 \pm 0.19$$. We validate our $$B_{||}$$ reconstruction with Galactic pulsars. Predicted RMs (computed by integrating $$n_{e}B_{||}$$ to each pulsar’s distance) correlates with observed RMs, and predicted dispersion measures (DMs) from the $$n_{e}$$ map also correlate with measured DMs, albeit with significant scatter. 

   more » « less
3. SADIR: Shape-Aware Diffusion Models for 3D Image Reconstruction

   https://doi.org/10.1007/978-3-031-46914-5_23  

   Jayakumar, N; Hossain, T; Zhang, M (October 2023, International MICCAI Workshop on Shape in Medical Imaging)

   3D image reconstruction from a limited number of 2D images has been a long-standing challenge in computer vision and image analysis. While deep learning-based approaches have achieved impressive performance in this area, existing deep networks often fail to effectively utilize the shape structures of objects presented in images. As a result, the topology of reconstructed objects may not be well preserved, leading to the presence of artifacts such as discontinuities, holes, or mismatched connections between different parts. In this paper, we propose a shape-aware network based on diffusion models for 3D image reconstruction, named SADIR, to address these issues. In contrast to previous methods that primarily rely on spatial correlations of image intensities for 3D reconstruction, our model leverages shape priors learned from the training data to guide the reconstruction process. To achieve this, we develop a joint learning network that simultaneously learns a mean shape under deformation models. Each reconstructed image is then considered as a deformed variant of the mean shape. We validate our model, SADIR, on both brain and cardiac magnetic resonance images (MRIs). Experimental results show that our method outperforms the baselines with lower reconstruction error and better preservation of the shape structure of objects within the images. 

   more » « less
4. SADIR: Shape-Aware Diffusion Models for 3D Image Reconstruction

   Jayakumar, N; Hossain, T; Zhang, M (October 2023, Springer)

   3D image reconstruction from a limited number of 2D images has been a long-standing challenge in computer vision and image analysis. While deep learning-based approaches have achieved impressive performance in this area, existing deep networks often fail to effectively utilize the shape structures of objects presented in images. As a result, the topology of reconstructed objects may not be well preserved, leading to the presence of artifacts such as discontinuities, holes, or mismatched connections between different parts. In this paper, we propose a shape-aware network based on diffusion models for 3D image reconstruction, named SADIR, to address these issues. In contrast to previous methods that primarily rely on spatial correlations of image intensities for 3D reconstruction, our model leverages shape priors learned from the training data to guide the reconstruction process. To achieve this, we develop a joint learning network that simultaneously learns a mean shape under deformation models. Each reconstructed image is then considered as a deformed variant of the mean shape. We validate our model, SADIR, on both brain and cardiac magnetic resonance images (MRIs). Experimental results show that our method outperforms the baselines with lower reconstruction error and better preservation of the shape structure of objects within the images. 

   more » « less
5. Diffusion Models To Predict 3D Late Mechanical Activation From Sparse 2D Cardiac MRIs

   Jayakumar, N; Xing, J; Hossain, T; Bilchick, K; Epstein, F; Zhang, M (December 2023, Proceedings of Machine Learning for Healthcare)

   Identifying regions of late mechanical activation (LMA) of the left ventricular (LV) myocardium is critical in determining the optimal pacing site for cardiac resynchronization therapy in patients with heart failure. Several deep learning-based approaches have been developed to predict 3D LMA maps of LV myocardium from a stack of sparse 2D cardiac magnetic resonance imaging (MRIs). However, these models often loosely consider the geometric shape structure of the myocardium. This makes the reconstructed activation maps suboptimal; hence leading to a reduced accuracy of predicting the late activating regions of hearts. In this paper, we propose to use shape-constrained diffusion models to better reconstruct a 3D LMA map, given a limited number of 2D cardiac MRI slices. In contrast to previous methods that primarily rely on spatial correlations of image intensities for 3D reconstruction, our model leverages object shape as priors learned from the training data to guide the reconstruction process. To achieve this, we develop a joint learning network that simultaneously learns a mean shape under deformation models. Each reconstructed image is then considered as a deformed variant of the mean shape. To validate the performance of our model, we train and test the proposed framework on a publicly available mesh dataset of 3D myocardium and compare it with state-of-the-art deep learning-based reconstruction models. Experimental results show that our model achieves superior performance in reconstructing the 3D LMA maps as compared to the state-of-the-art models. 

   more » « less