This paper introduces a deep neural network based method, i.e., DeepOrganNet, to generate and visualize fully high-fidelity 3D / 4D organ geometric models from single-view medical images with complicated background in real time. Traditional 3D / 4D medical image reconstruction requires near hundreds of projections, which cost insufferable computational time and deliver undesirable high imaging / radiation dose to human subjects. Moreover, it always needs further notorious processes to segment or extract the accurate 3D organ models subsequently. The computational time and imaging dose can be reduced by decreasing the number of projections, but the reconstructed image quality is degraded accordingly. To our knowledge, there is no method directly and explicitly reconstructing multiple 3D organ meshes from a single 2D medical grayscale image on the fly. Given single-view 2D medical images, e.g., 3D / 4D-CT projections or X-ray images, our end-to-end DeepOrganNet framework can efficiently and effectively reconstruct 3D / 4D lung models with a variety of geometric shapes by learning the smooth deformation fields from multiple templates based on a trivariate tensor-product deformation technique, leveraging an informative latent descriptor extracted from input 2D images. The proposed method can guarantee to generate high-quality and high-fidelity manifold meshes for 3D / 4D lung models; while, all current deep learning based approaches on the shape reconstruction from a single image cannot. The major contributions of this work are to accurately reconstruct the 3D organ shapes from 2D single-view projection, significantly improve the procedure time to allow on-the-fly visualization, and dramatically reduce the imaging dose for human subjects. Experimental results are evaluated and compared with the traditional reconstruction method and the state-of-the-art in deep learning, by using extensive 3D and 4D examples, including both synthetic phantom and real patient datasets. The efficiency of the proposed method shows that it only needs several milliseconds to generate organ meshes with 10K vertices, which has great potential to be used in real-time image guided radiation therapy (IGRT).
more »
« less
Accurate stochastic reconstruction of heterogeneous microstructures by limited x‐ray tomographic projections
An accurate knowledge of the complex microstructure of a heterogeneous material is crucial for its performance prediction, prognosis and optimization. X‐ray tomography has provided a nondestructive means for microstructure characterization in 3D and 4D (i.e. structural evolution over time), in which a material is typically reconstructed from a large number of tomographic projections using filtered‐back‐projection (FBP) method or algebraic reconstruction techniques (ART). Here, we present in detail a stochastic optimization procedure that enables one to accurately reconstruct material microstructure from a small number of absorption contrast x‐ray tomographic projections. This discrete tomography reconstruction procedure is in contrast to the commonly used FBP and ART, which usually requires thousands of projections for accurate microstructure rendition. The utility of our stochastic procedure is first demonstrated by reconstructing a wide class of two‐phase heterogeneous materials including sandstone and hard‐particle packing from simulated limited‐angle projections in both cone‐beam and parallel beam projection geometry. It is then applied to reconstruct tailored Sn‐sphere‐clay‐matrix systems from limited‐angle cone‐beam data obtained via a lab‐scale tomography facility at Arizona State University and parallel‐beam synchrotron data obtained at Advanced Photon Source, Argonne National Laboratory. In addition, we examine the information content of tomography data by successively incorporating larger number of projections and quantifying the accuracy of the reconstructions. We show that only a small number of projections (e.g. 20–40, depending on the complexity of the microstructure of interest and desired resolution) are necessary for accurate material reconstructions via our stochastic procedure, which indicates its high efficiency in using limited structural information. The ramifications of the stochastic reconstruction procedure in 4D materials science are also discussed.
more » « less- NSF-PAR ID:
- 10247399
- Publisher / Repository:
- Wiley-Blackwell
- Date Published:
- Journal Name:
- Journal of Microscopy
- Volume:
- 264
- Issue:
- 3
- ISSN:
- 0022-2720
- Page Range / eLocation ID:
- p. 339-350
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
As the use of X-ray computed tomography (CT) grows in medical diagnosis, so does the concern for the harm a radiation dose can cause and the biological risks it represents. StaticCodeCT is a new low-dose imaging architecture that uses a single-static coded aperture (CA) in a CT gantry. It exploits the highly correlated data in the projection domain to estimate the unobserved measurements on the detector. We previously analyzed the StaticCodeCT system by emulating the effect of the coded mask on experimental CT data. In contrast, this manuscript presents test-bed reconstructions using an experimental cone-beam X-ray CT system with a CA holder. We analyzed the reconstruction quality using three different techniques to manufacture the CAs: metal additive manufacturing, cold casting, and ceramic additive manufacturing. Furthermore, we propose an optimization method to design the CA pattern based on the algorithm developed for the measurement estimation. The obtained results point to the possibility of the real deployment of StaticCodeCT systems in practice.
-
more » « less
Abstract Purpose Forward and backprojections are the basis of all model‐based iterative reconstruction (MBIR) methods. However, computing these accurately is time‐consuming. In this paper, we present a method for MBIR in parallel X‐ray beam geometry that utilizes a Gram filter to efficiently implement forward and backprojection.
Methods We propose using voxel‐basis and modeling its footprint in a box spline framework to calculate the Gram filter exactly and improve the performance of backprojection. In the special case of parallel X‐ray beam geometry, the forward and backprojection can be implemented by an estimated Gram filter efficiently if the sinogram signal is bandlimited. In this paper, a specialized sinogram interpolation method is proposed to eliminate the bandlimited prerequisite and thus improve the reconstruction accuracy. We build on this idea by utilizing the continuity of the voxel‐basis' footprint, which provides a more accurate sinogram interpolation and further improves the efficiency and quality of backprojection. In addition, the detector blur effect can be efficiently accounted for in our method to better handle realistic scenarios.
Results The proposed method is tested on both phantom and real computed tomography (CT) images under different resolutions, sinogram sampling steps, and noise levels. The proposed method consistently outperforms other state‐of‐the‐art projection models in terms of speed and accuracy for both backprojection and reconstruction.
Conclusions We proposed a iterative reconstruction methodology for 3D parallel‐beam X‐ray CT reconstruction. Our experimental results demonstrate that the proposed methodology is accurate, fast, and reproducible, and outperforms alternative state‐of‐the‐art projection models on both backprojection and reconstruction results significantly.
-
more » « less
. Computed Tomography (CT) takes X-ray measurements on the subjects to reconstruct tomographic images. As X-ray is radioactive, it is desirable to control the total amount of dose of X-ray for safety concerns. Therefore, we can only select a limited number of measurement angles and assign each of them limited amount of dose. Traditional methods such as compressed sensing usually randomly select the angles and equally distribute the allowed dose on them. In most CT reconstruction models, the emphasize is on designing effective image representations, while much less emphasize is on improving the scanning strategy. The simple scanning strategy of random angle selection and equal dose distribution performs well in general, but they may not be ideal for each individual subject. It is more desirable to design a personalized scanning strategy for each subject to obtain better reconstruction result. In this paper, we propose to use Reinforcement Learning (RL) to learn a personalized scanning policy to select the angles and the dose at each chosen angle for each individual subject. We first formulate the CT scanning process as an Markov Decision Process (MDP), and then use modern deep RL methods to solve it. The learned personalized scanning strategy not only leads to better reconstruction results, but also shows strong generalization to be combined with different reconstruction algorithms.
-
more » « less
Abstract Purpose The constrained one‐step spectral CT image reconstruction (cOSSCIR) algorithm with a nonconvex alternating direction method of multipliers optimizer is proposed for addressing computed tomography (CT) metal artifacts caused by beam hardening, noise, and photon starvation. The quantitative performance of cOSSCIR is investigated through a series of photon‐counting CT simulations.
Methods cOSSCIR directly estimates basis material maps from photon‐counting data using a physics‐based forward model that accounts for beam hardening. The cOSSCIR optimization framework places constraints on the basis maps, which we hypothesize will stabilize the decomposition and reduce streaks caused by noise and photon starvation. Another advantage of cOSSCIR is that the spectral data need not be registered, so that a ray can be used even if some energy window measurements are unavailable. Photon‐counting CT acquisitions of a virtual pelvic phantom with low‐contrast soft tissue texture and bilateral hip prostheses were simulated. Bone and water basis maps were estimated using the cOSSCIR algorithm and combined to form a virtual monoenergetic image for the evaluation of metal artifacts. The cOSSCIR images were compared to a “two‐step” decomposition approach that first estimated basis sinograms using a maximum likelihood algorithm and then reconstructed basis maps using an iterative total variation constrained least‐squares optimization (MLE+TV). Images were also compared to a nonspectral TV reconstruction of the total number of counts detected for each ray with and without normalized metal artifact reduction (NMAR) applied. The simulated metal density was increased to investigate the effects of increasing photon starvation. The quantitative error and standard deviation in regions of the phantom were compared across the investigated algorithms. The ability of cOSSCIR to reproduce the soft‐tissue texture, while reducing metal artifacts, was quantitatively evaluated.
Results Noiseless simulations demonstrated the convergence of the cOSSCIR and MLE+TV algorithms to the correct basis maps in the presence of beam‐hardening effects. When noise was simulated, cOSSCIR demonstrated a quantitative error of −1 HU, compared to 2 HU error for the MLE+TV algorithm and −154 HU error for the nonspectral TV+NMAR algorithm. For the cOSSCIR algorithm, the standard deviation in the central iodine region of interest was 20 HU, compared to 299 HU for the MLE+TV algorithm, 41 HU for the MLE+TV+Mask algorithm that excluded rays through metal, and 55 HU for the nonspectral TV+NMAR algorithm. Increasing levels of photon starvation did not impact the bias or standard deviation of the cOSSCIR images. cOSSCIR was able to reproduce the soft‐tissue texture when an appropriate regularization constraint value was selected.
Conclusions By directly inverting photon‐counting CT data into basis maps using an accurate physics‐based forward model and a constrained optimization algorithm, cOSSCIR avoids metal artifacts due to beam hardening, noise, and photon starvation. The cOSSCIR algorithm demonstrated improved stability and accuracy compared to a two‐step method of decomposition followed by reconstruction.
