skip to main content


Title: A Two-Stage Algorithm for Joint Multimodal Image Reconstruction
We propose a new two-stage joint image reconstruction method by recovering edges directly from observed data and then assembling an image using the recovered edges. More specifically, we reformulate joint image reconstruction with vectorial total-variation regularization as an l1 minimization problem of the Jacobian of the underlying multimodality or multicontrast images. We provide detailed derivation of data fidelity for the Jacobian in Radon and Fourier transform domains. The new minimization problem yields an optimal convergence rate higher than that of existing primaldual based reconstruction algorithms, and the per-iteration cost remains low by using closed-form matrix-valued shrinkages. We conducted numerical tests on a number of multicontrast CT and MR image datasets, which demonstrate that the proposed method significantly improves reconstruction efficiency and accuracy compared to the state-of-the-art joint image reconstruction methods.  more » « less
Award ID(s):
1719932
NSF-PAR ID:
10189024
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
SIAM journal on imaging sciences
Volume:
12
Issue:
3
ISSN:
1936-4954
Page Range / eLocation ID:
1425--1463
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Motivation

    Joint reconstruction of multiple gene regulatory networks (GRNs) using gene expression data from multiple tissues/conditions is very important for understanding common and tissue/condition-specific regulation. However, there are currently no computational models and methods available for directly constructing such multiple GRNs that not only share some common hub genes but also possess tissue/condition-specific regulatory edges.

    Results

    In this paper, we proposed a new graphic Gaussian model for joint reconstruction of multiple gene regulatory networks (JRmGRN), which highlighted hub genes, using gene expression data from several tissues/conditions. Under the framework of Gaussian graphical model, JRmGRN method constructs the GRNs through maximizing a penalized log likelihood function. We formulated it as a convex optimization problem, and then solved it with an alternating direction method of multipliers (ADMM) algorithm. The performance of JRmGRN was first evaluated with synthetic data and the results showed that JRmGRN outperformed several other methods for reconstruction of GRNs. We also applied our method to real Arabidopsis thaliana RNA-seq data from two light regime conditions in comparison with other methods, and both common hub genes and some conditions-specific hub genes were identified with higher accuracy and precision.

    Availability and implementation

    JRmGRN is available as a R program from: https://github.com/wenpingd.

    Supplementary information

    Supplementary data are available at Bioinformatics online.

     
    more » « less
  2. Purpose

    We introduce and validate a scalable retrospective motion correction technique for brain imaging that incorporates a machine learning component into a model‐based motion minimization.

    Methods

    A convolutional neural network (CNN) trained to remove motion artifacts from 2D T2‐weighted rapid acquisition with refocused echoes (RARE) images is introduced into a model‐based data‐consistency optimization to jointly search for 2D motion parameters and the uncorrupted image. Our separable motion model allows for efficient intrashot (line‐by‐line) motion correction of highly corrupted shots, as opposed to previous methods which do not scale well with this refinement of the motion model. Final image generation incorporates the motion parameters within a model‐based image reconstruction. The method is tested in simulations and in vivo motion experiments of in‐plane motion corruption.

    Results

    While the convolutional neural network alone provides some motion mitigation (at the expense of introduced blurring), allowing it to guide the iterative joint‐optimization both improves the search convergence and renders the joint‐optimization separable. This enables rapid mitigation within shots in addition to between shots. For 2D in‐plane motion correction experiments, the result is a significant reduction of both image space root mean square error in simulations, and a reduction of motion artifacts in the in vivo motion tests.

    Conclusion

    The separability and convergence improvements afforded by the combined convolutional neural network+model‐based method shows the potential for meaningful postacquisition motion mitigation in clinical MRI.

     
    more » « less
  3. Abstract

    Whole brain neuroanatomy using tera‐voxel light‐microscopic data sets is of much current interest. A fundamental problem in this field is the mapping of individual brain data sets to a reference space. Previous work has not rigorously quantified in‐vivo to ex‐vivo distortions in brain geometry from tissue processing. Further, existing approaches focus on registering unimodal volumetric data; however, given the increasing interest in the marmoset model for neuroscience research and the importance of addressing individual brain architecture variations, new algorithms are necessary to cross‐register multimodal data sets including MRIs and multiple histological series. Here we present a computational approach for same‐subject multimodal MRI‐guided reconstruction of a series of consecutive histological sections, jointly with diffeomorphic mapping to a reference atlas. We quantify the scale change during different stages of brain histological processing using the Jacobian determinant of the diffeomorphic transformations involved. By mapping the final image stacks to the ex‐vivo post‐fixation MRI, we show that (a) tape‐transfer assisted histological sections can be reassembled accurately into 3D volumes with a local scale change of 2.0 ± 0.4% per axis dimension; in contrast, (b) tissue perfusion/fixation as assessed by mapping the in‐vivo MRIs to the ex‐vivo post fixation MRIs shows a larger median absolute scale change of 6.9 ± 2.1% per axis dimension. This is the first systematic quantification of local metric distortions associated with whole‐brain histological processing, and we expect that the results will generalize to other species. These local scale changes will be important for computing local properties to create reference brain maps.

     
    more » « less
  4. This paper presents a visual servoing method for controlling a robot in the configuration space by purely using its natural features. We first created a data collection pipeline that uses camera intrinsics, extrinsics, and forward kinematics to generate 2D projections of a robot's joint locations (keypoints) in image space. Using this pipeline, we are able to collect large sets of real-robot data, which we use to train realtime keypoint detectors. The inferred keypoints from the trained model are used as control features in an adaptive visual servoing scheme that estimates, in runtime, the Jacobian relating the changes of the keypoints and joint velocities. We compared the 2D configuration control performance of this method to the skeleton-based visual servoing method (the only other algorithm for purely vision-based configuration space visual servoing), and demonstrated that the keypoints provide more robust and less noisy features, which result in better transient response. We also demonstrate the first vision-based 3D configuration space control results in the literature, and discuss its limitations. Our data collection pipeline is available at https://github.com/JaniC-WPI/KPDataGenerator.git which can be utilized to collect image datasets and train realtime keypoint detectors for various robots and environments. 
    more » « less
  5. Electrical Impedance Tomography (EIT) is a well-known imaging technique for detecting the electrical properties of an object in order to detect anomalies, such as conductive or resistive targets. More specifically, EIT has many applications in medical imaging for the detection and location of bodily tumors since it is an affordable and non-invasive method, which aims to recover the internal conductivity of a body using voltage measurements resulting from applying low frequency current at electrodes placed at its surface. Mathematically, the reconstruction of the internal conductivity is a severely ill-posed inverse problem and yields a poor quality image reconstruction. To remedy this difficulty, at least in part, we regularize and solve the nonlinear minimization problem by the aid of a Krylov subspace-type method for the linear sub problem during each iteration. In EIT, a tumor or general anomaly can be modeled as a piecewise constant perturbation of a smooth background, hence, we solve the regularized problem on a subspace of relatively small dimension by the Flexible Golub-Kahan process that provides solutions that have sparse representation. For comparison, we use a well-known modified Gauss–Newton algorithm as a benchmark. Using simulations, we demonstrate the effectiveness of the proposed method. The obtained reconstructions indicate that the Krylov subspace method is better adapted to solve the ill-posed EIT problem and results in higher resolution images and faster convergence compared to reconstructions using the modified Gauss–Newton algorithm. 
    more » « less