The emerging potential of augmented reality (AR) to improve 3D medical image visualization for diagnosis, by immersing the user into 3D morphology is further enhanced with the advent of wireless head-mounted displays (HMD). Such information-immersive capabilities may also enhance planning and visualization of interventional procedures. To this end, we introduce a computational platform to generate an augmented reality holographic scene that fuses pre-operative magnetic resonance imaging (MRI) sets, segmented anatomical structures, and an actuated model of an interventional robot for performing MRI-guided and robot-assisted interventions. The interface enables the operator to manipulate the presented images and rendered structures using voice and gestures, as well as to robot control. The software uses forbidden-region virtual fixtures that alerts the operator of collisions with vital structures. The platform was tested with a HoloLens HMD in silico. To address the limited computational power of the HMD, we deployed the platform on a desktop PC with two-way communication to the HMD. Operation studies demonstrated the functionality and underscored the importance of interface customization to fit a particular operator and/or procedure, as well as the need for on-site studies to assess its merit in the clinical realm.
more »
« less
A Platform Integrating Acquisition, Reconstruction, Visualization, and Manipulator Control Modules for MRI-Guided Interventions
This work presents a platform that integrates a customized MRI data acquisition scheme with reconstruction and three-dimensional (3D) visualization modules along with a module for controlling an MRI-compatible robotic device to facilitate the performance of robot-assisted, MRI-guided interventional procedures. Using dynamically-acquired MRI data, the computational framework of the platform generates and updates a 3D model representing the area of the procedure (AoP). To image structures of interest in the AoP that do not reside inside the same or parallel slices, the MRI acquisition scheme was modified to collect a multi-slice set of intraoblique to each other slices; which are termed composing slices. Moreover, this approach interleaves the collection of the composing slices so the same k-space segments of all slices are collected during similar time instances. This time matching of the k-space segments results in spatial matching of the imaged objects in the individual composing slices. The composing slices were used to generate and update the 3D model of the AoP. The MRI acquisition scheme was evaluated with computer simulations and experimental studies. Computer simulations demonstrated that k-space segmentation and time-matched interleaved acquisition of these segments provide spatial matching of the structures imaged with composing slices. Experimental studies used the platform to image the maneuvering of an MRI-compatible manipulator that carried tubing filled with MRI contrast agent. In vivo experimental studies to image the abdomen and contrast enhanced heart on free-breathing subjects without cardiac triggering demonstrated spatial matching of imaged anatomies in the composing planes. The described interventional MRI framework could assist in performing real-time MRI-guided interventions.
more »
« less
- Award ID(s):
- 1646566
- NSF-PAR ID:
- 10082501
- Date Published:
- Journal Name:
- Journal of Digital Imaging
- ISSN:
- 0897-1889
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The emerging potential of augmented reality (AR) to improve 3D medical image visualization for diagnosis, by immersing the user into 3D morphology is further enhanced with the advent of wireless head-mounted displays (HMD). Such information-immersive capabilities may also enhance planning and visualization of interventional procedures. To this end, we introduce a computational platform to generate an augmented reality holographic scene that fuses pre-operative magnetic resonance imaging (MRI) sets, segmented anatomical structures, and an actuated model of an interventional robot for performing MRI-guided and robot-assisted interventions. The interface enables the operator to manipulate the presented images and rendered structures using voice and gestures, as well as to robot control. The software uses forbidden-region virtual fixtures that alerts the operator of collisions with vital structures. The platform was tested with a HoloLens HMD in silico. To address the limited computational power of the HMD, we deployed the platform on a desktop PC with two-way communication to the HMD. Operation studies demonstrated the functionality and underscored the importance of interface customization to fit a particular operator and/or procedure, as well as the need for on-site studies to assess its merit in the clinical realm. Index Terms—augmented reality, robot-assistance, imageguided interventions.more » « less
-
more » « less
Purpose To provide 3D real‐time MRI of speech production with improved spatio‐temporal sharpness using randomized, variable‐density, stack‐of‐spiral sampling combined with a 3D spatio‐temporally constrained reconstruction.
Methods We evaluated five candidate (
k ,t ) sampling strategies using a previously proposed gradient‐echo stack‐of‐spiral sequence and a 3D constrained reconstruction with spatial and temporal penalties. Regularization parameters were chosen by expert readers based on qualitative assessment. We experimentally determined the effect of spiral angle increment andkz temporal order. The strategy yielding highest image quality was chosen as the proposed method. We evaluated the proposed and original 3D real‐time MRI methods in 2 healthy subjects performing speech production tasks that invoke rapid movements of articulators seen in multiple planes, using interleaved 2D real‐time MRI as the reference. We quantitatively evaluated tongue boundary sharpness in three locations at two speech rates.Results The proposed data‐sampling scheme uses a golden‐angle spiral increment in the
kx −ky plane and variable‐density, randomized encoding alongkz . It provided a statistically significant improvement in tongue boundary sharpness score (P < .001) in the blade, body, and root of the tongue during normal and 1.5‐times speeded speech. Qualitative improvements were substantial during natural speech tasks of alternating high, low tongue postures during vowels. The proposed method was also able to capture complex tongue shapes during fast alveolar consonant segments. Furthermore, the proposed scheme allows flexible retrospective selection of temporal resolution.Conclusion We have demonstrated improved 3D real‐time MRI of speech production using randomized, variable‐density, stack‐of‐spiral sampling with a 3D spatio‐temporally constrained reconstruction.
-
more » « less
Real‐time magnetic resonance imaging (RT‐MRI) allows for imaging dynamic processes as they occur, without relying on any repetition or synchronization. This is made possible by modern MRI technology such as fast‐switching gradients and parallel imaging. It is compatible with many (but not all) MRI sequences, including spoiled gradient echo, balanced steady‐state free precession, and single‐shot rapid acquisition with relaxation enhancement. RT‐MRI has earned an important role in both diagnostic imaging and image guidance of invasive procedures. Its unique diagnostic value is prominent in areas of the body that undergo substantial and often irregular motion, such as the heart, gastrointestinal system, upper airway vocal tract, and joints. Its value in interventional procedure guidance is prominent for procedures that require multiple forms of soft‐tissue contrast, as well as flow information. In this review, we discuss the history of RT‐MRI, fundamental tradeoffs, enabling technology, established applications, and current trends.
Level of Evidence 5
Technical Efficacy Stage 1
-
more » « less
Abstract This study presents a particle filter based framework to track cardiac surface from a time sequence of single magnetic resonance imaging (MRI) slices with the future goal of utilizing the presented framework for interventional cardiovascular magnetic resonance procedures, which rely on the accurate and online tracking of the cardiac surface from MRI data. The framework exploits a low-order parametric deformable model of the cardiac surface. A stochastic dynamic system represents the cardiac surface motion. Deformable models are employed to introduce shape prior to control the degree of the deformations. Adaptive filters are used to model complex cardiac motion in the dynamic model of the system. Particle filters are utilized to recursively estimate the current state of the system over time. The proposed method is applied to recover biventricular deformations and validated with a numerical phantom and multiple real cardiac MRI datasets. The algorithm is evaluated with multiple experiments using fixed and varying image slice planes at each time step. For the real cardiac MRI datasets, the average root-mean-square tracking errors of 2.61 mm and 3.42 mm are reported respectively for the fixed and varying image slice planes. This work serves as a proof-of-concept study for modeling and tracking the cardiac surface deformations via a low-order probabilistic model with the future goal of utilizing this method for the targeted interventional cardiac procedures under MR image guidance. For the real cardiac MRI datasets, the presented method was able to track the points-of-interests located on different sections of the cardiac surface within a precision of 3 pixels. The analyses show that the use of deformable cardiac surface tracking algorithm can pave the way for performing precise targeted intracardiac ablation procedures under MRI guidance. The main contributions of this work are twofold. First, it presents a framework for the tracking of whole cardiac surface from a time sequence of single image slices. Second, it employs adaptive filters to incorporate motion information in the tracking of nonrigid cardiac surface motion for temporal coherence.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1007/s10278-018-0152-1
