skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Towards MRI-guided and actuated tetherless milli-robots: Preoperative planning and modeling of control
Image-guided and robot-assisted surgical procedures are rapidly evolving due to their potential to improve patient management and cost effectiveness. Magnetic Resonance Imaging (MRI) is used for pre-operative planning and is also investigated for real-time intra-operative guidance. A new type of technology is emerging that uses the magnetic field gradients of the MR scanner to maneuver ferromagnetic agents for local delivery of therapeutics. With this approach, MRI is both a sensor and forms a closed-loop controlled entity that behaves as a robot (we refer to them as MRbots). The objective of this paper is to introduce a computational framework for preoperative planning using MRI and modeling of MRbot maneuvering inside tortuous blood vessels. This platform generates a virtual corridor that represents a safety zone inside the vessel that is then used to access the safety of the MRbot maneuvering. In addition, to improve safety we introduce a control that sets speed based on the local curvature of the vessel. The functionality of the framework was then tested on a realistic operational scenario of accessing a neurological lesion, a meningioma. This virtual case study demonstrated the functionality and potential of MRbots as well as revealed two primary challenges: real-time MRI (during propulsion) and the need of very strong gradients for maneuvering small MRbots inside narrow cerebral vessels. Our ongoing research focuses on further developing the computational core, MR tracking methods, and on-line interfacing to the MR scanner.  more » « less
Award ID(s):
1646566
PAR ID:
10048943
Author(s) / Creator(s):
; ; ; ; ; ;
Date Published:
Journal Name:
Intelligent Robots and Systems (IROS), 2017 IEEE/RSJ International Conference on
Page Range / eLocation ID:
6440 to 6447
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; ; ; (, IEEE International Conference on Bioinformatics and Bioengineering (BIBE))
    The emerging potential of augmented reality (AR) to improve 3D medical image visualization for diagnosis, by immersing the user into 3D morphology is further enhanced with the advent of wireless head-mounted displays (HMD). Such information-immersive capabilities may also enhance planning and visualization of interventional procedures. To this end, we introduce a computational platform to generate an augmented reality holographic scene that fuses pre-operative magnetic resonance imaging (MRI) sets, segmented anatomical structures, and an actuated model of an interventional robot for performing MRI-guided and robot-assisted interventions. The interface enables the operator to manipulate the presented images and rendered structures using voice and gestures, as well as to robot control. The software uses forbidden-region virtual fixtures that alerts the operator of collisions with vital structures. The platform was tested with a HoloLens HMD in silico. To address the limited computational power of the HMD, we deployed the platform on a desktop PC with two-way communication to the HMD. Operation studies demonstrated the functionality and underscored the importance of interface customization to fit a particular operator and/or procedure, as well as the need for on-site studies to assess its merit in the clinical realm. Index Terms—augmented reality, robot-assistance, imageguided interventions. 
    more » « less
  2. ; ; ; ; ; ; ; ; ; (, 2019 IEEE 19th International Conference on Bioinformatics and Bioengineering (BIBE))
    The emerging potential of augmented reality (AR) to improve 3D medical image visualization for diagnosis, by immersing the user into 3D morphology is further enhanced with the advent of wireless head-mounted displays (HMD). Such information-immersive capabilities may also enhance planning and visualization of interventional procedures. To this end, we introduce a computational platform to generate an augmented reality holographic scene that fuses pre-operative magnetic resonance imaging (MRI) sets, segmented anatomical structures, and an actuated model of an interventional robot for performing MRI-guided and robot-assisted interventions. The interface enables the operator to manipulate the presented images and rendered structures using voice and gestures, as well as to robot control. The software uses forbidden-region virtual fixtures that alerts the operator of collisions with vital structures. The platform was tested with a HoloLens HMD in silico. To address the limited computational power of the HMD, we deployed the platform on a desktop PC with two-way communication to the HMD. Operation studies demonstrated the functionality and underscored the importance of interface customization to fit a particular operator and/or procedure, as well as the need for on-site studies to assess its merit in the clinical realm. 
    more » « less
  3. ; ; ; (, International Conference on Automation Science and Engineering (IEEE CASE 22-26 August 2019, Vancouver, Canada))
    Small-scale robots have great potential in medicine, micro-assembly and many other areas. For example, robots containing iron can be steered using the magnetic gradient generated by MRI scanners. Since the gradient is approximately the same everywhere inside the scanner, each robot receives the same input and therefore they all are subjected to the same force. A similar technique can be used with rotating magnetic fields. Each robot receives the same inputs, making motion planning challenging. This paper uses a Rapidly Exploring Random Tree (RRT) to plan paths that deliver multiple robots to goal positions by using obstacles to break the actuation symmetry. 
    more » « less
  4. (, Eurographics Workshop on Visual Computing for Biomedicine)
    Puig Puig, Anna and (Ed.)
    Motivated by the potential of holographic augmented reality (AR) to offer an immersive 3D appreciation of morphology and anatomy, the purpose of this work is to develop and assess an interface for image-based planning of prostate interventions with a head-mounted display (HMD). The computational system is a data and command pipeline that links a magnetic resonance imaging (MRI) scanner/data and the operator, that includes modules dedicated to image processing and segmentation, structure rendering, trajectory planning and spatial co-registration. The interface was developed with the Unity3D Engine (C#) and deployed and tested on a HoloLens HMD. For ergonomics in the surgical suite, the system was endowed with hands-free interactive manipulation of images and the holographic scene via hand gestures and voice commands. The system was tested in silico using MRI and ultrasound datasets of prostate phantoms. The holographic AR scene rendered by the HoloLens HMD was subjectively found superior to desktop-based volume or 3D rendering with regard to structure detection and appreciation of spatial relationships, planning access paths and manual co-registration of MRI and Ultrasound. By inspecting the virtual trajectory superimposed to rendered structures and MR images, the operator observes collisions of the needle path with vital structures (e.g. urethra) and adjusts accordingly. Holographic AR interfacing with wireless HMD endowed with hands-free gesture and voice control is a promising technology. Studies need to systematically assess the clinical merit of such systems and needed functionalities. 
    more » « less
  5. ; ; ; (, IEEE Robotics and Automation Letters)
    Untethered magnetic navigation of millirobots within a human body using a magnetic resonance imaging (MRI) scanner is a promising technology for minimally invasive surgery or drug delivery. Because MRI scanners have a large static magnetic field, they cannot generate torque on magnetic millirobots and must instead use gradient-based pulling. However, gradient values are too small to produce forces large enough to penetrate tissue. This letter presents a method to produce large pulsed forces on millirobots. A ferromagnetic sphere is placed inside a hollow robot body and can move back and forth. This movement is created by alternating the magnetic gradient direction. On the posterior side, a spring allows the sphere to change direction smoothly. On the anterior side, a hard rod creates a surface for the sphere to impact. This impact results in a large pulsed force. The purpose of this study was to understand the functioning of magnetic hammer actuation and control, as well as demonstrate the viability of this mechanism for tissue penetration. This letter begins with modeling and simulating this system. Next, different control strategies are presented and tested. The system successfully penetrated lamb brain samples. Finally, preliminary tests inside a clinical MRI scanner demonstrate the potential of this actuation system. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email