An official website of the United States government
Haptic feedback can render real-time force interactions with computer simulated objects. In several telerobotic applications, it is desired that a haptic simulation reflects a physical task space or interaction accurately. This is particularly true when excessive applied force can result in disastrous consequences, as with the case of robot-assisted minimally invasive surgery (RMIS) and tissue damage. Since force cannot be directly measured in RMIS, non-contact methods are desired. A promising direction of non-contact force estimation involves the primary use of vision sensors to estimate deformation. However, the required fidelity of non-contact force rendering of deformable interaction to maintain surgical operator performance is not well established. This work attempts to empirically evaluate the degree to which haptic feedback may deviate from ground truth yet result in acceptable teleoperated performance in a simulated RMIS-based palpation task. A preliminary user-study is conducted to verify the utility of the simulation platform, and the results of this work have implications in haptic feedback for RMIS and inform guidelines for vision-based tool-tissue force estimation. An adaptive thresholding method is used to collect the minimum and maximum tolerable errors in force orientation and magnitude of presented haptic feedback to maintain sufficient performance.
more »
« less
Autonomously Navigating a Surgical Tool Inside the Eye by Learning from Demonstration
A fundamental challenge in retinal surgery is safely navigating a surgical tool to a desired goal position on the retinal surface while avoiding damage to surrounding tissues, a procedure that typically requires tens-of-microns accuracy. In practice, the surgeon relies on depth-estimation skills to localize the tool-tip with respect to the retina and perform the tool-navigation task, which can be prone to human error. To alleviate such uncertainty, prior work has introduced ways to assist the surgeon by estimating the tool-tip distance to the retina and providing haptic or auditory feedback. However, automating the tool-navigation task itself remains unsolved and largely un-explored. Such a capability, if reliably automated, could serve as a building block to streamline complex procedures and reduce the chance for tissue damage. Towards this end, we propose to automate the tool-navigation task by mimicking the perception-action feedback loop of an expert surgeon. Specifically, a deep network is trained to imitate expert trajectories toward various locations on the retina based on recorded visual servoing to a given goal specified by the user. The proposed autonomous navigation system is evaluated in simulation and in real-life experiments using a silicone eye phantom. We show that the network can reliably navigate a surgical tool to various desired locations within 137 µm accuracy in phantom experiments and 94 µm in simulation, and generalizes well to unseen situations such as in the presence of auxiliary surgical tools, variable eye backgrounds, and brightness conditions.
more »
« less
- Award ID(s):
- 1637949
- PAR ID:
- 10136845
- Date Published:
- Journal Name:
- ICRA 2019
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Laparoscopic surgery presents practical benefits over traditional open surgery, including reduced risk of infection, discomfort and recovery time for patients. Introducing robot systems into surgical tasks provides additional enhancements, including improved precision, remote operation, and an intelligent software layer capable of filtering aberrant motion and scaling surgical maneuvers. However, the software interface in telesurgery also lends itself to potential adversarial cyber attacks. Such attacks can negatively effect both surgeon motion commands and sensory information relayed to the operator. To combat cyber attacks on the latter, one method to enhance surgeon feedback through multiple sensory pathways is to incorporate reliable, complementary forms of information across different sensory modes. Built-in partial redundancies or inferences between perceptual channels, or perception complementarities, can be used both to detect and recover from compromised operator feedback. In surgery, haptic sensations are extremely useful for surgeons to prevent undue and unwanted tissue damage from excessive tool-tissue force. Direct force sensing is not yet deployable due to sterilization requirements of the operating room. Instead, combinations of other sensing methods may be relied upon, such as noncontact model-based force estimation. This paper presents the design of a surgical simulator software that can be used for vision-based non-contact force sensing to inform the perception complementarity of vision and force feedback for telesurgery. A brief user study is conducted to verify the efficacy of graphical force feedback from vision-based force estimation, and suggests that vision may effectively complement direct force sensing.more » « less
-
Purpose To investigate relationships between blood pressure and the thickness of single retinal layers in the macula. Methods Participants of the population-based Beijing Eye Study, free of retinal or optic nerve disease, underwent medical and ophthalmological examinations including optical coherence tomographic examination of the macula. Applying a multiple-surface segmentation solution, we automatically segmented the retina into its various layers. Results The study included 2237 participants (mean age 61.8±8.4 years, range 50–93 years). Mean thicknesses of the retinal nerve fibre layer (RNFL), ganglion cell layer (GCL), inner plexiform layer, inner nuclear layer (INL), outer plexiform layer, outer nuclear layer/external limiting membrane, ellipsoid zone, photoreceptor outer segments (POS) and retinal pigment epithelium–Bruch membrane were 31.1±2.3 µm, 39.7±3.5 µm, 38.4±3.3 µm, 34.8±2.0 µm, 28.1±3.0 µm, 79.2±7.3 µm, 22.9±0.6 µm, 19.2±3.3 µm and 20.7±1.4 µm, respectively. In multivariable analysis, higher systolic blood pressure (SBP) and diastolic blood pressure (DBP) were associated with thinner GCL and thicker INL, after adjusting for age, sex and axial length (all p<0.0056). Higher SBP was additionally associated with thinner POS and higher DBP with thinner RNFL. For an elevation of SBP/DBP by 10 mm Hg, the RNFL, GCL, INL and POS changed by 2.0, 3.0, 1.5 and 2.0 µm, respectively. Conclusions Thickness of RNFL, GCL and POS was inversely and INL thickness was positively associated with higher blood pressure, while the thickness of the other retinal layers was not significantly correlated with blood pressure. The findings may be helpful for refinement of the morphometric detection of retinal diseases.more » « less
-
Augmented Reality (AR) is increasingly used in medical applications for visualizing medical information. In this paper, we present an AR-assisted surgical guidance system that aims to improve the accuracy of catheter placement in ventriculostomy, a common neurosurgical procedure. We build upon previous work on neurosurgical AR, which has focused on enabling the surgeon to visualize a patient’s ventricular anatomy, to additionally integrate surgical tool tracking and contextual guidance. Specifically, using accurate tracking of optical markers via an external multi-camera OptiTrack system, we enable Microsoft HoloLens 2-based visualizations of ventricular anatomy, catheter placement, and the information on how far the catheter tip is from its target. We describe the system we developed, present initial hologram registration results, and comment on the next steps that will prepare our system for clinical evaluations.more » « less
-
null (Ed.)The retina has a very high energy demand but lacks an internal blood supply in most vertebrates. Here we explore the hypothesis that oxygen diffusion limited the evolution of retinal morphology by reconstructing the evolution of retinal thickness and the various mechanisms for retinal oxygen supply, including capillarization and acid-induced haemoglobin oxygen unloading. We show that a common ancestor of bony fishes likely had a thin retina without additional retinal oxygen supply mechanisms and that three different types of retinal capillaries were gained and lost independently multiple times during the radiation of vertebrates, and that these were invariably associated with parallel changes in retinal thickness. Since retinal thickness confers multiple advantages to vision, we propose that insufficient retinal oxygen supply constrained the functional evolution of the eye in early vertebrates, and that recurrent origins of additional retinal oxygen supply mechanisms facilitated the phenotypic evolution of improved functional eye morphology.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
