Search for: All records

Analysing kinematic and video data can help identify potentially erroneous motions that lead to sub‐optimal surgeon performance and safety‐critical events in robot‐assisted surgery.

We develop a rubric for identifying task and gesture‐specific executional and procedural errors and evaluate dry‐lab demonstrations of suturing and needle passing tasks from the JIGSAWS dataset. We characterise erroneous parts of demonstrations by labelling video data, and use distribution similarity analysis and trajectory averaging on kinematic data to identify parameters that distinguish erroneous gestures.

Executional error frequency varies by task and gesture, and correlates with skill level. Some predominant error modes in each gesture are distinguishable by analysing error‐specific kinematic parameters. Procedural errors could lead to lower performance scores and increased demonstration times but also depend on surgical style.

This study provides insights into context‐dependent errors that can be used to design automated error detection mechanisms and improve training and skill assessment.

 

Purpose: We propose a formal framework for the modeling and segmentation of minimally invasive surgical tasks using a unified set of motion primitives (MPs) to enable more objective labeling and the aggregation of different datasets. Methods: We model dry-lab surgical tasks as finite state machines, representing how the execution of MPs as the basic surgical actions results in the change of surgical context, which characterizes the physical interactions among tools and objects in the surgical environment. We develop methods for labeling surgical context based on video data and for automatic translation of context to MP labels. We then use our framework to create the COntext and Motion Primitive Aggregate Surgical Set (COMPASS), including six dry-lab surgical tasks from three publicly available datasets (JIGSAWS, DESK, and ROSMA), with kinematic and video data and context and MP labels. Results: Our context labeling method achieves near-perfect agreement between consensus labels from crowd-sourcing and expert surgeons. Segmentation of tasks to MPs results in the creation of the COMPASS dataset that nearly triples the amount of data for modeling and analysis and enables the generation of separate transcripts for the left and right tools. Conclusion: The proposed framework results in high quality labeling of surgical data based on context and fine-grained MPs. Modeling surgical tasks with MPs enables the aggregation of different datasets and the separate analysis of left and right hands for bimanual coordination assessment. Our formal framework and aggregate dataset can support the development of explainable and multi-granularity models for improved surgical process analysis, skill assessment, error detection, and autonomy. 

Through computational fluid dynamics (CFD) simulations of a model manta ray body, the hydrodynamic role of manta-like bioinspired flapping is investigated. The manta ray model motion is reconstructed from synchronized high-resolution videos of manta ray swimming. Rotation angles of the model skeletal joints are altered to scale the pitching and bending, resulting in eight models with different pectoral fin pitching and bending ratios. Simulations are performed using an in-house developed immersed boundary method-based numerical solver. Pectoral fin pitching ratio (PR) is found to have significant implications in the thrust and efficiency of the manta model. This occurs due to more optimal vortex formation and shedding caused by the lower pitching ratio. Leading edge vortexes (LEVs) formed on the bottom of the fin, a characteristic of the higher PR cases, produced parasitic low pressure that hinders thrust force. Lowering the PR reduces the influence of this vortex while another LEV that forms on the top surface of the fin strengthens it. A moderately high bending ratio (BR) can slightly reduce power consumption. Finally, by combining a moderately high BR = 0.83 with PR = 0.67, further performance improvements can be made. This enhanced understanding of manta-inspired propulsive mechanics fills a gap in our understanding of the manta-like mobuliform locomotion. This motivates a new generation of manta-inspired robots that can mimic the high speed and efficiency of their biological counterpart 

Sensing is becoming more and more pervasive. New sensing modalities are enabling the collection of data not previously available. Artificial Intelligence (AI) and cognitive assistance technologies are improving rapidly. Cyber Physical Systems (CPS) are making significant progress in utilizing AI and Machine Learning (ML). This confluence of technologies is giving rise to the potential to achieve the vision of ambient intelligence. This paper describes some of the main challenges and research directions for ambient intelligence from a CPS perspective. Index Terms—Ambient Intelligence, Cyber Physical Systems, Cognitive Assistance, Intelligent Systems 

The endoscopic camera of a surgical robot pro- vides surgeons with a magnified 3D view of the surgical field, but repositioning it increases mental workload and operation time. Poor camera placement contributes to safety-critical events when surgical tools move out of the view of the camera. This paper presents a proof of concept of an autonomous camera system for the Raven II surgical robot that aims to reduce surgeon workload and improve safety by providing an optimal view of the workspace showing all objects of interest. This system uses transfer learning to localize and classify objects of interest within the view of a stereoscopic camera. The positions and centroid of the objects are estimated and a set of control rules determines the movement of the camera towards a more desired view. Our perception module had an accuracy of 61.21% overall for identifying objects of interest and was able to localize both graspers and multiple blocks in the environment. Comparison of the commands proposed by our system with the desired commands from a survey of 13 participants indicates that the autonomous camera system proposes appropriate movements for the tilt and pan of the camera. 

Abstract—We present SaSTL—a novel Spatial Aggregation Signal Temporal Logic—for the efficient runtime monitoring of safety and performance requirements in smart cities. We first describe a study of over 1,000 smart city requirements, some of which can not be specified using existing logic such as Signal Temporal Logic (STL) and its variants. To tackle this limitation, we develop two new logical operators in SaSTL to augment STL for expressing spatial aggregation and spatial counting characteristics that are commonly found in real city requirements. We also develop efficient monitoring algorithms that can check a SaSTL requirement in parallel over multiple data streams (e.g., generated by multiple sensors distributed spatially in a city).We evaluate our SaSTL monitor by applying to two case studies with large-scale real city sensing data (e.g., up to 10,000 sensors in one requirement). The results show that SaSTL has a much higher coverage expressiveness than other spatial-temporal logics, and with a significant reduction of computation time for monitoring requirements. We also demonstrate that the SaSTL monitor can help improve the safety and performance of smart cities via simulated experiments.