More Like this

1. Online Recognition of Bimanual Coordination Provides Important Context for Movement Data in Bimanual Teleoperated Robots

   Boehm, J.; Fey, N.; Majewicz Fey, A. (September 2021, IEEE International Conference on Robotics and Automation)

   null (Ed.)

   An important problem in designing human-robot systems is the integration of human intent and performance in the robotic control loop, especially in complex tasks. Bimanual coordination is a complex human behavior that is critical in many fine motor tasks, including robot-assisted surgery. To fully leverage the capabilities of the robot as an intelligent and assistive agent, online recognition of bimanual coordination could be important. Robotic assistance for a suturing task, for example, will be fundamentally different during phases when the suture is wrapped around the instrument (i.e., making a c- loop), than when the ends of the suture are pulled apart. In this study, we develop an online recognition method of bimanual coordination modes (i.e., the directions and symmetries of right and left hand movements) using geometric descriptors of hand motion. We (1) develop this framework based on ideal trajectories obtained during virtual 2D bimanual path following tasks performed by human subjects operating Geomagic Touch haptic devices, (2) test the offline recognition accuracy of bi- manual direction and symmetry from human subject movement trials, and (3) evalaute how the framework can be used to characterize 3D trajectories of the da Vinci Surgical System’s surgeon-side manipulators during bimanual surgical training tasks. In the human subject trials, our geometric bimanual movement classification accuracy was 92.3% for movement direction (i.e., hands moving together, parallel, or away) and 86.0% for symmetry (e.g., mirror or point symmetry). We also show that this approach can be used for online classification of different bimanual coordination modes during needle transfer, making a C loop, and suture pulling gestures on the da Vinci system, with results matching the expected modes. Finally, we discuss how these online estimates are sensitive to task environment factors and surgeon expertise, and thus inspire future work that could leverage adaptive control strategies to enhance user skill during robot-assisted surgery. 

   more » « less
2. Evaluation of Point-to-Point Reaching Performance in Mixed Reality and Virtual Reality

   https://doi.org/10.1162/pres_a_00418  

   Petrie, Tyler G; Zhao, Chen; Fu, Michael J (September 2024, PRESENCE: Virtual and Augmented Reality)

   Abstract Recent immersive mixed reality (MR) and virtual reality (VR) displays enable users to use their hands to interact with both veridical and virtual environments simultaneously. Therefore, it becomes important to understand the performance of human hand-reaching movement in MR. Studies have shown that different virtual environment visualization modalities can affect point-to-point reaching performance using a stylus, but it is not yet known if these effects translate to direct human-hand interactions in mixed reality. This paper focuses on evaluating human point-to-point motor performance in MR and VR for both finger-pointing and cup-placement tasks. Six performance measures relevant to haptic interface design were measured for both tasks under several different visualization conditions (“MR with indicator,” “MR without indicator,” and “VR”) to determine what factors contribute to hand-reaching performance. A key finding was evidence of a trade-off between reaching “motion confidence” measures (indicated by throughput, number of corrective movements, and peak velocity) and “accuracy” measures (indicated by end-point error and initial movement error). Specifically, we observed that participants tended to be more confident in the “MR without Indicator” condition for finger-pointing tasks. These results contribute critical knowledge to inform the design of VR/MR interfaces based on the application's user performance requirements. 

   more » « less
3. Multi-agent Crowd Simulation in an Active Shooter Environment

   https://doi.org/10.1007/978-3-031-06015-1_8  

   Sharma, S. (May 2022, Lecture notes in computer science)

   Chen, J.Y.C. (Ed.)

   In recent years there has been a sharp increase in active shooter events, but there has been no introduction of new technology or tactics capable of increasing preparedness and training for active shooter events. This has raised a major concern about the lack of tools that would allow robust predictions of realistic human movements and the lack of understanding about the interaction in designated simulation environments. It is impractical to carry out live experiments where thousands of people are evacuated from buildings designed for every possible emergency condition. There has been progress in understanding human movement, human motion synthesis, crowd dynamics, indoor environments, and their relationships with active shooter events, but challenges remain. This paper presents a virtual reality (VR) experimental setup for conducting virtual evacuation drills in response to extreme events and demonstrates the behavior of agents during an active shooter environment. The behavior of agents is implemented using behavior trees in the Unity gaming engine. The VR experimental setup can simulate human behavior during an active shooter event in a campus setting. A presence questionnaire (PQ) was used in the user study to evaluate the effectiveness and engagement of our active shooter environment. The results show that majority of users agreed that the sense of presence was increased when using the emergency response training environment for a building evacuation environment. 

   more » « less
4. Citizen ASAS-SN Data Release. I. Variable Star Classification Using Citizen Science

   https://doi.org/10.1088/1538-3873/ac44f0  

   Christy, C. T.; Jayasinghe, T.; Stanek, K. Z.; Kochanek, C. S.; Way, Z.; Prieto, J. L.; Shappee, B. J.; Holoien, T. W.-S.; Thompson, T. A.; Schneider, A. (February 2022, Publications of the Astronomical Society of the Pacific)

   Abstract We present the first results from Citizen ASAS-SN, a citizen science project for the All-Sky Automated Survey for Supernovae (ASAS-SN) hosted on the Zooniverse platform. Citizen ASAS-SN utilizes the newer, deeper, higher cadence ASAS-SN g -band data and tasks volunteers to classify periodic variable star candidates based on their phased light curves. We started from 40,640 new variable candidates from an input list of ∼7.4 million stars with δ < −60° and the volunteers identified 10,420 new discoveries which they classified as 4234 pulsating variables, 3132 rotational variables, 2923 eclipsing binaries, and 131 variables flagged as Unknown. They classified known variable stars with an accuracy of 89% for pulsating variables, 81% for eclipsing binaries, and 49% for rotational variables. We examine user performance, agreement between users, and compare the citizen science classifications with our machine learning classifier updated for the g -band light curves. In general, user activity correlates with higher classification accuracy and higher user agreement. We used the user’s “Junk” classifications to develop an effective machine learning classifier to separate real from false variables, and there is a clear path for using this “Junk” training set to significantly improve our primary machine learning classifier. We also illustrate the value of Citizen ASAS-SN for identifying unusual variables with several examples. 

   more » « less
5. Developing scalable hands-on virtual and mixed-reality science labs

   https://doi.org/10.1007/s10055-024-01062-4  

   Jiang, Yuanyuan; Hamadani, Kambiz; Ng, Karno; Ahmadinia, Ali; Aquino, Ariel; Palacio, Ryann; Huang, Jane; Macshane, Jared; Hadaegh, Ahmad (November 2024, Virtual Reality)

   Recent innovations in virtual and mixed-reality (VR/MR) technologies have enabled innovative hands-on training applications in high-risk/high-value fields such as medicine, flight, and worker-safety. Here, we present a detailed description of a novel VR/MR tactile user interactions/interface (TUI) hardware and software development framework that enables the rapid and cost-effective no-code development, optimization, and distribution of fully authentic hands-on VR/MR laboratory training experiences in the physical and life sciences. We applied our framework to the development and optimization of an introductory pipette calibration activity that is often carried out in real chemistry and biochemistry labs. Our approach provides users with nuanced real-time feedback on both their psychomotor skills during data acquisition and their attention to detail when conducting data analysis procedures. The cost-effectiveness of our approach relative to traditional face-to-face science labs improves access to quality hands-on science lab experiences. Importantly, the no-code nature of this Hands-On Virtual-Reality (HOVR) Lab platform enables faculties to iteratively optimize VR/MR experiences to meet their student’s targeted needs without costly software development cycles. Our platform also accommodates TUIs using either standard virtual-reality controllers (VR TUI mode) or fully functional hand-held physical lab tools (MR TUI mode). In the latter case, physical lab tools are strategically retrofitted with optical tracking markers to enable tactile, experimental, and analytical authenticity scientific experimentation. Preliminary user study data highlights the strengths and weaknesses of our generalized approach regarding student affective and cognitive student learning outcomes. 

   more » « less