More Like this

1. Ani-Bot: A Mixed-Reality Modular Robotics System

   https://doi.org/10.1145/3131785.3131833  

   Cao, Yuanzhi; Xu, Zhuangying; Glenn, Terrell; Huo, Ke; Ramani, Karthik (January 2017, ACM Symposium on User Interface Software & Technology (UIST))

   We present Ani-Bot, a modular robotics system that allows users to construct Do-It-Yourself (DIY) robots and use mixed-reality approach to interact with them. Ani-Bot enables novel user experience by embedding Mixed-Reality Interaction (MRI) in the three phases of interacting with a modular construction kit, namely, Creation, Tweaking, and Usage. In this paper, we first present the system design that allows users to instantly perform MRI once they finish assembling the robot. Further, we discuss the augmentations offered by MRI in the three phases in specific. 

   more » « less
2. Mixed Reality Technology: A Virtual Training Tool in Fluid Power Engineering

   https://doi.org/10.1115/FPMC2023-111715  

   Azzam, Israa; Pate, Keith; Breidi, Farid (October 2023, American Society of Mechanical Engineers)

   Abstract Fluid power systems can be expensive and difficult to access, making it challenging to provide hands-on training. This work discusses the incorporation of Mixed Reality (MR) technology in Fluid Power applications for providing a virtual training environment that simulates the behavior of fluid power systems, allowing users to receive immediate feedback on the system’s performance. Mixed reality is a digitized-based technology that integrates a virtual environment with our real world by utilizing real-world sensor data and computer models. This technology allows running simulations that examine the complexity of highly-coupled systems, producing new digital environments where physical and digital elements can interact in real-time. With all these features, MR technology can be a practical training tool for running virtual simulations that mimic real-life industry settings. It can extend the user with a virtual training environment, thus preparing the next generation of fluid power engineers and specialists. Throughout this work, we present the development and capabilities of a digitized virtual copy of a hydraulic excavator’s arm in an MR environment as a proof of concept. The MR arm module is developed and deployed using Microsoft’s Mixed Reality Tool Kit (MRTK) for Unity through HoloLens 2 MR headset. The MR development involves generating virtual copies of the mechanical and hydraulic subsystems, conducting the virtual assembly, and creating a user interface in the MR environment to visualize and interact with the model. The developed MR module enables visualizing the excavator’s internal structure, conducting the virtual assembly, and running virtual simulations, all of which assist in training future fluid power operators. It is an effective training tool that helps train junior engineers/technicians, cutting down on cost and time. 

   more » « less
3. Understanding Multi-user, Handheld Mixed Reality for Group-based MR Games

   https://doi.org/10.1145/3653688  

   Bautista_Isaza, Carlos Augusto; Enriquez, Daniel; Moon, Hayoun; Jeon, Myounghoon; Lee, Sang Won (April 2024, Proceedings of the ACM on Human-Computer Interaction)

   Research has identified applications of handheld-based VR, which utilizes handheld displays or mobile devices, for developing systems that involve users in mixed reality (MR) without the need for head-worn displays (HWDs). Such systems can potentially accommodate large groups of users participating in MR. However, we lack an understanding of how group sizes and interaction methods affect the user experience. In this paper, we aim to advance our understanding of handheld-based MR in the context of multiplayer, co-located games. We conducted a study (N = 38) to understand how user experiences vary by group size (2, 4, and 8) and interaction method (proximity-based or pointing-based). For our experiment, we implemented a multiuser experience for up to ten users. We found that proximity-based interaction that encouraged dynamic movement positively affected social presence and physical/temporal workload. In bigger group settings, participants felt less challenged and less positive. Individuals had varying preferences for group size and interaction type. The findings of the study will advance our understanding of the design space for handheld-based MR in terms of group sizes and interaction schemes. To make our contributions explicit, we conclude our paper with design implications that can inform user experience design in handheld-based mixed reality contexts. 

   more » « less
4. Decoupled Hands: An Approach for Aligning Perspectives in Collaborative Mixed Reality

   https://doi.org/10.1145/3706599.3720219  

   Gottsacker, Matt; Numan, Nels; Steed, Anthony; Bruder, Gerd; Welch, Gregory F; Feiner, Steven (April 2025, ACM)

   When collaborating relative to a shared 3D virtual object in mixed reality (MR), users may experience communication issues arising from differences in perspective. These issues include occlusion (e.g., one user not being able to see what the other is referring to) and inefficient spatial references (e.g., “to the left of this” may be confusing when users are positioned opposite to each other). This paper presents a novel technique for automatic perspective alignment in collaborative MR involving co-located interaction centered around a shared virtual object. To align one user’s perspective on the object with a collaborator’s, a local copy of the object and any other virtual elements that reference it (e.g., the collaborator’s hands) are dynamically transformed. The technique does not require virtual travel and preserves face-to-face interaction. We created a prototype application to demonstrate our technique and present an evaluation methodology for related MR collaboration and perspective alignment scenarios. 

   more » « less
5. Robot-generated Mixed Reality Gestures Improve Human-Robot Interaction

   https://doi.org/10.1007/978-3-030-90525-5_69  

   Tran, Nhan; Grant, Trevor; Phung, Thao; Hirshfield, Leanne; Wickens, Christopher; Williams, Tom (January 2022, International Conference on Social Robotics (Brief Research Reports))

   We investigate the effectiveness of robot-generated mixed reality gestures. Our findings demonstrate how these gestures increase user effectiveness by decreasing user response time, and that robots can pair long referring expressions with mixed reality gestures without cognitively overloading users. 

   more » « less