More Like this

1. Integrated Design of Augmented Reality Spaces Using Virtual Environments

   https://doi.org/10.1109/ISMAR55827.2022.00045  

   Scargill, Tim; Chen, Ying; Marzen, Nathan; Gorlatova, Maria (October 2022, 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR))

   Demand is growing for markerless augmented reality (AR) experiences, but designers of the real-world spaces that host them still have to rely on inexact, qualitative guidelines on the visual environment to try and facilitate accurate pose tracking. Furthermore, the need for visual texture to support markerless AR is often at odds with human aesthetic preferences, and understanding how to balance these competing requirements is challenging due to the siloed nature of the relevant research areas. To address this, we present an integrated design methodology for AR spaces, that incorporates both tracking and human factors into the design process. On the tracking side, we develop the first VI-SLAM evaluation technique that combines the flexibility and control of virtual environments with real inertial data. We use it to perform systematic, quantitative experiments on the effect of visual texture on pose estimation accuracy; through 2000 trials in 20 environments, we reveal the impact of both texture complexity and edge strength. On the human side, we show how virtual reality (VR) can be used to evaluate user satisfaction with environments, and highlight how this can be tailored to AR research and use cases. Finally, we demonstrate our integrated design methodology with a case study on AR museum design, in which we conduct both VI-SLAM evaluations and a VR-based user study of four different museum environments. 

   more » « less
2. Demo: UI Based Attacks in WebXR

   https://doi.org/10.1145/3711875.3734381  

   Mukherjee, Chandrika; Aburas, Reham Mohamed; Arunasalam, Arjun; Farrukh, Habiba; Celik, Z Berkay (June 2025, ACM)

   The WebXR API enables immersive AR/VR experiences directly through web browsers on head-mounted displays (HMDs). However, prior research shows that security-sensitive UI properties and the lack of an like element that separates different origins can be exploited to manipulate user actions, particularly within the advertising ecosystem. In our prior work, we proposed five novel UI-based attacks in WebXR, targeting the ad ecosystem. This demo presents these attacks in a unified gaming application, embedding each into distinct interactive scenarios. Our work highlights the need to address design challenges and requirements for improving immersive web-based experiences. We provide our demo video at: https://youtu.be/lTBQbxnNq34. 

   more » « less
3. Force-Aware Interface via Electromyography for Natural VR/AR Interaction

   https://doi.org/10.1145/3550454.3555461  

   Zhang, Yunxiang; Liang, Benjamin; Chen, Boyuan; Torrens, Paul M.; Atashzar, S. Farokh; Lin, Dahua; Sun, Qi (December 2022, ACM Transactions on Graphics)

   While tremendous advances in visual and auditory realism have been made for virtual and augmented reality (VR/AR), introducing a plausible sense of physicality into the virtual world remains challenging. Closing the gap between real-world physicality and immersive virtual experience requires a closed interaction loop: applying user-exerted physical forces to the virtual environment and generating haptic sensations back to the users. However, existing VR/AR solutions either completely ignore the force inputs from the users or rely on obtrusive sensing devices that compromise user experience. By identifying users' muscle activation patterns while engaging in VR/AR, we design a learning-based neural interface for natural and intuitive force inputs. Specifically, we show that lightweight electromyography sensors, resting non-invasively on users' forearm skin, inform and establish a robust understanding of their complex hand activities. Fuelled by a neural-network-based model, our interface can decode finger-wise forces in real-time with 3.3% mean error, and generalize to new users with little calibration. Through an interactive psychophysical study, we show that human perception of virtual objects' physical properties, such as stiffness, can be significantly enhanced by our interface. We further demonstrate that our interface enables ubiquitous control via finger tapping. Ultimately, we envision our findings to push forward research towards more realistic physicality in future VR/AR. 

   more » « less
4. Using Narrative to Engage Girls in Museum Engineering Design Tasks

   Bennett, D.; Letourneau, S.; Liu, C.C.J. (July 2018, Visitors Studies Association: Fostering Transparency, Fostering Public Trust)

   In this design-based research study, the New York Hall of Science, in collaboration with the Amazeum (Bentonville, AR), the Tech Museum (San Jose, CA), and the Creativity Labs (Indiana University), are examining how narratives can shape girls’ engagement in museum-based engineering activities. The project involves developing and testing six engineering activities that incorporate elements of narrative.Through iterative activity development, the project team is exploring how design prompts and materials can suggest narratives via characters, settings, or scenarios in a way that invites visitors’ perspective-taking and empathy. In addition, by comparing narrative and non-narrative versions of each activity, the team is gathering evidence about whether and how narratives support girls’ participation in engineering activities, as well as their persistence in creating and iterating solutions. This poster shares the beginning theoretical framework for defining narrative and empathy in museum-based engineering activities, as well as the results from three activities developed and tested by summer 2018. 

   more » « less
5. Measuring Millimeter Wave based Link Bandwidth Fluctuations during Indoor Immersive Experience

   https://doi.org/10.1109/LNET.2022.3157324  

   Wu, Zongshen; Huang, Chin-Ya; Ramanathan, Parameswaran (March 2022, IEEE Networking Letters)

   This paper presents the results of motion-tracking synchronized millimeter wave (mmWave) link bandwidth fluctuations while a user is engaged in immersive augmented/virtual reality applications. Our system, called MITRAS, supports ex- tensive exploration of human-induced impacts on mmWave link bandwidth during immersive experience. MITRAS adopts the packet train measurement application to track link bandwidth fluctuations. Meanwhile, the user movements are tracked using an Oculus Quest 2 headset. Through investigating the impacts of human movements on link bandwidth fluctuations, we further propose a link state prediction model to shed light on higher layer protocol design for immersive applications over mmWave links. 

   more » « less