More Like this

1. Poster: BystandAR: Protecting Bystander Visual Data in Augmented Reality Systems

   https://doi.org/10.1145/3581791.3597377  

   Corbett, Matthew; David-John, Brendan; Shang, Jiacheng; Hu, Y. Charlie; Ji, Bo (June 2023, the 21st Annual International Conference on Mobile Systems, Applications and Services)

   Augmented Reality (AR) devices are set apart from other mobile devices by the immersive experience they offer. While the powerful suite of sensors on modern AR devices is necessary for enabling such an immersive experience, they can create unease in bystanders (i.e., those surrounding the device during its use) due to potential bystander data leaks, which is called the bystander privacy problem. In this poster, we propose BystandAR, the first practical system that can effectively protect bystander visual (camera and depth) data in real-time with only on-device processing. BystandAR builds on a key insight that the device user's eye gaze and voice are highly effective indicators for subject/bystander detection in interpersonal interaction, and leverages novel AR capabilities such as eye gaze tracking, wearer-focused microphone, and spatial awareness to achieve a usable frame rate without offloading sensitive information. Through a 16-participant user study, we show that BystandAR correctly identifies and protects 98.14% of bystanders while allowing access to 96.27% of subjects. We accomplish this with average frame rates of 52.6 frames per second without the need to offload unprotected bystander data to another device. 

   more » « less
2. A User Study of a Wearable System to Enhance Bystanders’ Facial Privacy

   https://doi.org/10.3390/iot1020013  

   Perez, Alfredo J.; Zeadally, Sherali; Griffith, Scott; Garcia, Luis Y.; Mouloud, Jaouad A. (December 2020, IoT)

   null (Ed.)

   The privacy of users and information are becoming increasingly important with the growth and pervasive use of mobile devices such as wearables, mobile phones, drones, and Internet of Things (IoT) devices. Today many of these mobile devices are equipped with cameras which enable users to take pictures and record videos anytime they need to do so. In many such cases, bystanders’ privacy is not a concern, and as a result, audio and video of bystanders are often captured without their consent. We present results from a user study in which 21 participants were asked to use a wearable system called FacePET developed to enhance bystanders’ facial privacy by providing a way for bystanders to protect their own privacy rather than relying on external systems for protection. While past works in the literature focused on privacy perceptions of bystanders when photographed in public/shared spaces, there has not been research with a focus on user perceptions of bystander-based wearable devices to enhance privacy. Thus, in this work, we focus on user perceptions of the FacePET device and/or similar wearables to enhance bystanders’ facial privacy. In our study, we found that 16 participants would use FacePET or similar devices to enhance their facial privacy, and 17 participants agreed that if smart glasses had features to conceal users’ identities, it would allow them to become more popular. 

   more » « less
3. GazeXR: A generally eye-tracking system enabling invariable gaze data in virtual environment

   https://doi.org/10.1007/978-3-030-77599-5_4  

   Lenart, C. (January 2021, Virtual, augmented and mixed reality)

   J. Y. C., Chen (Ed.)

   Controlling and standardizing experiments is imperative for quantitative research methods. With the increase in the availability and quantity of low-cost eye-tracking devices, gaze data are considered as an important user input for quantitative analysis in many social science research areas, especially incorporating with virtual reality (VR) and augmented reality (AR) technologies. This poses new challenges in providing a default interface for gaze data in a common method. This paper propose GazeXR, which focuses on designing a general eye-tracking system interfacing two eye-tracking devices and creating a hardware independent virtual environment. We apply GazeXR to the in-class teaching experience analysis use case using external eye-tracking hardware to collect the gaze data for the gaze track analysis. 

   more » « less
4. GazePair: Efficient Pairing of Augmented Reality Devices Using Gaze Tracking

   https://doi.org/10.1109/TMC.2023.3255841  

   Corbett, Matthew; Shang, Jiacheng; Ji, Bo (March 2023, IEEE Transactions on Mobile Computing)

   As Augmented Reality (AR) devices become more prevalent and commercially viable, the need for quick, efficient, and secure schemes for pairing these devices has become more pressing. Current methods to securely exchange holograms require users to send this information through large data centers, creating security and privacy concerns. Existing techniques to pair these devices on a local network and share information fall short in terms of usability and scalability. These techniques either require hardware not available on AR devices, intricate physical gestures, removal of the device from the head, do not scale to multiple pairing partners, or rely on methods with low entropy to create encryption keys. To that end, we propose a novel pairing system, called GazePair, that improves on all existing local pairing techniques by creating an efficient, effective, and intuitive pairing protocol. GazePair uses eye gaze tracking and a spoken key sequence cue (KSC) to generate identical, independently generated symmetric encryption keys with 64 bits of entropy. GazePair also achieves improvements in pairing success rates and times over current methods. Additionally, we show that GazePair can extend to multiple users. Finally, we assert that GazePair can be used on any Mixed Reality (MR) device equipped with eye gaze tracking. 

   more » « less
5. Color-Perception-Guided Display Power Reduction for Virtual Reality

   https://doi.org/10.1145/3550454.3555473  

   Duinkharjav, Budmonde; Chen, Kenneth; Tyagi, Abhishek; He, Jiayi; Zhu, Yuhao; Sun, Qi (December 2022, ACM Transactions on Graphics)

   Battery life is an increasingly urgent challenge for today's untethered VR and AR devices. However, the power efficiency of head-mounted displays is naturally at odds with growing computational requirements driven by better resolution, refresh rate, and dynamic ranges, all of which reduce the sustained usage time of untethered AR/VR devices. For instance, the Oculus Quest 2, under a fully-charged battery, can sustain only 2 to 3 hours of operation time. Prior display power reduction techniques mostly target smartphone displays. Directly applying smartphone display power reduction techniques, however, degrades the visual perception in AR/VR with noticeable artifacts. For instance, the "power-saving mode" on smartphones uniformly lowers the pixel luminance across the display and, as a result, presents an overall darkened visual perception to users if directly applied to VR content. Our key insight is that VR display power reduction must be cognizant of the gaze-contingent nature of high field-of-view VR displays. To that end, we present a gaze-contingent system that, without degrading luminance, minimizes the display power consumption while preserving high visual fidelity when users actively view immersive video sequences. This is enabled by constructing 1) a gaze-contingent color discrimination model through psychophysical studies, and 2) a display power model (with respect to pixel color) through real-device measurements. Critically, due to the careful design decisions made in constructing the two models, our algorithm is cast as a constrained optimization problem with a closed-form solution, which can be implemented as a real-time, image-space shader. We evaluate our system using a series of psychophysical studies and large-scale analyses on natural images. Experiment results show that our system reduces the display power by as much as 24% (14% on average) with little to no perceptual fidelity degradation. 

   more » « less