More Like this

1. Trade-offs in Augmented Reality User Interfaces for Controlling a Smart Environment

   https://doi.org/10.1145/3485279.3485288  

   Flick, Connor Daniel; Harris, Courtney J; Yonkers, Nikolas T; Norouzi, Nahal; Erickson, Austin; Choudhary, Zubin; Gottsacker, Matt; Bruder, Gerd; Welch, Greg (November 2021, Proc. of the ACM Symposium on Spatial User Interaction (SUI ’21))

   Smart devices and Internet of Things (IoT) technologies are replacing or being incorporated into traditional devices at a growing pace. The use of digital interfaces to interact with these devices has become a common occurrence in homes, work spaces, and various industries around the world. The most common interfaces for these connected devices focus on mobile apps or voice control via intelligent virtual assistants. However, with augmented reality (AR) becoming more popular and accessible among consumers, there are new opportunities for spatial user interfaces to seamlessly bridge the gap between digital and physical affordances. In this paper, we present a human-subject study evaluating and comparing four user interfaces for smart connected environments: gaze input, hand gestures, voice input, and a mobile app. We assessed participants’ user experience, usability, task load, completion time, and preferences. Our results show multiple trade-offs between these interfaces across these measures. In particular, we found that gaze input shows great potential for future use cases, while both gaze input and hand gestures suffer from limited familiarity among users, compared to voice input and mobile apps. 

   more » « less
2. Use of Intelligent Voice Assistants by Older Adults with Low Technology Use

   Pradhan, Alisha; Lazar, A; Findlater, Leah (April 2020, ACM transactions on computerhuman interaction)

   Voice assistants embodied in smart speakers (e.g., Amazon Echo, Google Home) enable conversational interaction that does not necessarily rely on expertise with mobile or desktop computing. Hence, these voice assistants offer new opportunities to different populations, including individuals who are not interested or able to use traditional computing devices such as computers and smartphones. To understand how older adults who use technology infrequently perceive and use these voice assistants, we conducted a three-week field deployment of the Amazon Echo Dot in the homes of seven older adults. Participants described increased confidence using digital technology and found the conversational voice interfaces easy to use. While some types of usage dropped over the three-week period (e.g., playing music), we observed consistent usage for finding online information. Given that much of this information was health-related, this finding emphasizes the need to revisit concerns about credibility of information with this new interaction medium. Although features to support memory (e.g., setting timers, reminders) were initially perceived as useful, the actual usage was unexpectedly low due to reliability concerns. We discuss how these findings apply to other user groups along with design implications and recommendations for future work on voice user interfaces. 

   more » « less
3. “I Can See Your Password”: A Case Study About Cybersecurity Risks in Mid-Air Interactions of Mixed Reality-Based Smart Manufacturing Applications

   https://doi.org/10.1115/1.4062658  

   Yang, Wenhao; Dengxiong, Xiwen; Wang, Xueting; Hu, Yidan; Zhang, Yunbo (March 2024, Journal of Computing and Information Science in Engineering)

   Abstract This paper aims to present a potential cybersecurity risk existing in mixed reality (MR)-based smart manufacturing applications that decipher digital passwords through a single RGB camera to capture the user’s mid-air gestures. We first created a test bed, which is an MR-based smart factory management system consisting of mid-air gesture-based user interfaces (UIs) on a video see-through MR head-mounted display. To interact with UIs and input information, the user’s hand movements and gestures are tracked by the MR system. We setup the experiment to be the estimation of the password input by users through mid-air hand gestures on a virtual numeric keypad. To achieve this goal, we developed a lightweight machine learning-based hand position tracking and gesture recognition method. This method takes either video streaming or recorded video clips (taken by a single RGB camera in front of the user) as input, where the videos record the users’ hand movements and gestures but not the virtual UIs. With the assumption of the known size, position, and layout of the keypad, the machine learning method estimates the password through hand gesture recognition and finger position detection. The evaluation result indicates the effectiveness of the proposed method, with a high accuracy of 97.03%, 94.06%, and 83.83% for 2-digit, 4-digit, and 6-digit passwords, respectively, using real-time video streaming as input with known length condition. Under the unknown length condition, the proposed method reaches 85.50%, 76.15%, and 77.89% accuracy for 2-digit, 4-digit, and 6-digit passwords, respectively. 

   more » « less
4. An Overview of Smart Wheelchair Technologies

   Kovacevic, Nikol; Ahmad, Ishfaq; Clark, Addison; Saifullah, Khalid (June 2025, Springer Nature)

   Arabnia, Hamid; Deligiannidis, Leonidas; Tinetti, Fernando; Tran, Quoc-Nam (Ed.)

   Many individuals who are in need of mobility assistance do not have access to the proper wheelchair for their type of mobility disability. There is growing research towards creating smart wheelchairs using a variety of methods, such as biopotential signals or eye tracking for input and LiDAR, ultrasonic sensors, or using a camera to create a map or track position. There have been other methods, such as voice control, sip and puff, and hand gestures, but there are disadvantages of these that can limit their usefulness. Smart wheelchairs should account for collisions, but also emphasize the safety and comfort of the user. In this paper, we review and classify state-of-the-art research in smart wheelchairs. Many machine learning models are used for various parts of wheelchairs, from mapping and signal processing to input classification. Smart wheelchairs rely on various hardware devices, such as eye trackers, electrode caps, EMG armbands, RPLidar, RGB-cameras, and ultrasonic sensors. Some hybrid models use a combination of methods to account for some of their limitations. Some research has leaned towards training games to help teach users. Future work should include improvement of classification methods for various input signals and improvement on the accessibility of the technology. 

   more » « less
5. A Continuous Articulatory Gesture Based Liveness Detection for Voice Authentication on Smart Devices

   https://doi.org/10.1109/JIOT.2022.3199995  

   Zhang, Linghan; Tan, Sheng; Chen, Yingying; Yang, Jie (August 2022, IEEE Internet of Things Journal)

   Voice biometrics is drawing increasing attention to user authentication on smart devices. However, voice biometrics is vulnerable to replay attacks, where adversaries try to spoof voice authentication systems using pre-recorded voice samples collected from genuine users. To this end, we propose VoiceGesture, a liveness detection solution for voice authentication on smart devices such as smartphones and smart speakers. With audio hardware advances on smart devices, VoiceGesture leverages built-in speaker and microphone pairs on smart devices as Doppler Radar to sense articulatory gestures for liveness detection during voice authentication. The experiments with 21 participants and different smart devices show that VoiceGesture achieves over 99% and around 98% detection accuracy for text-dependent and text-independent liveness detection, respectively. Moreover, VoiceGesture is robust to different device placements, low audio sampling frequency, and supports medium range liveness detection on smart speakers in various use scenarios, including smart homes and smart vehicles. 

   more » « less