An official website of the United States government
Accuracy and speed are pivotal when it comes to typing. Mixed reality headsets offer users the groundbreaking ability to project virtual objects into the physical world. However, when typing on a virtual keyboard in mixed reality space, users lose the tactile feedback that comes with a physical keyboard, making typing much more difficult. Our goal was to explore the capability of users to type using all ten fingers on a virtual key in mixed reality. We measured user performance when typing with index fingers versus all ten fingers. We also examined the usage of eye-tracking to disable all keys the user wasn’t looking at, and the effect it had on improving speed and accuracy. Our findings so far indicate that, while eyetracking seems to help accuracy, it is not enough to bring 10 finger typing up to the same level of performance as index finger typing.
more »
« less
This content will become publicly available on February 1, 2026
RSVP keyboard with inquiry preview: mixed performance and user experience with an adaptive, multimodal typing interface combining EEG and switch input
Abstract Objective.The RSVP Keyboard is a non-implantable, event-related potential-based brain-computer interface (BCI) system designed to support communication access for people with severe speech and physical impairments. Here we introduce inquiry preview (IP), a new RSVP Keyboard interface incorporating switch input for users with some voluntary motor function, and describe its effects on typing performance and other outcomes.Approach.Four individuals with disabilities participated in the collaborative design of possible switch input applications for the RSVP Keyboard, leading to the development of IP and a method of fusing switch input with language model and electroencephalography (EEG) evidence for typing. Twenty-four participants without disabilities and one potential end user with incomplete locked-in syndrome took part in two experiments investigating the effects of IP and two modes of switch input on typing accuracy and speed during a copy-spelling task.Main results.For participants without disabilities, IP and switch input tended to worsen typing performance compared to the standard RSVP Keyboard condition, with more consistent effects across participants for speed than for accuracy. However, there was considerable variability, with some participants demonstrating improved typing performance and better user experience (UX) with IP and switch input. Typing performance for the potential end user was comparable to that of participants without disabilities. He typed most quickly and accurately with IP and switch input and gave favorable UX ratings to those conditions, but preferred standard RSVP Keyboard.Significance.IP is a novel multimodal interface for the RSVP Keyboard BCI, incorporating switch input as an additional control signal. Typing performance and UX and preference varied widely across participants, reinforcing the need for flexible, customizable BCI systems that can adapt to individual users. ClinicalTrials.gov Identifier: NCT04468919.
more »
« less
- Award ID(s):
- 1750193
- PAR ID:
- 10628764
- Publisher / Repository:
- IOP Publishing Ltd.
- Date Published:
- Journal Name:
- Journal of Neural Engineering
- Volume:
- 22
- Issue:
- 1
- ISSN:
- 1741-2560
- Page Range / eLocation ID:
- 016022
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Bottoni, Paolo; Panizzi, Emanuele (Ed.)Many questions regarding single-hand text entry on modern smartphones (in particular, large-screen smartphones) remain under-explored, such as, (i) will the existing prevailing single-handed keyboards fit for large-screen smartphone users? and (ii) will individual customization improve single-handed keyboard performance? In this paper we study single-handed typing behaviors on several representative keyboards on large-screen mobile devices.We found that, (i) the user-adaptable-shape curved keyboard performs best among all the studied keyboards; (ii) users’ familiarity with the Qwerty layout plays a significant role at the beginning, but after several sessions of training, the user-adaptable curved keyboard can have the best learning curve and performs best; (iii) generally the statistical decoding algorithms via spatial and language models can well handle the input noise from single-handed typing.more » « less
-
null (Ed.)We investigate typing on a QWERTY keyboard rendered in virtual reality. Our system tracks users’ hands in the virtual environment via a Leap Motion mounted on the front of a head mounted display. This allows typing on an auto-correcting midair keyboard without the need for auxiliary input devices such as gloves or handheld controllers. It supports input via the index fingers of one or both hands. We compare two keyboard designs: a normal QWERTY layout and a split layout. We found users typed at around 16 words-per-minute using one or both index fingers on the normal layout, and about 15 words-per-minute using both index fingers on the split layout. Users had a corrected error rate below 2% in all cases. To explore midair typing with limited or no visual feedback, we had users type on an invisible keyboard. Users typed on this keyboard at 11 words-per-minute at an error rate of 3.3% despite the keyboard providing almost no visual feedback.more » « less
-
Typing on a midair keyboard in mixed reality can be difficult due to the lack of tactile feedback when virtual keys are tapped. Locating the keyboard over a real-world surface offers a potential way to mitigate this issue. We measured user performance and preference when a virtual keyboard was located on a table, on a wall, or in midair. Despite the additional tactile feedback offered by the table and wall locations, we found the midair location had a significantly higher entry rate with a similar error rate compared to the other locations. Participants also preferred the midair location over the other locations.more » « less
-
Voice-enabled technologies such as VoiceOver (screen reader) and the Seeing AI app (image recognition) have revolutionized daily tasks for people with visual disabilities, fostering greater independence and information access. However, a gap remains in understanding the user experience (UX) of these technologies. This study investigated how those with visual disabilities interacted with VoiceOver and the Seeing AI app. A convenience sample of eight participants with visual disabilities engaged in direct observations while using these technologies. The study utilized the System Usability Scale (SUS) to assess perceived usability and analyzed findings using descriptive statistics. Results indicated a poorer UX with VoiceOver compared to the Seeing AI app, with challenges identified in graphical user interfaces (GUIs), voice and gesture commands. Relevant recommendations were made to enhance usability. The study emphasizes the need for more intuitive GUIs and optimized voice/gesture interactions for users with visual disabilities.more » « less
