Search for: All records

Accuracy and speed are pivotal when it comes to typing. Mixed reality headsets offer users the groundbreaking ability to project virtual objects into the physical world. However, when typing on a virtual keyboard in mixed reality space, users lose the tactile feedback that comes with a physical keyboard, making typing much more difficult. Our goal was to explore the capability of users to type using all ten fingers on a virtual key in mixed reality. We measured user performance when typing with index fingers versus all ten fingers. We also examined the usage of eye-tracking to disable all keys the user wasn’t looking at, and the effect it had on improving speed and accuracy. Our findings so far indicate that, while eyetracking seems to help accuracy, it is not enough to bring 10 finger typing up to the same level of performance as index finger typing. 

Text entry is a common and important part of many intelligent user interfaces. However, inferring a user’s intended text from their input can be challenging: motor actions can be imprecise, input sensors can be noisy, and situations or disabilities can hamper a user’s perception of interface feedback. Numerous prior studies have explored input on touchscreen phones, smartwatches, in midair, and on desktop keyboards. Based on these prior studies, we are releasing a large and diverse data set of noisy typing input consisting of thousands of sentences written by hundreds of users on QWERTY-layout keyboards. This paper describes the various subsets contained in this new research dataset as well as the data format. 

In many situations, it may be impractical or impossible to enter text by selecting precise locations on a physical or touchscreen keyboard. We present an ambiguous keyboard with four character groups that has potential applications for eyes-free text entry, as well as text entry using a single switch or a brain-computer interface.We develop a procedure for optimizing these character groupings based on a disambiguation algorithm that leverages a long-span language model. We produce both alphabetically-constrained and unconstrained character groups in an offline optimization experiment and compare them in a longitudinal user study. Our results did not show a significant difference between the constrained and unconstrained character groups after four hours of practice. As expected, participants had significantly more errors with the unconstrained groups in the first session, suggesting a higher barrier to learning the technique.We therefore recommend the alphabetically-constrained character groups, where participants were able to achieve an average entry rate of 12.0 words per minute with a 2.03% character error rate using a single hand and with no visual feedback. 

Some individuals with motor impairments communicate using a single switch - such as a button click, air puff, or blink. Our software, Nomon, provides a method for single-switch users to select between items on a screen. Nomon’s flexibility stems from its probabilistic selection method, which allows potential options to be arranged arbitrarily rather than requiring they be arranged in a grid. As a result, Nomon can be used for a host of applications - including gaming, drawing, and web browsing. Focusing on accessibility, we updated the Nomon interface in collaboration with a switch user and with experts in Augmentative and Alternative Communication (AAC). We present our updated Nomon interface as an open-source web application. 

Some individuals with motor impairments communicate using a single switch - such as a button click, air puff, or blink. Row-column scanning provides a method for choosing items arranged in a grid using a single switch. An alternative, Nomon, allows potential selections to be arranged arbitrarily rather than requiring a grid (as desired for gaming, drawing, etc.) - and provides an alternative probabilistic selection method. While past results suggest that Nomon may be faster and easier to use than row-column scanning, no work has yet quantified performance of the two methods over longer time periods or in tasks beyond writing. In this paper, we also develop and validate a webcam-based switch that allows a user without a motor impairment to approximate the response times of a motor-impaired single switch user; although the approximation is not a replacement for testing with single-switch users, it allows us to better initialize, calibrate, and evaluate our method. Over 10 sessions with the webcam switch, we found users typed faster and more easily with Nomon than with row-column scanning. The benefits of Nomon were even more pronounced in a picture-selection task. Evaluation and feedback from a motor-impaired switch user further supports the promise of Nomon. 

Typing on a midair keyboard in mixed reality can be difficult due to the lack of tactile feedback when virtual keys are tapped. Locating the keyboard over a real-world surface offers a potential way to mitigate this issue. We measured user performance and preference when a virtual keyboard was located on a table, on a wall, or in midair. Despite the additional tactile feedback offered by the table and wall locations, we found the midair location had a significantly higher entry rate with a similar error rate compared to the other locations. Participants also preferred the midair location over the other locations.