Search for: All records

Abstract Objective.The RSVP Keyboard is a non-implantable, event-related potential-based brain-computer interface (BCI) system designed to support communication access for people with severe speech and physical impairments. Here we introduce inquiry preview (IP), a new RSVP Keyboard interface incorporating switch input for users with some voluntary motor function, and describe its effects on typing performance and other outcomes.Approach.Four individuals with disabilities participated in the collaborative design of possible switch input applications for the RSVP Keyboard, leading to the development of IP and a method of fusing switch input with language model and electroencephalography (EEG) evidence for typing. Twenty-four participants without disabilities and one potential end user with incomplete locked-in syndrome took part in two experiments investigating the effects of IP and two modes of switch input on typing accuracy and speed during a copy-spelling task.Main results.For participants without disabilities, IP and switch input tended to worsen typing performance compared to the standard RSVP Keyboard condition, with more consistent effects across participants for speed than for accuracy. However, there was considerable variability, with some participants demonstrating improved typing performance and better user experience (UX) with IP and switch input. Typing performance for the potential end user was comparable to that of participants without disabilities. He typed most quickly and accurately with IP and switch input and gave favorable UX ratings to those conditions, but preferred standard RSVP Keyboard.Significance.IP is a novel multimodal interface for the RSVP Keyboard BCI, incorporating switch input as an additional control signal. Typing performance and UX and preference varied widely across participants, reinforcing the need for flexible, customizable BCI systems that can adapt to individual users. ClinicalTrials.gov Identifier: NCT04468919. 

Autism is a neurodevelopmental disability that impacts one’s social communication and interaction. When left unsupported, this can increase the amount of loneliness felt by autistic people. Communication technology, such as AAC, can be helpful in supporting social communication, especially when co-designed with autistic people. We conducted a series of design workshops to co-design a new AAC system specifically supporting social communication. In this paper, we focus on the accessibility issues that were identified when running our workshops and provide recommendations on how to improve the process. We found that it is critical to build support for information processing time into the workshops, include a variety of AAC stakeholders, and create a shared vocabulary between the workshop participants to make design workshops more accessible to autistic adults. 

It is a right for people who use AAC to communicate with other people and engage with their communities. This scoping review explores how AAC is currently being used by people with communication disabilities in social and community contexts and the impact the design of AAC systems can have on this communication. A total of 13 studies were included that proposed new AAC system designs, conducted interviews with people who use AAC and their chosen communication partners, or performed an AAC intervention. Six themes emerged from these studies that indicate people who use AAC desire interaction with others, can benefit from greater agency in the communication process, can leverage the script-like nature of certain interactions to improve their communication competency, make use of online and asynchronous methods of communication, use multiple modes of communication and AAC content representation, and can benefit from instruction in social communication and community engagement. Suggestions for future research on how to incorporate each theme into the design of new AAC systems are also provided. 

Augmentative and alternative communication (AAC) is a field of research and practice that works with people who have a communication disability. One form AAC can take is a high-tech tool, such as a software-based communication system. Like all user interfaces, these systems must be designed and it is critical to include AAC users in the design process for their systems. A participatory design approach can include AAC users in the design process, but modifications may be necessary to make these methods more accessible. We present a two-part design process we are investigating for improving the participatory design for high-tech AAC systems. We discuss our plans to refine the accessibility of this process based on participant feedback. 

Communication Brain-Computer Interfaces (cBCIs) represent a crucial technological advancement for individuals with severe motor disabilities as they offer a direct pathway to express their thoughts and needs without physical movement. These systems commonly leverage the P300 ERP, a distinct neural response approximately 300-500ms after a novel stimulus. Language modeling presents a promising approach to enhancing the performance and usability of cBCIs. However, integrating language models with cBCI systems presents unique challenges, including balancing model complexity with real-time processing requirements and optimizing system performance parameters. This study utilizes simulations of online cBCI data to investigate the impact of different language models on typing rate and accuracy. 

Users of Augmentative and Alternative Communication (AAC) may write letter-by-letter via an interface that uses a character language model. However, most state-of-the-art large pretrained language models predict subword tokens of variable length. We investigate how to practically use such models to make accurate and efficient character predictions. Our algorithm for producing character predictions from a subword large language model (LLM) provides more accurate predictions than using a classification layer, a byte-level LLM, or an n-gram model. Additionally, we investigate a domain adaptation procedure based on a large dataset of sentences we curated based on scoring how useful each sentence might be for spoken or written AAC communication. We find our procedure further improves model performance on simple, conversational text. 

Text entry is a common and important part of many intelligent user interfaces. However, inferring a user’s intended text from their input can be challenging: motor actions can be imprecise, input sensors can be noisy, and situations or disabilities can hamper a user’s perception of interface feedback. Numerous prior studies have explored input on touchscreen phones, smartwatches, in midair, and on desktop keyboards. Based on these prior studies, we are releasing a large and diverse data set of noisy typing input consisting of thousands of sentences written by hundreds of users on QWERTY-layout keyboards. This paper describes the various subsets contained in this new research dataset as well as the data format. 

Some individuals with motor impairments communicate using a single switch - such as a button click, air puff, or blink. Row-column scanning provides a method for choosing items arranged in a grid using a single switch. An alternative, Nomon, allows potential selections to be arranged arbitrarily rather than requiring a grid (as desired for gaming, drawing, etc.) - and provides an alternative probabilistic selection method. While past results suggest that Nomon may be faster and easier to use than row-column scanning, no work has yet quantified performance of the two methods over longer time periods or in tasks beyond writing. In this paper, we also develop and validate a webcam-based switch that allows a user without a motor impairment to approximate the response times of a motor-impaired single switch user; although the approximation is not a replacement for testing with single-switch users, it allows us to better initialize, calibrate, and evaluate our method. Over 10 sessions with the webcam switch, we found users typed faster and more easily with Nomon than with row-column scanning. The benefits of Nomon were even more pronounced in a picture-selection task. Evaluation and feedback from a motor-impaired switch user further supports the promise of Nomon.