An official website of the United States government
People with visual impairments (PVIs) are less likely to participate in physical activity than their sighted peers. One barrier is the lack of accessible group-based aerobic exercise classes, often due to instructors not giving accessible verbal instructions. While there is research in exercise tracking, these tools often require vision or familiarity with the exercise. There are accessible solutions that give personalized verbal feedback in slower-paced exercises, not generalizing to aerobics. In response, we have developed an algorithm that detects shoeprints on a sensor mat using computer vision and a CNN. We can infer whether a person is following along with a step aerobics workout and are designing reactive verbal feedback to guide the person to rejoin the class. Future work will include finishing development and conducting a user study to assess the effectiveness of the reactive verbal feedback.
more »
« less
Determining a Taxonomy of Accessible Phrases During Exercise Instruction for People with Visual Impairments for Text Analysis
Physical activity is an important part of quality life, however people with visual impairments (PVIs) are less likely to participate in physical activity than their sighted peers. One barrier is that exercise instructors may not give accessible verbal instructions. There is a potential for text analysis to determine these phrases, and in response provide more accessible instructions. First, a taxonomy of accessible phrases needs to be developed. To address this problem, we conducted user studies with 10 PVIs exercising along with audio and video aerobic workouts. We analyzed video footage of their exercise along with interviews to determine a preliminary set of phrases that are helpful or confusing. We then conducted an iterative qualitative analysis of six other exercise videos and sought expert feedback to derive our taxonomy. We hope these findings inform systems that analyze instructional phrases for accessibility to PVIs.
more »
« less
- Award ID(s):
- 1849822
- PAR ID:
- 10319223
- Date Published:
- Journal Name:
- Determining a Taxonomy of Accessible Phrases During Exercise Instruction for People with Visual Impairments for Text Analysis
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Augmented/Virtual reality and video-based media play a vital role in the digital learning revolution to train novices in spatial tasks. However, creating content for these different media requires expertise in several fields. We present EditAR, a unified authoring, and editing environment to create content for AR, VR, and video based on a single demonstration. EditAR captures the user’s interaction within an environment and creates a digital twin, enabling users without programming backgrounds to develop content. We conducted formative interviews with both subject and media experts to design the system. The prototype was developed and reviewed by experts. We also performed a user study comparing traditional video creation with 2D video creation from 3D recordings, via a 3D editor, which uses freehand interaction for in-headset editing. Users took 5 times less time to record instructions and preferred EditAR, along with giving significantly higher usability scores.more » « less
-
Regular exercise provides many mental and physical health benefits. However, when exercises are done incorrectly, it can lead to injuries. Because the COVID-19 pandemic made it challenging to exercise in communal spaces, the growth of virtual fitness programs was accelerated, putting people at risk of sustaining exercise-related injuries as they received little to no feedback on their exercising techniques. Colocated robots could be one potential enhancement to virtual training programs as they can cause higher learning gains, more compliance, and more enjoyment than non-co-located robots. In this study, we compare the effects of a physically present robot by having a person exercise either with a robot (robot condition) or a video of a robot displayed on a tablet (tablet condition). Participants (N=25) had an exercise system in their homes for two weeks. Participants who exercised with the colocated robot made fewer mistakes than those who exercised with the video-displayed robot. Furthermore, participants in the robot condition reported a higher fitness increase and more motivation to exercise than participants in the tablet condition.more » « less
-
Captions play a major role in making educational videos accessible to all and are known to benefit a wide range of learners. However, many educational videos either do not have captions or have inaccurate captions. Prior work has shown the benefits of using crowdsourcing to obtain accurate captions in a cost-efficient way, though there is a lack of understanding of how learners edit captions of educational videos either individually or collaboratively. In this work, we conducted a user study where 58 learners (in a course of 387 learners) participated in the editing of captions in 89 lecture videos that were generated by Automatic Speech Recognition (ASR) technologies. For each video, different learners conducted two rounds of editing. Based on editing logs, we created a taxonomy of errors in educational video captions (e.g., Discipline-Specific, General, Equations). From the interviews, we identified individual and collaborative error editing strategies. We then further demonstrated the feasibility of applying machine learning models to assist learners in editing. Our work provides practical implications for advancing video-based learning and for educational video caption editing.more » « less
-
People use videos to learn new recipes, exercises, and crafts. Such videos remain difficult for blind and low vision (BLV) people to follow as they rely on visual comparison. Our observations of visual rehabilitation therapists (VRTs) guiding BLV people to follow how-to videos revealed that VRTs provide both proactive and responsive support including detailed descriptions, non-visual workarounds, and progress feedback. We propose Vid2Coach, a system that transforms how-to videos into wearable camera-based assistants that provide accessible instructions and mixed-initiative feedback. From the video, Vid2Coach generates accessible instructions by augmenting narrated instructions with demonstration details and completion criteria for each step. It then uses retrieval-augmented-generation to extract relevant non-visual workarounds from BLV-specific resources. Vid2Coach then monitors user progress with a camera embedded in commercial smart glasses to provide context-aware instructions, proactive feedback, and answers to user questions. BLV participants (N=8) using Vid2Coach completed cooking tasks with 58.5\% fewer errors than when using their typical workflow and wanted to use Vid2Coach in their daily lives. Vid2Coach demonstrates an opportunity for AI visual assistance that strengthens rather than replaces non-visual expertise.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3441852.3476567
