An official website of the United States government
Conversational systems rely heavily on speech recognition to interpret and respond to user commands and queries. Despite progress on speech recognition accuracy, errors may still sometimes occur and can significantly affect the end-user utility of such systems. While visual feedback can help detect errors, it may not always be practical, especially for people who are blind or low-vision. In this study, we investigate ways to improve error detection by manipulating the audio output of the transcribed text based on the recognizer's confidence level in its result. Our findings show that selectively slowing down the audio when the recognizer exhibited uncertainty led to a 12% relative increase in participants' ability to detect errors compared to uniformly slowing the audio. It also reduced the time it took participants to listen to the recognition result and decide if there was an error by 11%.
more »
« less
Kinesthetic Feedback for Understanding Program Execution
To better prepare future generations, knowledge about computers and programming are one of the many skills that are part of almost all Science, Technology, Engineering, and Mathematic programs; however, teaching and learning programming is a complex task that is generally considered difficult by students and teachers alike. One approach to engage and inspire students from a variety of backgrounds is the use of educational robots. Unfortunately, previous research presents mixed results on the effectiveness of educational robots on student learning. One possibility for this lack of clarity may be because students have a wide variety of styles of learning. It is possible that the use of kinesthetic feedback, in addition to the normally used visual feedback, may improve learning with educational robots by providing a richer, multi-modal experience that may appeal to a larger number of students with different learning styles. It is also possible, however, that the addition of kinesthetic feedback, and how it may interfere with the visual feedback, may decrease a student’s ability to interpret the program commands being executed by a robot, which is critical for program debugging. In this work, we investigated whether human participants were able to accurately determine a sequence of program commands performed by a robot when both kinesthetic and visual feedback were being used together. Command recall and end point location determination were compared to the typically used visual-only method, as well as a narrative description. Results from 10 sighted participants indicated that individuals were able to accurately determine a sequence of movement commands and their magnitude when using combined kinesthetic + visual feedback. Participants’ recall accuracy of program commands was actually better with kinesthetic + visual feedback than just visual feedback. Although the recall accuracy was even better with the narrative description, this was primarily due to participants confusing an absolute rotation command with a relative rotation command with the kinesthetic + visual feedback. Participants’ zone location accuracy of the end point after a command was executed was significantly better for both the kinesthetic + visual feedback and narrative methods compared to the visual-only method. Together, these results suggest that the use of both kinesthetic + visual feedback improves an individual’s ability to interpret program commands, rather than decreases it.
more »
« less
- Award ID(s):
- 1742242
- PAR ID:
- 10461741
- Date Published:
- Journal Name:
- Sensors
- Volume:
- 23
- Issue:
- 11
- ISSN:
- 1424-8220
- Page Range / eLocation ID:
- 5159
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Adebisi, John (Ed.)Non-expert users can now program robots using various end-user robot programming methods, which have widened the use of robots and lowered barriers preventing robot use by laypeople. Kinesthetic teaching is a common form of end-user robot programming, allowing users to forgo writing code by physically guiding the robot to demonstrate behaviors. Although it can be more accessible than writing code, kinesthetic teaching is difficult in practice because of users’ unfamiliarity with kinematics or limitations of robots and programming interfaces. Developing good kinesthetic demonstrations requires physical and cognitive skills, such as the ability to plan effective grasps for different task objects and constraints, to overcome programming difficulties. How to help users learn these skills remains a largely unexplored question, with users conventionally learning through self-guided practice. Our study compares how self-guided practice compares with curriculum-based training in building users’ programming proficiency. While we found no significant differences between study participants who learned through practice compared to participants who learned through our curriculum, our study reveals insights into factors contributing to end-user robot programmers’ confidence and success during programming and how learning interventions may contribute to such factors. Our work paves the way for further research on how to best structure training interventions for end-user robot programmers.more » « less
-
Abstract Engaging youth in early and sustained conservation education has important implications for promoting positive attitudes and behaviors in those who will become the future of conservation and management. Toward this goal, visual narratives (comic books, graphic novels) are an increasingly popular method used by conservation scientists to educate young people due to their approachable use of art and narrative storytelling. However, no studies have directly assessed how visual narratives compare with more traditional forms of conservation education for youth. We asked, how does education about biodiversity through visual narrative affect student perceptions and knowledge of science content relative to a traditional resource, and is there a novelty effect when using visual narrative versus traditional resources? To assess our questions, we utilized a semistructured approach to develop a biodiversity education program. Specifically, we developed an original graphic novel (visual narrative treatment) and a slideshow presentation (traditional treatment) with the same content to educate children about wetland biodiversity. We recruited, trained, and randomized 26 third‐grade teachers to deliver either the visual narrative or traditional resource in their classrooms. Students completed pretest, posttest, and follow‐up surveys assessing their perceptions of science and knowledge of the lesson content. Students in the visual narrative treatment held more positive perceptions of science (by 3.79%,p = 0.001), whereas students in the traditional treatment performed better on content quizzes (by 7.97%,p = 0.002). We found evidence for a novelty bias when using the visual narrative but not the traditional resource. These findings point to the importance of understanding the target audience and clearly defining educational goals. Overall, our results contribute to broader understanding of the relative benefits and limitations of conservation education through nontraditional means and of practices for successfully delivering effective, accessible, and rewarding conservation education to educators and youth.more » « less
-
Automated writing evaluation (AWE) systems automatically assess and provide students with feedback on their writing. Despite learning benefits, students may not effectively interpret and utilize AI-generated feedback, thereby not maximizing their learning outcomes. A closely related issue is the accuracy of the systems, that students may not understand, are not perfect. Our study investigates whether students differentially addressed false positive and false negative AI-generated feedback errors on their science essays. We found that students addressed nearly all the false negative feedback; however, they addressed less than one-fourth of the false positive feedback. The odds of addressing a false positive feedback was 99% lower than addressing a false negative feedback, representing significant missed opportunities for revision and learning. We discuss the implications of these findings in the context of students’ learning.more » « less
-
Integrating narrative elements into a game is a key element in de- signing an immersive experience. Narrative has been hypothesized to improve engagement, motivation, and learning within educational environments. While empirical results have been produced to show that narrative enhances engagement and motivation, its effects on learning were shown to either be insignificant or negative. We, there- fore, aim to address the question of how to integrate narrative in a game to improve learning. We address this through the design of May’s Journey, an educational game that teaches basic programming concepts where a story is integrated. The game design seamlessly integrates learning goals, core mechanic and narrative elements. In this paper, we discuss the game design as well as a study we con- ducted to compare two game versions, one with rich narrative and the other with light narrative. Results demonstrate that participants who interacted with the rich narrative version had fewer program- ming errors and increased engagement within the game. We present our contributions in the form of educational design principles for narrative integration supported by our study and results.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.3390/s23115159
