Step-based tutoring systems are known to be more effective than traditional answer-based
systems. They however require that each step in a student’s work be accepted and evaluated automatically to provide effective feedback. In the domain of linear circuit analysis, it is frequently necessary to allow students to draw or edit circuits on their screen to simplify or otherwise transform them. Here, the interface developed to accept such input and provide
immediate feedback in the Circuit Tutor system is described, along with systematic assessment data. Advanced simplification methods such as removing circuit sections that are removably hinged, voltage-splittable, or current-splittable are taught to students in an interactive tutorial and then supported in the circuit editor itself. To address the learning curve associated with such an interface, ~70 video tutorials were created to demonstrate exactly how to work the randomly generated problems at each level of each of the tutorials in the system. A complete written record or “transcript” of student’s work in the system is being made available, showing both incorrect and correct steps. Introductory interactive (multiple choice) tutorials are now included on most topics. Assessment of exercises using the interactive editor was carried out by professional evaluators for several institutions, including three that heavily serve underrepresented minorities. Both quantitative and qualitative methods were used, including focus groups, surveys, and interviews. Controlled, randomized, blind evaluations were carried out in three different course sections in Spring and Fall 2019 to evaluate three tutorials using the interactive editor, comparing use of Circuit Tutor to both a commercial answer-based system and to conventional textbook-based paper homework. In Fall 2019, students rated the software a mean of 4.14/5 for being helpful to learn the material vs. 3.05/5 for paper homework (HW), p < 0.001 and effect size d = 1.11σ. On relevant exam questions that semester, students scored significantly (p = 0.014) higher with an effect size of d = 0.64σ when using Circuit Tutor compared to paper HW in one class section, with no significant difference in the other section.
more »
« less
“We Make a Great Team!”: Adults with Low Prior Domain Knowledge Learn more from a Peer Robot than a Tutor Robot
In peer tutoring, the learner is taught by a colleague
rather than by a traditional tutor. This strategy has been shown
to be effective in human tutoring, where students have higher
learning gains when taught by a peer instead of a traditional
tutor. Similar results have been shown in child-robot interactions
studies, where a peer robot was more effective than a tutor
robot at teaching children. In this work, we compare skill
increase and perception of a peer robot to a tutor robot when
teaching adults. We designed a system in which a robot provides
personalized help to adults in electronic circuit construction.
We compare the number of learned skills and preferences of
a peer robot to a tutor robot. Participants in both conditions
improved their circuit skills after interacting with the robot.
There were no significant differences in number of skills learned
between conditions. However, participants with low prior domain
knowledge learned significantly more with a peer robot than
a tutor robot. Furthermore, the peer robot was perceived as
friendlier, more social, smarter, and more respectful than the
tutor robot, regardless of initial skill level.
more »
« less
- NSF-PAR ID:
- 10354179
- Date Published:
- Journal Name:
- ACM/IEEE International Conference on Human-Robot Interaction (HRI 2022)
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Abstract Autism spectrum disorder (ASD) is a lifelong developmental condition that affects an individual’s ability to communicate and relate to others. Despite such challenges, early intervention during childhood development has shown to have positive long-term benefits for individuals with ASD. Namely, early childhood development of communicative speech skills has shown to improve future literacy and academic achievement. However, the delivery of such interventions is often time-consuming. Socially assistive robots (SARs) are a potential strategic technology that could help support intervention delivery for children with ASD and increase the number of individuals that healthcare professionals can positively affect. For SARs to be effectively integrated in real-world treatment for individuals with ASD, they should follow current evidence-based practices used by therapists such as Applied Behavior Analysis (ABA). In this work, we present a study that investigates the efficacy of applying well-known ABA techniques to a robot-mediated listening comprehension intervention delivered to children with ASD at a university-based ABA clinic. The interventions were delivered in place of human therapists to teach study participants a new skill as a part of their overall treatment plan. All the children participating in the intervention improved in the skill being taught by the robot and enjoyed interacting with the robot, as evident by high occurrences of positive affect as well as engagement during the sessions. One of the three participants has also reached mastery of the skill via the robot-mediated interventions.more » « less
-
Selecting appropriate tutoring help actions that account for both a student’s content mastery and engagement level is essential for effective human tutors, indicating the critical need for these skills in autonomous tutors. In this work, we formulate the robot-student tutoring help action selection problem as the Assistive Tutor partially observable Markov decision process (AT-POMDP). We designed the AT-POMDP and derived its parameters based on data from a prior robot-student tutoring study. The policy that results from solving the ATPOMDP allows a robot tutor to decide upon the optimal tutoring help action to give a student, while maintaining a belief of the student’s mastery of the material and engagement with the task. This approach is validated through a between-subjects field study, which involved 4th grade students (n = 28) interacting with a social robot solving long division problems over five sessions. Students who received help from a robot using the AT-POMDP policy demonstrated significantly greater learning gains than students who received help from a robot with a fixed help action selection policy. Our results demonstrate that this robust computational framework can be used effectively to deliver diverse and personalized tutoring support over time for students.more » « less
-
ABET lists the ability to communicate in writing to both technical and non-technical audiences as a required outcome for baccalaureate engineering students [1]. From emails and memos to formal reports, the ability to communicate is vital to the engineering profession. This Work in Progress paper describes research being done as part of an NSF-funded project, Writing Assignment Tutor Training in STEM (WATTS). The method is designed to improve feedback writing tutors without technical backgrounds give to engineering students on technical reports. Students in engineering programs have few opportunities to develop their writing skills. Usually, composition courses are part of the general education curriculum. Students often see these courses as unrelated to their majors and careers [2]. Ideally, writing support should be integrated throughout a program. Since WATTs capitalizes on existing resources and requires only a modest amount of faculty time, it could enable engineering programs to provide additional writing support to students in multiple courses and provide a bridge for them to see the connection between writing concepts learned in composition courses and their technical reports. WATTS was developed in a junior-level circuit analysis course, where students were completing the same lab and writing individual reports. This paper focuses on a senior capstone course that utilizes concepts taught in previous courses to prepare students to complete an independent team research or design project. Projects are unique, usually based on the needs of an industrial sponsor, and are completed over three consecutive semesters. Each semester, teams write a report based on their activities during that semester, with a comprehensive report in the final semester. The multi-semester nature of the senior design project provides an opportunity for the researchers to chart longitudinal changes from the first to the students’ third semester interactions with the writing tutors, assessing the value of an integrated approach. The program’s impact on students’ attitudes toward revision and the value of tutoring, as well as the impact on tutors, are part of the assessment plan. The program hopes to change the students’ focus from simply presenting their results to communicating them. The goals of the project are to demonstrate to students that revision is essential to the writing process and that feedback can improve their written communication abilities. The expectation is that after graduation they will continue to seek critical feedback as part of their career growth. Surveys given to both students and tutors revealed that the sessions were taken seriously by the students and that meaningful collaboration was achieved between them. An evaluation of the writing in pre-tutored to final submitted report shows statistically significant improvement. Preliminary and current results will be included within the paper. [1] Criteria for Accrediting Engineering Technology Programs, ABET, Baltimore, MD., 2020, p.5, ETAC Criteria (abet.org) [2] Bergmann, L. S. and Zepernick, J., “Disciplinarity and Transfer: Students’ Perceptions of Learning to Write,” Writing Program Administration, 31, Fall/Winter 2007.more » « less
-
more » « less
Abstract Background A robotic rehabilitation gym can be defined as multiple patients training with multiple robots or passive sensorized devices in a group setting. Recent work with such gyms has shown positive rehabilitation outcomes; furthermore, such gyms allow a single therapist to supervise more than one patient, increasing cost-effectiveness. To allow more effective multipatient supervision in future robotic rehabilitation gyms, we propose an automated system that could dynamically assign patients to different robots within a session in order to optimize rehabilitation outcome.
Methods As a first step toward implementing a practical patient-robot assignment system, we present a simplified mathematical model of a robotic rehabilitation gym. Mixed-integer nonlinear programming algorithms are used to find effective assignment and training solutions for multiple evaluation scenarios involving different numbers of patients and robots (5 patients and 5 robots, 6 patients and 5 robots, 5 patients and 7 robots), different training durations (7 or 12 time steps) and different complexity levels (whether different patients have different skill acquisition curves, whether robots have exit times associated with them). In all cases, the goal is to maximize total skill gain across all patients and skills within a session.
Results Analyses of variance across different scenarios show that disjunctive and time-indexed optimization models significantly outperform two baseline schedules: staying on one robot throughout a session and switching robots halfway through a session. The disjunctive model results in higher skill gain than the time-indexed model in the given scenarios, and the optimization duration increases as the number of patients, robots and time steps increases. Additionally, we discuss how different model simplifications (e.g., perfectly known and predictable patient skill level) could be addressed in the future and how such software may eventually be used in practice.
Conclusions Though it involves unrealistically simple scenarios, our study shows that intelligently moving patients between different rehabilitation robots can improve overall skill acquisition in a multi-patient multi-robot environment. While robotic rehabilitation gyms are not yet commonplace in clinical practice, prototypes of them already exist, and our study presents a way to use intelligent decision support to potentially enable more efficient delivery of technologically aided rehabilitation.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/HRI53351.2022.9889441
