skip to main content


Title: Modified Playback of Avatar Clip Sequences Based on Student Attention in Educational VR
We demonstrate a system that sequences teacher avatar clips considering student eye tracking. We are investigating subjective suitability of avatar responses to student misunderstandings or inattention. Three different avatar behaviors are demonstrated to allow a teacher pedagogical agent to behave more appropriately to student attention or distraction. An in-game mobile device provides an experiment control mechanism for 2 levels of distractions.  more » « less
Award ID(s):
1815976
NSF-PAR ID:
10168891
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)
Page Range / eLocation ID:
850 to 851
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    We study student experiences of social VR for remote instruction, with students attending class from home. The study evaluates student experiences when: (1) viewing remote lectures with VR headsets, (2) viewing with desktop displays, (3) presenting with VR headsets, and (4) reflecting on several weeks of VR-based class attendance. Students rated factors such as presence, social presence, simulator sickness, communication methods, avatar and application features, and tradeoffs with other remote approaches. Headset-based viewing and presenting produced higher presence than desktop viewing, but had less-clear impact on overall experience and on most social presence measures. We observed higher attentional allocation scores for headset-based presenting than for both viewing methods. For headset VR, there were strong negative correlations between simulator sickness (primarily reported as general discomfort) and ratings of co-presence, overall experience, and some other factors. This suggests that comfortable users experienced substantial benefits of headset viewing and presenting, but others did not. Based on the type of virtual environment, student ratings, and comments, reported discomfort appears related to physical ergonomic factors or technical problems. Desktop VR appears to be a good alternative for uncomfortable students, and students report that they prefer a mix of headset and desktop viewing. We additionally provide insight from students and a teacher about possible improvements for VR class technology, and we summarize student opinions comparing viewing and presenting in VR to other remote class technologies. 
    more » « less
  2. We investigated student perceptions of cold calling on their feelings of anxiousness and how graduate teaching assistants (GTAs) alleviated these feelings when students shared their ideas publicly in the context of tutorial and laboratory sessions. Physics and chemistry GTAs who led active-learning tutorials and labs practiced cold calling paired with error framing with avatar-students in a mixed-reality simulator at the beginning of the semester. Then, we observed the GTAs teaching real students in their actual classroom. We recruited eleven students from sections led by GTAs who were observed to use cold calling in their classroom to participate in semi-structured interviews. Several students reported that cold calling increased their feelings of anxiousness. However, students also reported that GTAs used strategies paired with cold calling that reduced their feelings of anxiousness, such as acknowledging student responses as valuable and remembering student names. We discuss implications for professional development on active learning strategies. 
    more » « less
  3. Wang, N. ; Rebolledo-Mendez, G. ; Matsuda, N. ; Santos, O.C. ; Dimitrova, V. (Ed.)
    Research indicates that teachers play an active and important role in classrooms with AI tutors. Yet, our scientific understanding of the way teacher practices around AI tutors mediate student learning is far from complete. In this paper, we investigate spatiotemporal factors of student-teacher interactions by analyzing student engagement and learning with an AI tutor ahead of teacher visits (defined as episodes of a teacher being in close physical proximity to a student) and immediately following teacher visits. To conduct such integrated, temporal analysis around the moments when teachers visit students, we collect fine-grained, time-synchronized data on teacher positions in the physical classroom and student interactions with the AI tutor. Our case study in a K12 math classroom with a veteran math teacher provides some indications on factors that might affect a teacher’s decision to allocate their limited classroom time to their students and what effects these interactions have on students. For instance, teacher visits were associated more with students’ in-the-moment behavioral indicators (e.g., idleness) than a broader, static measure of student needs such as low prior knowledge. While teacher visits were often associated with positive changes in student behavior afterward (e.g., decreased idleness), there could be a potential mismatch between students visited by the teacher and who may have needed it more at that time (e.g., students who were disengaged for much longer). Overall, our findings indicate that teacher visits may yield immediate benefits for students but also that it is challenging for teachers to meet all needs - suggesting the need for better tool support. 
    more » « less
  4. Knowledge distillation is a popular technique for training a small student network to emulate a larger teacher model, such as an ensemble of networks. We show that while knowledge distillation can improve student generalization, it does not typically work as it is commonly understood: there often remains a surprisingly large discrepancy between the predictive distributions of the teacher and the student, even in cases when the student has the capacity to perfectly match the teacher. We identify difficulties in optimization as a key reason for why the student is unable to match the teacher. We also show how the details of the dataset used for distillation play a role in how closely the student matches the teacher --- and that more closely matching the teacher paradoxically does not always lead to better student generalization. 
    more » « less
  5. Policy distillation, which transfers a teacher policy to a student policy has achieved great success in challenging tasks of deep reinforcement learning. This teacher-student framework requires a well-trained teacher model which is computationally expensive. Moreover, the performance of the student model could be limited by the teacher model if the teacher model is not optimal. In the light of collaborative learning, we study the feasibility of involving joint intellectual efforts from diverse perspectives of student models. In this work, we introduce dual policy distillation (DPD), a student-student framework in which two learners operate on the same environment to explore different perspectives of the environment and extract knowledge from each other to enhance their learning. The key challenge in developing this dual learning framework is to identify the beneficial knowledge from the peer learner for contemporary learning-based reinforcement learning algorithms, since it is unclear whether the knowledge distilled from an imperfect and noisy peer learner would be helpful. To address the challenge, we theoretically justify that distilling knowledge from a peer learner will lead to policy improvement and propose a disadvantageous distillation strategy based on the theoretical results. The conducted experiments on several continuous control tasks show that the proposed framework achieves superior performance with a learning-based agent and function approximation without the use of expensive teacher models.

     
    more » « less