More Like this

1. Promoting Social Engagement With a Multi-Role Dancing Robot for In-Home Autism Care

   https://doi.org/10.3389/frobt.2022.880691  

   Javed, Hifza; Park, Chung Hyuk (June 2022, Frontiers in Robotics and AI)

   This work describes the design of real-time dance-based interaction with a humanoid robot, where the robot seeks to promote physical activity in children by taking on multiple roles as a dance partner. It acts as a leader by initiating dances but can also act as a follower by mimicking a child’s dance movements. Dances in the leader role are produced by a sequence-to-sequence (S2S) Long Short-Term Memory (LSTM) network trained on children’s music videos taken from YouTube. On the other hand, a music orchestration platform is implemented to generate background music in the follower mode as the robot mimics the child’s poses. In doing so, we also incorporated the largely unexplored paradigm of learning-by-teaching by including multiple robot roles that allow the child to both learn from and teach to the robot. Our work is among the first to implement a largely autonomous, real-time full-body dance interaction with a bipedal humanoid robot that also explores the impact of the robot roles on child engagement. Importantly, we also incorporated in our design formal constructs taken from autism therapy, such as the least-to-most prompting hierarchy, reinforcements for positive behaviors, and a time delay to make behavioral observations. We implemented a multimodal child engagement model that encompasses both affective engagement (displayed through eye gaze focus and facial expressions) as well as task engagement (determined by the level of physical activity) to determine child engagement states. We then conducted a virtual exploratory user study to evaluate the impact of mixed robot roles on user engagement and found no statistically significant difference in the children’s engagement in single-role and multiple-role interactions. While the children were observed to respond positively to both robot behaviors, they preferred the music-driven leader role over the movement-driven follower role, a result that can partly be attributed to the virtual nature of the study. Our findings support the utility of such a platform in practicing physical activity but indicate that further research is necessary to fully explore the impact of each robot role. 

   more » « less
2. Finding Alignment in Framing: Dynamics of Collaborative Disciplinary Engagement in Science

   Krishnan, H.; Jaber, L. Z.; Schellinger, J.; & Southerland, S. A. (January 2021, National Association for Research in Science Teaching Annual Meeting 2021)

   null (Ed.)

   Recent educational reforms conceptualize science classrooms as spaces where students collaboratively engage in disciplinary practices to construct and evaluate scientific explanations of phenomena. For students to effectively collaborate with each other, they need to develop a shared framing of the nature of the science activity and the expectations surrounding their engagement in it. Such framing does not only pertain to the conceptual work but also involves myriad epistemological, social, and affective dimensions. We conceptualize collaborative disciplinary engagement as the process of aligning the group’s framing along these dimensions and, we argue, student negotiations to achieve this alignment are in part what initiate and sustain collaborative disciplinary engagement in the science classroom. By focusing on student negotiations, this study builds on existing research on group dynamics involved in science learning and contributes nuanced empirical insights on the nature of student negotiations along the conceptual, epistemological, social, and affective dimensions of argumentation in science. Moreover, the findings provide a proof of concept regarding the key role that student negotiations of framing have in driving collaborative disciplinary engagement. The study findings have implications for research and practice to support learners’ productive disciplinary engagement in group work in the science classroom and beyond. 

   more » « less
3. Impact of Interaction Context on the Student Affect-Learning Relationship in Child-Robot Interaction

   https://doi.org/10.1145/3319502.3374822  

   Chen, Huili; Park, Hae Won; Zhang, Xiajie; Breazeal, Cynthia (March 2020, ACM/IEEE International Conference on Human-Robot Interaction)

   Prior work in affect-aware educational robots has often relied on a common belief that the relationship between student affect and learning is independent of agent behaviors (child’s/robot’s) or unidirectional (positive/negative but not both) throughout the entire student-robot interaction.We argue that the student affect-learning relationship should be interpreted in two contexts: (1) social learning paradigm and (2) sub-events within child-robot interaction. In our paper, we examine two different social learning paradigms where children interact with a robot that acts either as a tutor or a tutee. Sub-events within child-robot interaction are defined as task-related events occurring in specific phases of an interaction (e.g., when the child/robot gets a wrong answer). We examine subevents at a macro level (entire interaction) and a micro level (within specific sub-events). In this paper, we provide an in-depth correlation analysis of children’s facial affect and vocabulary learning. We found that children’s affective displays became more predictive of their vocabulary learning when children interacted with a tutee robot who did not scaffold their learning. Additionally, children’s affect displayed during micro-level events was more predictive of their learning than during macro-level events. Last, we found that the affect-learning relationship is not unidirectional, but rather is modulated by context, i.e., several affective states facilitated student learning when displayed in some sub-events but inhibited learning when displayed in others. These findings indicate that both social learning paradigm and sub-events within interaction modulate student affect-learning relationship. 

   more » « less
4. In-the-Wild Affect Analysis of Children with ASD Using Heart Rate

   https://doi.org/10.3390/s23146572  

   Ali, Kamran; Shah, Sachin; Hughes, Charles E. (July 2023, Sensors)

   Recognizing the affective state of children with autism spectrum disorder (ASD) in real-world settings poses challenges due to the varying head poses, illumination levels, occlusion and a lack of datasets annotated with emotions in in-the-wild scenarios. Understanding the emotional state of children with ASD is crucial for providing personalized interventions and support. Existing methods often rely on controlled lab environments, limiting their applicability to real-world scenarios. Hence, a framework that enables the recognition of affective states in children with ASD in uncontrolled settings is needed. This paper presents a framework for recognizing the affective state of children with ASD in an in-the-wild setting using heart rate (HR) information. More specifically, an algorithm is developed that can classify a participant’s emotion as positive, negative, or neutral by analyzing the heart rate signal acquired from a smartwatch. The heart rate data are obtained in real time using a smartwatch application while the child learns to code a robot and interacts with an avatar. The avatar assists the child in developing communication skills and programming the robot. In this paper, we also present a semi-automated annotation technique based on facial expression recognition for the heart rate data. The HR signal is analyzed to extract features that capture the emotional state of the child. Additionally, in this paper, the performance of a raw HR-signal-based emotion classification algorithm is compared with a classification approach based on features extracted from HR signals using discrete wavelet transform (DWT). The experimental results demonstrate that the proposed method achieves comparable performance to state-of-the-art HR-based emotion recognition techniques, despite being conducted in an uncontrolled setting rather than a controlled lab environment. The framework presented in this paper contributes to the real-world affect analysis of children with ASD using HR information. By enabling emotion recognition in uncontrolled settings, this approach has the potential to improve the monitoring and understanding of the emotional well-being of children with ASD in their daily lives. 

   more » « less
5. Building a reinforcement learning environment from limited data to optimize teachable robot interventions.

   Maidment, T.; Yu, M.; Lobczowski, N. G; Kovashka, A.; Walker, E.; Litman, D.; & Nokes-Malach, T. (January 2022, Proceedings of the 15th International Conference on Educational Data Mining)

   Mitrovic, A.; & Bosch, N. (Ed.)

   Working collaboratively in groups can positively impact performance and student engagement. Intelligent social agents can provide a source of personalized support for students, and their benefits likely extend to collaborative settings, but it is difficult to determine how these agents should interact with students. Reinforcement learning (RL) offers an opportunity for adapting the interactions between the social agent and the students to better support collaboration and learning. However, using RL in education with social agents typically involves training using real students. In this work, we train an RL agent in a high-quality simulated environment to learn how to improve students’ collaboration. Data was collected during a pilot study with dyads of students who worked together to tutor an intelligent teachable robot. We explore the process of building an environment from the data, training a policy, and the impact of the policy on different students, compared to various baselines. 

   more » « less