An official website of the United States government
The incorporation of technology into primary and secondary education has facilitated the creation of curricula that utilize computational tools for problem-solving. In Open-Ended Learning Environments (OELEs), students participate in learning-by- modeling activities that enhance their understanding of (Science, technology, engineering, and mathematics) STEM and computational concepts. This research presents an innovative multimodal emotion recognition approach that analyzes facial expressions and speech data to identify pertinent learning-centered emotions, such as engagement, delight, confusion, frustration, and boredom. Utilizing sophisticated machine learning algorithms, including High-Speed Face Emotion Recognition (HSEmotion) model for visual data and wav2vec 2.0 for auditory data, our method is refined with a modality verification step and a fusion layer for accurate emotion classification. The multimodal technique significantly increases emotion detection accuracy, with an overall accuracy of 87%, and an Fl -score of 84%. The study also correlates these emotions with model building strategies in collaborative settings, with statistical analyses indicating distinct emotional patterns associated with effective and ineffective strategy use for tasks model construction and debugging tasks. These findings underscore the role of adaptive learning environments in fostering students' emotional and cognitive development.
more »
« less
This content will become publicly available on June 10, 2026
Using Markov Chain Analysis to Study the Relations Between Emotions, Cognitive Actions, and Performance in Collaborative Learning
Collaborative problem-solving (CPS) in STEM+C education involves cognitive coordination and emotional regulation during joint tasks. Prior research has examined discrete affective states in learning environments but less is known about how these emotions evolve over time and affect CPS behavior. This study investigates the temporal dynamics of five emotions—engagement, confusion, boredom, delight, and frustration—using Markov Chain analysis of data from high school pairs building computational models in the C2STEM environment. Emotional transitions aligned with cognitive processes, seen in interaction patterns like PLAY, ADJUST, and BUILD, to analyze affect during modeling. Results show that emotional trajectories closely relate to cognitive actions, including construction, simulation testing, and debugging. Transitions that maintained engagement linked to productive collaboration and stronger performance, while ongoing frustration and boredom indicated disengagement progress.
more »
« less
- PAR ID:
- 10612387
- Publisher / Repository:
- Proceedings of the 2025 Meeting of the International Society of the Learning Sciences
- Date Published:
- Page Range / eLocation ID:
- 440 to 444
- Format(s):
- Medium: X
- Location:
- https://repository.isls.org//handle/1/11890
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Research on social, emotional, and academic development of children often notes the critical role of parents. Yet, how parents perceive and engage with children’s reactions to difficulty and perceived failure, to then shape their perspective and engagement with learning remains under investigated. The current study explored children and parents’ perceptions of and reactions to frustration and failure within an out-of-school, home-based engineering program. Specifically, we asked 1) How was failure perceived by participating families? and 2) What was the subsequent action/reaction to that failure? Data were derived from post-program interviews with children and parents who participated in a home-based, elementary engineering program involving take-home kits and self-identified engineering projects. Findings derived from descriptive qualitative methods and thematic analysis illustrated development of parent thinking around failure and frustration, both within themselves and their reactions to seeing such emotions in their children. Analysis further revealed how such emotions emerge within their children and impact their experiences. These findings shed light on ways child-parent engagement and the tactics employed by parents may influence a child’s perseverance and willingness to work through difficulty. This research represents an entry point for investigating how parents perceive and react to failures and challenges, and how these reactions shape their communication around failure with their children. Such parental reactions and communication may shape children’s mindset development, perspectives, and engagement. Implications for family engagement and influence on children’s learning through academic emotions in STEM and engineering are discussed.more » « less
-
Science Education has transitioned to science proficiency-- students are to gain the ability to engage in sense making about the natural world (National Research Council [NRC, 2012])--learning to “figure things out” (Passmore, 2014). One emerging area of focus is the emotional work students participate in during science sense making. There is growing recognition that these emotions are not just unnecessary by-products of scientific work, but rather they are part-and-parcel of doing science, as these emotions are part of what “instigates and stabilizes disciplinary engagement” in scientific pursuits (Jaber & Hammer, 2016b, p. 189). The research question that guided this study is: What is the teacher's role in reframing moments of epistemic vexation, so students experience productive meta-affect in the science classroom? After reviewing video footage and student and teacher interviews, three themes emerged: (1) Without reframing from the teacher during moments of epistemic vexation, students disengage from sense-making, (2) Productive meta-affect is more likely to occur when students understand why the teacher allows for failure to connect ideas or understand scientific concepts, and (3) When the teacher does not reframe moments of epistemic vexation, students build solidarity and reach out to each other for emotional support in developing productive meta-affect.more » « less
-
In the past 2.5 years, STEM faculty around the world have faced unprecedent challenges, complete upheaval of routine, and staggering loss. Managing these new realities has required significant emotional labor. This paper offers one perspective on the emotional reality of teaching collegiate chemistry at a large public research university in the United States. Considering and describing emotions such as frustration, grief, anger and hope, I share a hidden reality of being faculty during a pandemic. I also discuss how we might learn from the path traveled and more deftly navigate the road ahead.more » « less
-
Abstract The positivity principle states that people learn better from instructors who display positive emotions rather than negative emotions. In two experiments, students viewed a short video lecture on a statistics topic in which an instructor stood next to a series of slides as she lectured and then they took either an immediate test (Experiment 1) or a delayed test (Experiment 2). In a between-subjects design, students saw an instructor who used her voice, body movement, gesture, facial expression, and eye gaze to display one of four emotions while lecturing: happy (positive/active), content (positive/passive), frustrated (negative/active), or bored (negative/passive). First, learners were able to recognize the emotional tone of the instructor in an instructional video lecture, particularly by more strongly rating a positive instructor as displaying positive emotions and a negative instructor as displaying negative emotions (in Experiments 1 and 2). Second, concerning building a social connection during learning, learners rated a positive instructor as more likely to facilitate learning, more credible, and more engaging than a negative instructor (in Experiments 1 and 2). Third, concerning cognitive engagement during learning, learners reported paying more attention during learning for a positive instructor than a negative instructor (in Experiments 1 and 2). Finally, concerning learning outcome, learners who had a positive instructor scored higher than learners who had a negative instructor on a delayed posttest (Experiment 2) but not an immediate posttest (Experiment 1). Overall, there is evidence for the positivity principle and the cognitive-affective model of e-learning from which it is derived.more » « less
