More Like this

1. Says Who? How different ground truth measures of emotion impact student affective modeling.

   Zambrano, A; Ocumpaugh, J; Nasiar, N; Goslen, A; Zhang, J; Rowe, J; Esiason, J; Vandenberg, J; Hutt, S (January 2024, Proceedings of the 17th International Conference on Educational Data Mining.)

   Research into student affect detection has historically relied on ground truth measures of emotion that utilize one of three sources of data: (1) self-report data, (2) classroom observations, or (3) sen- sor data that is retrospectively labeled. Although a few studies have compared sensor- and observation-based approaches to student af- fective modeling, less work has explored the relationship between self-report and classroom observations. In this study, we use both recurring self-reports (SR) and classroom observation (BROMP) to measure student emotion during a study involving middle school students interacting with a game-based learning environment for microbiology education. We use supervised machine learning to develop two sets of affect detectors corresponding to SR and BROMP-based measures of student emotion, respectively. We compare the two sets of detectors in terms of their most relevant features, as well as correlations of their output with measures of student learning and interest. Results show that highly predictive features in the SR detectors are different from those selected for BROMP-based detectors. The associations with interest and moti- vation measures show that while SR detectors captured underlying motivations, the BROMP detectors seemed to capture more in-the- moment information about the student’s experience. Evidence sug- gests that there is benefit of using both sources of data to model different components of student affect. 

   more » « less
2. Studying Affect Dynamics and Chronometry Using Sensor-Free Detectors

   Botelho, A. F.; Baker, R. S.; Ocumpaugh, J.; Heffernan, N. T. (January 2018, Proceedings of the 11th International Conference on Educational Data Mining)

   Student affect has been found to correlate with short- and long-term learning outcomes, including college attendance as well as interest and involvement in Science, Technology, Engineering, and Mathematics (STEM) careers. However, there still remain significant questions about the processes by which affect shifts and develops during the learning process. Much of this research can be split into affect dynamics, the study of the temporal transitions between affective states, and affective chronometry, the study of how an affect state emerges and dissipates over time. Thus far, these affective processes have been primarily studied using field observations, sensors, or student self-report measures; however, these approaches can be coarse, and obtaining finer grained data produces challenges to data fidelity. Recent developments in sensor-free detectors of student affect, utilizing only the data from student interactions with a computer based learning platform, open an opportunity to study affect dynamics and chronometry at moment-to-moment levels of granularity. This work presents a novel approach, applying sensor-free detectors to study these two prominent problems in affective research. 

   more » « less
3. Introducing The Focus & Action of Students & Teachers Observation Protocol (FASTOP)

   https://doi.org/10.18260/1-2--43856  

   Kussmaul, Clifton; Campbell, Patricia; Torres-Demas, Maria; Mayfield, Chris; Hu, Helen (June 2023, ASEE Conferences)

   This methods paper describes the development, use, and initial findings for the Focus and Actions of Students and Teachers Observation Protocol (FASTOP). The ICAP model describes the benefits of interactive (I), constructive (C), and active (A) learning over passive (P) learning. However, instructors who seek to adopt more effective pedagogies often overestimate their use of such practices and/or omit key elements. Thus, our research seeks to enhance understanding of classroom practice by combining data from student surveys, instructor surveys, and classroom observations (both live and video recorded). This paper describes a new classroom observation protocol intended to monitor the focus (e.g., solo, pair, team, or whole class) and action (e.g., discuss, speak/present, watch/listen, or distracted) of both students and teachers (instructors). The paper summarizes relevant background on evidence-based learning, student engagement, and classroom observation protocols, describes the development and structure of FASTOP, presents results from different pedagogies (e.g., lecture, laboratory, POGIL), and describes lessons learned and future directions. Results show distinctive patterns of student and teacher behaviors for different pedagogies. 

   more » « less
4. Language measures correlate with other measures used to study emotion

   https://doi.org/10.1038/s44271-025-00212-x  

   Munin, Shaina; Ong, Desmond_C; Okland, Sydney; Freedman, Gili; Beer, Jennifer_S (February 2025, Communications Psychology)

   Abstract Researchers are increasingly using language measures to study emotion, yet less is known about whether language relates to other measures often used to study emotion. Building on previous work which focuses on associations between language and self-report, we test associations between language and a broader range of measures (self-report, observer report, facial cues, vocal cues). Furthermore, we examine associations across different dictionaries (LIWC-22, NRC, Lexical Suite, ANEW, VADER) used to estimate valence (i.e., positive versus negative emotion) or discrete emotions (i.e., anger, fear, sadness) in language. Associations were tested in three large, multimodal datasets (Ns = 193–1856; average word count = 316.7–2782.8). Language consistently related to observer report and consistently related to self-report in two of the three datasets. Statistically significant associations between language and facial cues emerged for language measures of valence but not for language measures of discrete emotions. Language did not consistently show significant associations with vocal cues. Results did not tend to significantly vary across dictionaries. The current research suggests that language measures (in particular, language measures of valence) are correlated with a range of other measures used to study emotion. Therefore, researchers may wish to use language to study emotion when other measures are unavailable or impractical for their research question. 

   more » « less
5. Machine-Learned or Expert-Engineered Features? Exploring Feature Engineering Methods in Detectors of Student Behavior and Affect

   Botelho, A. F.; Baker, R. S.; Heffernan, N. T. (July 2019, The Twelfth International Conference on Educational Data Mining)

   There is a long history of research on the development of models to detect and study student behavior and affect. Developing computer-based models has allowed the study of learning constructs at fine levels of granularity and over long periods of time. For many years, these models were developed using features based on previous educational research from the raw log data. More recently, however, the application of deep learning models has often skipped this feature-engineering step by allowing the algorithm to learn features from the fine-grained raw log data. As many of these deep learning models have led to promising results, researchers have asked which situations may lead to machine-learned features performing better than expert-generated features. This work addresses this question by comparing the use of machine-learned and expert-engineered features for three previously-developed models of student affect, off-task behavior, and gaming the system. In addition, we propose a third feature-engineering method that combines expert features with machine learning to explore the strengths and weaknesses of these approaches to build detectors of student affect and unproductive behaviors. 

   more » « less