More Like this

1. Multimodal analytics for collaborative teacher reflection of human-ai hybrid teaching: design opportunities and constraints

   https://doi.org/10.1007/978-3-031-42682-7_45  

   Karumbaiah, S.; Liu, P.; Maksimova, A.; De Vylder, L.; Rummel, N.; Aleven, V. (August 2023, Springer)

   Viberg, O.; Jivet, I.; Muñoz-Merino, P.; Perifanou, M.; Papathoma, T. (Ed.)

   Past research shows that teachers benefit immensely from reflecting on their classroom practices. At the same time, adaptive and artificially intelligent (AI) tutors are shown to be highly effective for students, especially when teachers are involved in supporting students’ learning. Yet, there is little research on how to support teachers to reflect on their practices around AI tutors. We posit that analytics built on multimodal data from the classroom (e.g., teacher position, student-AI interaction) would be beneficial in providing effective scaffolding and evidence for teachers’ collaborative reflection on human-AI hybrid teaching. To better understand the design opportunities and constraints of a future tool for teacher reflection, we conducted storyboarding sessions with seven in-service teachers. Our analysis revealed that certain modalities (e.g., position v. video) might be more beneficial and less constrained than others in identifying reflection-worthy moments and trends. We discuss teachers’ needs for reflection in classrooms with AI tutors and their boundaries in using multimodal analytics. 

   more » « less
2. Leveraging multimodal classroom data for teacher reflection: teachers’ preferences, practices, and privacy considerations

   Yang, K; Borchers, C; Falhs, A-C; Echeverria, V; Karumbaiah, S; Rummel, N; Aleven, V (September 2024, Springer; European Conference on Technology Enhanced Learning, EC-TEL 2024)

   Ferreira_Mello, R; Rummel, N; Jivet; I; Pishtari, G; Ruipérez_Valiente, J A (Ed.)

   Teacher reflection is essential for K-12 classrooms, including effective and personalized instruction. Multimodal Learning Analytics (MMLA), integrating data from digital and physical learning environments, could support teacher reflection. Classroom data collected from sensors and TEL environments are needed to produce such analytics. These novel data collection methods pose an open challenge of how MMLA research practices can ensure alignment with teachers’ needs and concerns. This study explores K-12 teachers’ perceptions and preferences regarding MMLA analytics and data sharing. Through a mixed-method survey, we explore teachers’ (N=100) preferences for analytics that help them reflect on their teaching practices, their favored data collection modalities, and data-sharing preferences. Results indicate that teachers were most interested in student learning analytics and their interactions and ways of motivating students. However, they were also significantly less accepting of collecting students’ audio and position data compared to such data about themselves. Finally, teachers were less willing to share data about themselves than their students. Our findings contribute ethical, practical, and pedagogical considerations of MMLA analytics for teacher reflection, informing the research practices and development of MMLA within TEL. 

   more » « less
3. The Science Student Electronic Exit Ticket (SEET) System: Visualizations to Help Teachers Notice and Reflect on Classroom Inequalities

   https://doi.org/10.18608/jla.2024.8199  

   Raza, Ali; Sumner, Tamara; Penuel, William R (March 2024, Journal of Learning Analytics)

   This study examined the ways in which an equity analytics tool — the SEET system — supported middle school science teachers’ reflections on the experiences of diverse students in their classrooms. The tool provides teachers with “equity visualizations” — disaggregated classroom data by gender and race/ethnicity — designed to support teachers to notice and reflect on inequitable patterns in student participation in classroom knowledge-building activities, as well as “whole class visualizations” that enable teachers to look at participation patterns. The visualizations were based on survey data collected from students reflecting on the day’s lessons, responding to questions aligned with three theoretical constructs indicative of equitable participation in science classrooms: coherence, relevance, and contribution. The study involved 42 teachers, divided into two cohorts, participating in a two-month professional learning series. Diary studies and semi-structured interviews were used to probe teachers’ perceptions of the visualizations’ usability, usefulness, and utility for supporting their reflections on student experiences and instructional practices. A key result is that only the “equity visualizations” prompted teacher reflections on diverse student experiences. However, despite the support equity visualizations provided for this core task, the teachers consistently ranked the whole class visualizations as more usable and useful. 

   more » « less
4. How Expertise Levels Shape Preferences and Reflection Needs: Towards AI Reflection Systems for Teacher Empowerment

   https://doi.org/10.1007/978-3-032-03873-9_11  

   Falhs, Ann-Christin; Borchers, Conrad; Echeverria, Vanessa; Yang, Kexin; Rummel, Nikol; Aleven, Vincent (September 2025, Springer Nature Switzerland)

   Tammets, K; Sosnovsky, S; Ferreira_Mello, R; Pishtari, G; Nazaretsky, T (Ed.)

   The increasing use of AI systems opens up opportunities to use learner and teacher-related data for reflection, supporting instruction and professional growth. However, teacher-related data is rarely collected in technology-enhanced learning environments. Understanding teacher preferences can inform the design of AI reflection systems that align with their priorities. The current survey study explores how K-12 teachers' reflection practices and preferences for analytic data types (e.g., learning-related vs. engagement-related) vary by experience, an aspect often overlooked in prior works. Findings from our survey study with N=100 teachers in the U.S. revealed significant differences. Less experienced teachers focus on classroom behavior and prefer analytics on classroom issues, while experienced teachers prefer broader strategies to improve student learning and favor sustained data analysis. Our results reflect the need to consider teachers’ experience level by tuning the complexity and concreteness of analytic recommendations to support ongoing professionalization. 

   more » « less
5. Teacher noticing and student learning in human-AI partnered classrooms: A multimodal analysis

   Karumbaiah, S.; Borchers, C.; Falhs, A.; Holstein, K.; Rummel, N.; Aleven, V. (June 2023, International Society of the Learning Sciences)

   Blikstein, P.; Van Aalst, J.; Kizito, R.; Brennan, K. (Ed.)

   Past research shows that teacher noticing matters for student learning, but little is known about the effects of AI-based tools designed to augment teachers’ attention and sensemaking. In this paper, we investigate three multimodal measures of teacher noticing (i.e., gaze, deep dive into learning analytics in a teacher tool, and visits to individual students), gleaned from a mixed reality teacher awareness tool across ten classrooms. Our analysis suggests that of the three noticing measures, deep dive exhibited the largest association with learning gains when adjusting for students’ prior knowledge and tutor interactions. This finding may indicate that teachers identified students most in need based on the deep dive analytics and offered them support. We discuss how these multimodal measures can make the constraints and effects of teacher noticing in human-AI partnered classrooms visible. 

   more » « less