More Like this

1. Multimodal analytics for collaborative teacher reflection of human-ai hybrid teaching: design opportunities and constraints

   https://doi.org/10.1007/978-3-031-42682-7_45  

   Karumbaiah, S.; Liu, P.; Maksimova, A.; De Vylder, L.; Rummel, N.; Aleven, V. (August 2023, Springer)

   Viberg, O.; Jivet, I.; Muñoz-Merino, P.; Perifanou, M.; Papathoma, T. (Ed.)

   Past research shows that teachers benefit immensely from reflecting on their classroom practices. At the same time, adaptive and artificially intelligent (AI) tutors are shown to be highly effective for students, especially when teachers are involved in supporting students’ learning. Yet, there is little research on how to support teachers to reflect on their practices around AI tutors. We posit that analytics built on multimodal data from the classroom (e.g., teacher position, student-AI interaction) would be beneficial in providing effective scaffolding and evidence for teachers’ collaborative reflection on human-AI hybrid teaching. To better understand the design opportunities and constraints of a future tool for teacher reflection, we conducted storyboarding sessions with seven in-service teachers. Our analysis revealed that certain modalities (e.g., position v. video) might be more beneficial and less constrained than others in identifying reflection-worthy moments and trends. We discuss teachers’ needs for reflection in classrooms with AI tutors and their boundaries in using multimodal analytics. 

   more » « less
2. Leveraging multimodal classroom data for teacher reflection: teachers’ preferences, practices, and privacy considerations

   Yang, K; Borchers, C; Falhs, A-C; Echeverria, V; Karumbaiah, S; Rummel, N; Aleven, V (September 2024, Springer; European Conference on Technology Enhanced Learning, EC-TEL 2024)

   Ferreira_Mello, R; Rummel, N; Jivet; I; Pishtari, G; Ruipérez_Valiente, J A (Ed.)

   Teacher reflection is essential for K-12 classrooms, including effective and personalized instruction. Multimodal Learning Analytics (MMLA), integrating data from digital and physical learning environments, could support teacher reflection. Classroom data collected from sensors and TEL environments are needed to produce such analytics. These novel data collection methods pose an open challenge of how MMLA research practices can ensure alignment with teachers’ needs and concerns. This study explores K-12 teachers’ perceptions and preferences regarding MMLA analytics and data sharing. Through a mixed-method survey, we explore teachers’ (N=100) preferences for analytics that help them reflect on their teaching practices, their favored data collection modalities, and data-sharing preferences. Results indicate that teachers were most interested in student learning analytics and their interactions and ways of motivating students. However, they were also significantly less accepting of collecting students’ audio and position data compared to such data about themselves. Finally, teachers were less willing to share data about themselves than their students. Our findings contribute ethical, practical, and pedagogical considerations of MMLA analytics for teacher reflection, informing the research practices and development of MMLA within TEL. 

   more » « less
3. The Science Student Electronic Exit Ticket (SEET) System: Visualizations to Help Teachers Notice and Reflect on Classroom Inequalities

   https://doi.org/10.18608/jla.2024.8199  

   Raza, Ali; Sumner, Tamara; Penuel, William R (March 2024, Journal of Learning Analytics)

   This study examined the ways in which an equity analytics tool — the SEET system — supported middle school science teachers’ reflections on the experiences of diverse students in their classrooms. The tool provides teachers with “equity visualizations” — disaggregated classroom data by gender and race/ethnicity — designed to support teachers to notice and reflect on inequitable patterns in student participation in classroom knowledge-building activities, as well as “whole class visualizations” that enable teachers to look at participation patterns. The visualizations were based on survey data collected from students reflecting on the day’s lessons, responding to questions aligned with three theoretical constructs indicative of equitable participation in science classrooms: coherence, relevance, and contribution. The study involved 42 teachers, divided into two cohorts, participating in a two-month professional learning series. Diary studies and semi-structured interviews were used to probe teachers’ perceptions of the visualizations’ usability, usefulness, and utility for supporting their reflections on student experiences and instructional practices. A key result is that only the “equity visualizations” prompted teacher reflections on diverse student experiences. However, despite the support equity visualizations provided for this core task, the teachers consistently ranked the whole class visualizations as more usable and useful. 

   more » « less
4. How Expertise Levels Shape Preferences and Reflection Needs: Towards AI Reflection Systems for Teacher Empowerment

   https://doi.org/10.1007/978-3-032-03873-9_11  

   Falhs, Ann-Christin; Borchers, Conrad; Echeverria, Vanessa; Yang, Kexin; Rummel, Nikol; Aleven, Vincent (September 2025, Springer Nature Switzerland)

   Tammets, K; Sosnovsky, S; Ferreira_Mello, R; Pishtari, G; Nazaretsky, T (Ed.)

   The increasing use of AI systems opens up opportunities to use learner and teacher-related data for reflection, supporting instruction and professional growth. However, teacher-related data is rarely collected in technology-enhanced learning environments. Understanding teacher preferences can inform the design of AI reflection systems that align with their priorities. The current survey study explores how K-12 teachers' reflection practices and preferences for analytic data types (e.g., learning-related vs. engagement-related) vary by experience, an aspect often overlooked in prior works. Findings from our survey study with N=100 teachers in the U.S. revealed significant differences. Less experienced teachers focus on classroom behavior and prefer analytics on classroom issues, while experienced teachers prefer broader strategies to improve student learning and favor sustained data analysis. Our results reflect the need to consider teachers’ experience level by tuning the complexity and concreteness of analytic recommendations to support ongoing professionalization. 

   more » « less
5. Technology ecosystem for orchestrating dynamic transitions between individual and collaborative AI-tutored problem solving

   https://doi.org/10.1007/978-3-031-11644-5_66  

   Yang, K.; Lu, Z.; Echeverria, V.; Sewall, J.; Lawrence, L.; Rummel, N.; & Aleven, V. (January 2022, Proceedings of the 23rd International Conference on Artificial Intelligence in Education, AIED 2023)

   Rodrigo, M.M.; Matsuda, N.; Cristea, A.I.; Dimitrova, V. (Ed.)

   It might be highly effective if students could transition dynamically between individual and collaborative learning activities, but how could teachers manage such complex classroom scenarios? Although recent work in AIED has focused on teacher tools, little is known about how to orchestrate dynamic transitions between individual and collaborative learning. We created a novel technology ecosystem that supports these dynamic transitions. The ecosystem integrates a novel teacher orchestration tool that provides monitoring support and pairing suggestions with two AI-based tutoring systems that support individual and collaborative learning, respectively. We tested the feasibility of this ecosystem in a classroom study with 5 teachers and 199 students over 22 class sessions. We found that the teachers were able to manage the dynamic transitions and valued them. The study contributes a new technology ecosystem for dynamically transitioning between individual and collaborative learning, plus insight into the orchestration functionality that makes these transitions feasible. 

   more » « less