More Like this

1. Leveraging multimodal classroom data for teacher reflection: teachers’ preferences, practices, and privacy considerations

   Yang, K; Borchers, C; Falhs, A-C; Echeverria, V; Karumbaiah, S; Rummel, N; Aleven, V (September 2024, Springer; European Conference on Technology Enhanced Learning, EC-TEL 2024)

   Ferreira_Mello, R; Rummel, N; Jivet; I; Pishtari, G; Ruipérez_Valiente, J A (Ed.)

   Teacher reflection is essential for K-12 classrooms, including effective and personalized instruction. Multimodal Learning Analytics (MMLA), integrating data from digital and physical learning environments, could support teacher reflection. Classroom data collected from sensors and TEL environments are needed to produce such analytics. These novel data collection methods pose an open challenge of how MMLA research practices can ensure alignment with teachers’ needs and concerns. This study explores K-12 teachers’ perceptions and preferences regarding MMLA analytics and data sharing. Through a mixed-method survey, we explore teachers’ (N=100) preferences for analytics that help them reflect on their teaching practices, their favored data collection modalities, and data-sharing preferences. Results indicate that teachers were most interested in student learning analytics and their interactions and ways of motivating students. However, they were also significantly less accepting of collecting students’ audio and position data compared to such data about themselves. Finally, teachers were less willing to share data about themselves than their students. Our findings contribute ethical, practical, and pedagogical considerations of MMLA analytics for teacher reflection, informing the research practices and development of MMLA within TEL. 

   more » « less
2. Multimodal analytics for collaborative teacher reflection of human-ai hybrid teaching: design opportunities and constraints

   https://doi.org/10.1007/978-3-031-42682-7_45  

   Karumbaiah, S.; Liu, P.; Maksimova, A.; De Vylder, L.; Rummel, N.; Aleven, V. (August 2023, Springer)

   Viberg, O.; Jivet, I.; Muñoz-Merino, P.; Perifanou, M.; Papathoma, T. (Ed.)

   Past research shows that teachers benefit immensely from reflecting on their classroom practices. At the same time, adaptive and artificially intelligent (AI) tutors are shown to be highly effective for students, especially when teachers are involved in supporting students’ learning. Yet, there is little research on how to support teachers to reflect on their practices around AI tutors. We posit that analytics built on multimodal data from the classroom (e.g., teacher position, student-AI interaction) would be beneficial in providing effective scaffolding and evidence for teachers’ collaborative reflection on human-AI hybrid teaching. To better understand the design opportunities and constraints of a future tool for teacher reflection, we conducted storyboarding sessions with seven in-service teachers. Our analysis revealed that certain modalities (e.g., position v. video) might be more beneficial and less constrained than others in identifying reflection-worthy moments and trends. We discuss teachers’ needs for reflection in classrooms with AI tutors and their boundaries in using multimodal analytics. 

   more » « less
3. Pair-Up: Prototyping Human-AI Co-orchestration of Dynamic Transitions between Individual and Collaborative Learning in the Classroom

   https://doi.org/10.1145/3544548.3581398  

   Yang, Kexin Bella; Echeverria, Vanessa; Lu, Zijing; Mao, Hongyu; Holstein, Kenneth; Rummel, Nikol; Aleven, Vincent (April 2023, CHI ‘23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems)

   Schmidt, A.; Väänänen, K.; Goyal, T.; Kristensson, P. O.; Peters, A.; Mueller, S.; Williamson, J. R.; Wilson, M. L. (Ed.)

   Enabling students to dynamically transition between individual and collaborative learning activities has great potential to support better learning. We explore how technology can support teachers in orchestrating dynamic transitions during class. Working with five teachers and 199 students over 22 class sessions, we conducted classroom-based prototyping of a co-orchestration technology ecosystem that supports the dynamic pairing of students working with intelligent tutoring systems. Using mixed-methods data analysis, we study the resulting observed classroom dynamics, and how teachers and students perceived and experienced dynamic transitions as supported by our technology. We discover a potential tension between teachers’ and students’ preferred level of control: students prefer more control over the dynamic transitions that teachers are hesitant to grant. Our study reveals design implications and challenges for future human-AI co-orchestration in classroom use, bringing us closer to realizing the vision of highly-personalized smart classrooms that can address the unique needs of each student. 

   more » « less
4. Reflecto: A teacher reflection tool leveraging multimodal learning and teaching analytics. Interactive Event at the 26th International Conference on Artificial Intelligence in Education, AIED 2025

   Yang, K; Zhao, Y; Borchers, C; Fang, Y; Kim, J; Butt, A; Popescu, O; Gupte, S; Chiu, A; Shen, P; et al (August 2025, OSF_preprints_EdArXiv)

   Prior research has found that teacher reflection is important for both teachers’ professional development and students’ learning, but frequent, effective reflection faces many obstacles. Could an analytics-based reflection tool help address these barriers? Reflecto is a novel teacher-facing tool designed to support teachers in reflecting (after-the-fact) on their classroom practices during sessions when students are engaged with intelligent tutoring systems. Whereas many teacher analytics tools support real-time decision-making in class, Reflecto aims to promote teacher agency through out-of-class teacher-initiated, data-driven exploration of possible trends in their classroom practice. Unlike many other analytics tools, it combines learning analytics and teaching analytics. 

   more » « less
5. Automated feedback on discourse moves: Teachers’ perceived utility of a big data tool.

   https://doi.org/10.3102/1887987  

   Scornavacco, K.; Jacobs, J.; Clevenger, C. (March 2022, Annual meeting of the American Educational Research Association)

   Using new technology to provide automated feedback on classroom discourse offers a unique opportunity for educators to engage in self-reflection on their teaching, in particular to ensure that the instructional environment is equitable and productive for all students. More information is needed about how teachers experience automated data tools, including what they perceive as relevant and helpful for their everyday teaching. This mixed-methods study explored the perceptions and engagement of 21 math teachers over two years with a big data tool that analyzes classroom recordings and generates information about their discourse practices in near real-time. Findings revealed that teachers perceived the tool as having utility, yet the specific feedback that teachers perceived as most useful changed over time. In addition, teachers who used the tool throughout both years increased their use of talk moves over time, suggesting that they were making changes due to their review of the personalized feedback. These findings speak to promising directions for the development of AI-based, big data tools that help shape teacher learning and instruction, particularly tools that have strong perceived utility. 

   more » « less