Title: Leveraging multimodal classroom data for teacher reflection: teachers’ preferences, practices, and privacy considerations
Teacher reflection is essential for K-12 classrooms, including effective and personalized instruction. Multimodal Learning Analytics (MMLA), integrating data from digital and physical learning environments, could support teacher reflection. Classroom data collected from sensors and TEL environments are needed to produce such analytics. These novel data collection methods pose an open challenge of how MMLA research practices can ensure alignment with teachers’ needs and concerns. This study explores K-12 teachers’ perceptions and preferences regarding MMLA analytics and data sharing. Through a mixed-method survey, we explore teachers’ (N=100) preferences for analytics that help them reflect on their teaching practices, their favored data collection modalities, and data-sharing preferences. Results indicate that teachers were most interested in student learning analytics and their interactions and ways of motivating students. However, they were also significantly less accepting of collecting students’ audio and position data compared to such data about themselves. Finally, teachers were less willing to share data about themselves than their students. Our findings contribute ethical, practical, and pedagogical considerations of MMLA analytics for teacher reflection, informing the research practices and development of MMLA within TEL. more »« less
The increasing use of AI systems opens up opportunities to use learner and teacher-related data for reflection, supporting instruction and professional growth. However, teacher-related data is rarely collected in technology-enhanced learning environments. Understanding teacher preferences can inform the design of AI reflection systems that align with their priorities. The current survey study explores how K-12 teachers' reflection practices and preferences for analytic data types (e.g., learning-related vs. engagement-related) vary by experience, an aspect often overlooked in prior works. Findings from our survey study with N=100 teachers in the U.S. revealed significant differences. Less experienced teachers focus on classroom behavior and prefer analytics on classroom issues, while experienced teachers prefer broader strategies to improve student learning and favor sustained data analysis. Our results reflect the need to consider teachers’ experience level by tuning the complexity and concreteness of analytic recommendations to support ongoing professionalization.
Yang, K; Zhao, Y; Borchers, C; Fang, Y; Kim, J; Butt, A; Popescu, O; Gupte, S; Chiu, A; Shen, P; et al
(, OSF_preprints_EdArXiv)
Prior research has found that teacher reflection is important for both teachers’ professional development and students’ learning, but frequent, effective reflection faces many obstacles. Could an analytics-based reflection tool help address these barriers? Reflecto is a novel teacher-facing tool designed to support teachers in reflecting (after-the-fact) on their classroom practices during sessions when students are engaged with intelligent tutoring systems. Whereas many teacher analytics tools support real-time decision-making in class, Reflecto aims to promote teacher agency through out-of-class teacher-initiated, data-driven exploration of possible trends in their classroom practice. Unlike many other analytics tools, it combines learning analytics and teaching analytics.
Karumbaiah, S.; Liu, P.; Maksimova, A.; De Vylder, L.; Rummel, N.; Aleven, V.
(, Springer)
Viberg, O.; Jivet, I.; Muñoz-Merino, P.; Perifanou, M.; Papathoma, T.
(Ed.)
Past research shows that teachers benefit immensely from reflecting on their classroom practices. At the same time, adaptive and artificially intelligent (AI) tutors are shown to be highly effective for students, especially when teachers are involved in supporting students’ learning. Yet, there is little research on how to support teachers to reflect on their practices around AI tutors. We posit that analytics built on multimodal data from the classroom (e.g., teacher position, student-AI interaction) would be beneficial in providing effective scaffolding and evidence for teachers’ collaborative reflection on human-AI hybrid teaching. To better understand the design opportunities and constraints of a future tool for teacher reflection, we conducted storyboarding sessions with seven in-service teachers. Our analysis revealed that certain modalities (e.g., position v. video) might be more beneficial and less constrained than others in identifying reflection-worthy moments and trends. We discuss teachers’ needs for reflection in classrooms with AI tutors and their boundaries in using multimodal analytics.
Johnson, Matthew M.; Gil, Minyoung
(, ISLS Annual Conference)
Despite the recent emphasis on the importance of K-12 students engaging in engineering content and practices, there has been little research done about how teachers learn engineering practices through teacher workshops and even less on how they utilize those experiences to teach engineering in their classes. Using methods of interactional ethnography, we analyzed data from an online teacher workshop in which elementary teachers engineered solutions to a multi-criteria problem in which balancing tradeoffs was a key practice. We found that teachers tended to focus on one criterion rather than both and lacked strategies to consider balancing these tradeoffs. We also found that a second iteration afforded all groups to demonstrate learning through improvement. Implications are discussed related to the importance of a focus on balancing tradeoffs in teacher learning and on pedagogy of engineering projects.
As K-12 engineering education becomes more ubiquitous in the U.S, increased attention has been paid to preparing the heterogeneous group of in-service teachers who have taken on the challenge of teaching engineering. Standards have emerged for professional development along with research on teacher learning in engineering that call for teachers to facilitate and support engineering learning environments. Given that many teachers may not have experienced engineering practice calls have been made to engage teaches K-12 teachers in the “doing” of engineering as part of their preparation. However, there is a need for research studying more specific nature of the “doing” and the instructional implications for engaging teachers in “doing” engineering. In general, to date, limited time and constrained resources necessitate that many professional development programs for K-12 teachers to engage participants in the same engineering activities they will enact with their students. While this approach supports teachers’ familiarity with curriculum and ability to anticipate students’ ideas, there is reason to believe that these experiences may not be authentic enough to support teachers in developing a rich understanding of the “doing” of engineering. K-12 teachers are often familiar with the materials and curricular solutions, given their experiences as adults, which means that engaging in the same tasks as their students may not be challenging enough to develop their understandings about engineering. This can then be consequential for their pedagogy: In our prior work, we found that teachers’ linear conceptions of the engineering design process can limit them from recognizing and supporting student engagement in productive design practices. Research on the development of engineering design practices with adults in undergraduate and professional engineering settings has shown significant differences in how adults approach and understand problems. Therefore, we conjectured that engaging teachers in more rigorous engineering challenges designed for adult engineering novices would more readily support their developing rich understandings of the ways in which professional engineers move through the design process. We term this approach meaningful engineering for teachers, and it is informed by work in science education that highlights the importance of learning environments creating a need for learners to develop and engage in disciplinary practices. We explored this approach to teachers’ professional learning experiences in doing engineering in an online graduate program for in-service teachers in engineering education at Tufts University entitled the Teacher Engineering Education Program (teep.tufts.edu). In this exploratory study, we asked: 1. How did teachers respond to engaging in meaningful engineering for teachers in the TEEP program? 2. What did teachers identify as important things they learned about engineering content and pedagogy? This paper focuses on one theme that emerged from teachers’ reflections. Our analysis found that teachers reported that meaningful engineering supported their development of epistemic empathy (“the act of understanding and appreciating someone's cognitive and emotional experience within an epistemic activity”) as a result of their own affective experiences in doing engineering that required significant iteration as well as using novel robotic materials. We consider how epistemic empathy may be an important aspect of teacher learning in K-12 engineering education and the potential implications for designing engineering teacher education.
Yang, K, Borchers, C, Falhs, A-C, Echeverria, V, Karumbaiah, S, Rummel, N, and Aleven, V. Leveraging multimodal classroom data for teacher reflection: teachers’ preferences, practices, and privacy considerations. Retrieved from https://par.nsf.gov/biblio/10556449.
Yang, K, Borchers, C, Falhs, A-C, Echeverria, V, Karumbaiah, S, Rummel, N, and Aleven, V.
"Leveraging multimodal classroom data for teacher reflection: teachers’ preferences, practices, and privacy considerations". Country unknown/Code not available: Springer; European Conference on Technology Enhanced Learning, EC-TEL 2024. https://par.nsf.gov/biblio/10556449.
@article{osti_10556449,
place = {Country unknown/Code not available},
title = {Leveraging multimodal classroom data for teacher reflection: teachers’ preferences, practices, and privacy considerations},
url = {https://par.nsf.gov/biblio/10556449},
abstractNote = {Teacher reflection is essential for K-12 classrooms, including effective and personalized instruction. Multimodal Learning Analytics (MMLA), integrating data from digital and physical learning environments, could support teacher reflection. Classroom data collected from sensors and TEL environments are needed to produce such analytics. These novel data collection methods pose an open challenge of how MMLA research practices can ensure alignment with teachers’ needs and concerns. This study explores K-12 teachers’ perceptions and preferences regarding MMLA analytics and data sharing. Through a mixed-method survey, we explore teachers’ (N=100) preferences for analytics that help them reflect on their teaching practices, their favored data collection modalities, and data-sharing preferences. Results indicate that teachers were most interested in student learning analytics and their interactions and ways of motivating students. However, they were also significantly less accepting of collecting students’ audio and position data compared to such data about themselves. Finally, teachers were less willing to share data about themselves than their students. Our findings contribute ethical, practical, and pedagogical considerations of MMLA analytics for teacher reflection, informing the research practices and development of MMLA within TEL.},
journal = {},
publisher = {Springer; European Conference on Technology Enhanced Learning, EC-TEL 2024},
author = {Yang, K and Borchers, C and Falhs, A-C and Echeverria, V and Karumbaiah, S and Rummel, N and Aleven, V},
editor = {Ferreira_Mello, R and Rummel, N and Jivet and I and Pishtari, G and Ruipérez_Valiente, J A}
}
Warning: Leaving National Science Foundation Website
You are now leaving the National Science Foundation website to go to a non-government website.
Website:
NSF takes no responsibility for and exercises no control over the views expressed or the accuracy of
the information contained on this site. Also be aware that NSF's privacy policy does not apply to this site.
An official website of the United States government
