More Like this

1. Towards Actionable Pedagogical Feedback: A Multi-Perspective Analysis of Mathematics Teaching and Tutoring Dialogue

   https://doi.org/10.5281/zenodo.15870177  

   Naim, Jannatun; Cao, Jie; Tasneem, Fareen; Jacobs, Jennifer; Milne, Brent; Martin, James; Sumner, Tamara (July 2025, International Educational Data Mining Society)

   Mills, Caitlin; Alexandron, Giora; Taibi, Davide; Lo_Bosco, Giosuè; Paquette, Luc (Ed.)

   Effective feedback is essential for refining instructional practices in mathematics education, and researchers often turn to advanced natural language processing (NLP) models to analyze classroom dialogues from multiple perspectives. However, utterance-level discourse analysis encounters two primary challenges: (1) multi-functionality, where a single utterance may serve multiple purposes that a single tag cannot capture, and (2) the exclusion of many utterances from domain-specific discourse move classifications, leading to their omission in feedback. To address these challenges, we proposed a multi-perspective discourse analysis that integrates domain-specific talk moves with dialogue act (using the flattened multi-functional SWBD-MASL schema with 43 tags) and discourse relation (applying Segmented Discourse Representation Theory with 16 relations). Our top-down analysis framework enables a comprehensive understanding of utterances that contain talk moves, as well as utterances that do not contain talk moves. This is applied to two mathematics education datasets: TalkMoves (teaching) and SAGA22 (tutoring). Through distributional unigram analysis, sequential talk move analysis, and multi-view deep dive, we discovered meaningful discourse patterns, and revealed the vital role of utterances without talk moves, demonstrating that these utterances, far from being mere fillers, serve crucial functions in guiding, acknowledging, and structuring classroom discourse. These insights underscore the importance of incorporating discourse relations and dialogue acts into AI-assisted education systems to enhance feedback and create more responsive learning environments. Our framework may prove helpful for providing human educator feedback, but also aiding in the development of AI agents that can effectively emulate the roles of both educators and students. 

   more » « less
2. Enhancing talk moves analysis in mathematics tutoring through classroom teaching discourse

   Cao, J; Suresh, A; Jacobs, J; Clevenger, C E; Howard, A; Summer, T; Brown, C; Milne, B; Fischaber, T; Sumner, T; et al (January 2025, Association for Computational Linguistics)

   Rambow, Owen; Wanner, Leo; Apidianaki, Marianna; Al-Khalifa, Hend; Di_Eugenio, Barbara; Schockaert, Steven (Ed.)

   Human tutoring interventions play a crucial role in supporting student learning, improving academic performance, and promoting personal growth. This paper focuses on analyzing mathematics tutoring discourse using talk moves—a framework of dialogue acts grounded in Accountable Talk theory. However, scaling the collection, annotation, and analysis of extensive tutoring dialogues to develop machine learning models is a challenging and resource-intensive task. To address this, we present SAGA22, a compact dataset, and explore various modeling strategies, including dialogue context, speaker information, pretraining datasets, and further fine-tuning. By leveraging existing datasets and models designed for classroom teaching, our results demonstrate that supplementary pretraining on classroom data enhances model performance in tutoring settings, particularly when incorporating longer context and speaker information. Additionally, we conduct extensive ablation studies to underscore the challenges in talk move modeling. 

   more » « less
3. Enhancing talk moves analysis in mathematics tutoring through classroom teaching discourse

   Cao, J; Suresh, A; Jacobs, J; Clevenger, C E; Howard, A; Summer, T; Brown, C; Milne, B; Fischaber, T; Sumner, T; et al (January 2025, Association for Computational Linguistics)

   Rambow, Owen; Wanner, Owen; Apidianaki, Marianna; Al-Khalifa, Hend; Di_Eugenio, Barbara; Schockaert, Steven (Ed.)

   Human tutoring interventions play a crucial role in supporting student learning, improving academic performance, and promoting personal growth. This paper focuses on analyzing mathematics tutoring discourse using talk moves—a framework of dialogue acts grounded in Accountable Talk theory. However, scaling the collection, annotation, and analysis of extensive tutoring dialogues to develop machine learning models is a challenging and resource-intensive task. To address this, we present SAGA22, a compact dataset, and explore various modeling strategies, including dialogue context, speaker information, pretraining datasets, and further fine-tuning. By leveraging existing datasets and models designed for classroom teaching, our results demonstrate that supplementary pretraining on classroom data enhances model performance in tutoring settings, particularly when incorporating longer context and speaker information. Additionally, we conduct extensive ablation studies to underscore the challenges in talk move modeling. 

   more » « less
4. Using transformers to provide teachers with personalized feedback on their classroom discourse: The TalkMoves application

   Suresh, A.; Jacobs, J.; Lai, V.; Tan, C.; Ward, W.; Martin, J.H.; Sumner, T. (January 2021, Association for the Advancement of Artificial Intelligence)

   null (Ed.)

   TalkMoves is an innovative application designed to support K-12 mathematics teachers to reflect on, and continuously improve their instructional practices. This application combines state-of-the-art natural language processing capabilities with automated speech recognition to automatically analyze classroom recordings and provide teachers with personalized feedback on their use of specific types of discourse aimed at broadening and deepening classroom conversations about mathematics. These specific discourse strategies are referred to as “talk moves” within the mathematics education community and prior research has documented the ways in which systematic use of these discourse strategies can positively impact student engagement and learning. In this article, we describe the TalkMoves application’s cloud-based infrastructure for managing and processing classroom recordings, and its interface for providing teachers with feedback on their use of talk moves during individual teaching episodes. We present the series of model architectures we developed, and the studies we conducted, to develop our best-performing, transformer-based model (F1 = 79.3%). We also discuss several technical challenges that need to be addressed when working with real-world speech and language data from noisy K-12 classrooms. 

   more » « less
5. Using transformers to provide teachers with personalized feedback on their classroom discourse: The TalkMoves application

   Suresh, A.; Jacobs, J.; Lai, V.; Tan, C.; Ward, W.; Martin, J.H.; Sumner, T. (January 2021, Association for the Advancement of Artificial Intelligence in Education)

   null (Ed.)

   TalkMoves is an innovative application designed to support K-12 mathematics teachers to reflect on, and continuously improve their instructional practices. This application com- bines state-of-the-art natural language processing capabilities with automated speech recognition to automatically analyze classroom recordings and provide teachers with personalized feedback on their use of specific types of discourse aimed at broadening and deepening classroom conversations about mathematics. These specific discourse strategies are referred to as “talk moves” within the mathematics education com- munity and prior research has documented the ways in which systematic use of these discourse strategies can positively impact student engagement and learning. In this article, we describe the TalkMoves application’s cloud-based infrastruc- ture for managing and processing classroom recordings, and its interface for providing teachers with feedback on their use of talk moves during individual teaching episodes. We present the series of model architectures we developed, and the studies we conducted, to develop our best-performing, transformer-based model (F1 = 79.3%). We also discuss sev- eral technical challenges that need to be addressed when working with real-world speech and language data from noisy K-12 classrooms. 

   more » « less