skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: The TalkMoves dataset: K-12 mathematics lesson transcripts annotated for teacher and student discursive moves.
Transcripts of teaching episodes can be effective tools to understand discourse patterns in classroom instruction. According to most educational experts, sustained classroom discourse is a critical component of equitable, engaging, and rich learning environments for students. This paper describes the TalkMoves dataset, composed of 567 human annotated K-12 mathematics lesson transcripts (including entire lessons or portions of lessons) derived from video recordings. The set of transcripts primarily includes in-person lessons with whole-class discussions and/or small group work, as well as some online lessons. All of the transcripts are human-transcribed, segmented by the speaker (teacher or student), and annotated at the sentence level for ten discursive moves based on accountable talk theory. In addition, the transcripts include utterance-level information in the form of dialogue act labels based on the Switchboard Dialog Act Corpus. The dataset can be used by educators, policymakers, and researchers to understand the nature of teacher and student discourse in K-12 math classrooms. Portions of this dataset have been used to develop the TalkMoves application, which provides teachers with automated, immediate, and actionable feedback about their mathematics instruction.  more » « less
Award ID(s):
1837986
PAR ID:
10387284
Author(s) / Creator(s):
; ; ; ; ;
Date Published:
Journal Name:
Proceedings of the 13th Conference on Language Resources and Evaluation
Page Range / eLocation ID:
4654-4662
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; (, International Conference on Artificial Intelligence in Education)
    null (Ed.)
    Inclusion in mathematics education is strongly tied to discourse rich classrooms, where students ideas play a central role. Talk moves are specific discursive practices that promote inclusive and equitable participation in classroom discussions. This paper describes the development of the TalkMoves application, which provides teachers with detailed feedback on their usage of talk moves based on accountable talk theory. Building on our recent work to automate the classification of teacher talk moves, we have expanded the application to also include feedback on a set of student talk moves. We present results from several deep learning models trained to classify student sentences into student talk moves with performance up to 73% F1. The classroom data used for training these models were collected from multiple sources that were pre-processed and annotated by highly reliable experts. We validated the performance of the model on an out-of-sample dataset which included 166 classroom transcripts collected from teachers piloting the application. 
    more » « less
  2. ; ; ; ; ; ; (, Association for the Advancement of Artificial Intelligence)
    null (Ed.)
    TalkMoves is an innovative application designed to support K-12 mathematics teachers to reflect on, and continuously improve their instructional practices. This application combines state-of-the-art natural language processing capabilities with automated speech recognition to automatically analyze classroom recordings and provide teachers with personalized feedback on their use of specific types of discourse aimed at broadening and deepening classroom conversations about mathematics. These specific discourse strategies are referred to as “talk moves” within the mathematics education community and prior research has documented the ways in which systematic use of these discourse strategies can positively impact student engagement and learning. In this article, we describe the TalkMoves application’s cloud-based infrastructure for managing and processing classroom recordings, and its interface for providing teachers with feedback on their use of talk moves during individual teaching episodes. We present the series of model architectures we developed, and the studies we conducted, to develop our best-performing, transformer-based model (F1 = 79.3%). We also discuss several technical challenges that need to be addressed when working with real-world speech and language data from noisy K-12 classrooms. 
    more » « less
  3. ; ; ; ; ; ; (, Association for the Advancement of Artificial Intelligence in Education)
    null (Ed.)
    TalkMoves is an innovative application designed to support K-12 mathematics teachers to reflect on, and continuously improve their instructional practices. This application com- bines state-of-the-art natural language processing capabilities with automated speech recognition to automatically analyze classroom recordings and provide teachers with personalized feedback on their use of specific types of discourse aimed at broadening and deepening classroom conversations about mathematics. These specific discourse strategies are referred to as “talk moves” within the mathematics education com- munity and prior research has documented the ways in which systematic use of these discourse strategies can positively impact student engagement and learning. In this article, we describe the TalkMoves application’s cloud-based infrastruc- ture for managing and processing classroom recordings, and its interface for providing teachers with feedback on their use of talk moves during individual teaching episodes. We present the series of model architectures we developed, and the studies we conducted, to develop our best-performing, transformer-based model (F1 = 79.3%). We also discuss sev- eral technical challenges that need to be addressed when working with real-world speech and language data from noisy K-12 classrooms. 
    more » « less
  4. ; ; ; ; ; ; ; ; (, Education Sciences)
    Meaningful discourse in the mathematics classroom involves creating a learning community that empowers students to articulate their reasoning and make sense of the contributions of their peers while advancing the learning of mathematics for the entire class. This study assessed the degree to which secondary mathematics (Grades 7–12) teacher candidates incorporated discourse into their lessons and the factors influencing their decisions. An explanatory mixed methods design was used, in which data were collected sequentially. Lesson videos were analyzed, followed by interviews of teacher candidates with high-discourse lessons. This study found that participants showed significant growth in their use of effective teaching practices from the beginning of the semester to the end of the semester, including mathematics discourse. The interviews revealed four contributing factors: intentional effort, learning experiences, professional relationships, and pedagogical knowledge. Understanding the experiences of teacher candidates during their own learning of mathematics, their mathematical identities, and beliefs about mathematics could help generate knowledge regarding the implementation of mathematics discourse and other reform-based practices in teacher instruction. 
    more » « less
  5. ; ; ; (, American Educational Research Association)
    This paper investigates the relationship between teacher and student discourse patterns, measured by accountable talk moves (Michaels & O’Connor, 2015) and the quality of mathematics instruction as measured by the Mathematical Quality of Instruction (MQI) rubric. This study uses a large public dataset of human coded MQI lesson transcripts and validated AI coding for talk moves to explore how different talk moves predict instructional quality. Results indicate that certain talk moves at certain frequencies, especially those relating to accountability to the learning community and rigorous thinking, positively correlate with higher MQI scores. Thus the nature and frequency of nuanced discourse patterns are crucial for high-quality mathematics instruction, while simple metrics like the amount of student talk have little impact. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email