An official website of the United States government
Transcripts of teaching episodes can be effective tools to understand discourse patterns in classroom instruction. According to most educational experts, sustained classroom discourse is a critical component of equitable, engaging, and rich learning environments for students. This paper describes the TalkMoves dataset, composed of 567 human annotated K-12 mathematics lesson transcripts (including entire lessons or portions of lessons) derived from video recordings. The set of transcripts primarily includes in-person lessons with whole-class discussions and/or small group work, as well as some online lessons. All of the transcripts are human-transcribed, segmented by the speaker (teacher or student), and annotated at the sentence level for ten discursive moves based on accountable talk theory. In addition, the transcripts include utterance-level information in the form of dialogue act labels based on the Switchboard Dialog Act Corpus. The dataset can be used by educators, policymakers, and researchers to understand the nature of teacher and student discourse in K-12 math classrooms. Portions of this dataset have been used to develop the TalkMoves application, which provides teachers with automated, immediate, and actionable feedback about their mathematics instruction.
more »
« less
Automatically Measuring Question Authenticity in Real-World Classrooms
Analyzing the quality of classroom talk is central to educational research and improvement efforts. In particular, the presence of authentic teacher questions, where answers are not predetermined by the teacher, helps constitute and serves as a marker of productive classroom discourse. Further, authentic questions can be cultivated to improve teaching effectiveness and consequently student achievement. Unfortunately, current methods to measure question authenticity do not scale because they rely on human observations or coding of teacher discourse. To address this challenge, we set out to use automatic speech recognition, natural language processing, and machine learning to train computers to detect authentic questions in real-world classrooms automatically. Our methods were iteratively refined using classroom audio and human coded observational data from two sources: (a) a large archival database of text transcripts of 451 observations from 112 classrooms; and (b) a newly collected sample of 132 high-quality audio recordings from 27 classrooms, obtained under technical constraints that anticipate large-scale automated data collection and analysis. Correlations between human coded and computer-coded authenticity at the classroom level were sufficiently high (r = .602 for archival transcripts and .687 for audio recordings) to provide a valuable complement to human coding in research efforts.
more »
« less
- Award ID(s):
- 1735785
- PAR ID:
- 10066510
- Date Published:
- Journal Name:
- Educational researcher
- Volume:
- XX
- Issue:
- X
- ISSN:
- 1935-102X
- Page Range / eLocation ID:
- 1-14
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
This paper investigates the relationship between teacher and student discourse patterns, measured by accountable talk moves (Michaels & O’Connor, 2015) and the quality of mathematics instruction as measured by the Mathematical Quality of Instruction (MQI) rubric. This study uses a large public dataset of human coded MQI lesson transcripts and validated AI coding for talk moves to explore how different talk moves predict instructional quality. Results indicate that certain talk moves at certain frequencies, especially those relating to accountability to the learning community and rigorous thinking, positively correlate with higher MQI scores. Thus the nature and frequency of nuanced discourse patterns are crucial for high-quality mathematics instruction, while simple metrics like the amount of student talk have little impact.more » « less
-
Gardner, Stephanie (Ed.)Stronger metacognition, or awareness and regulation of thinking, is related to higher academic achievement. Most metacognition research has focused at the level of the individual learner. However, a few studies have shown that students working in small groups can stimulate metacognition in one another, leading to improved learning. Given the increased adoption of interactive group work in life science classrooms, there is a need to study the role of social metacognition, or the awareness and regulation of the thinking of others, in this context. Guided by the frameworks of social metacognition and evidence-based reasoning, we asked: 1) What metacognitive utterances (words, phrases, statements, or questions) do students use during small-group problem solving in an upper-division biology course? 2) Which metacognitive utterances are associated with small groups sharing higher-quality reasoning in an upper-division biology classroom? We used discourse analysis to examine transcripts from two groups of three students during breakout sessions. By coding for metacognition, we identified seven types of metacognitive utterances. By coding for reasoning, we uncovered four categories of metacognitive utterances associated with higher-quality reasoning. We offer suggestions for life science educators interested in promoting social metacognition during small-group problem solving.more » « less
-
We present the Teacher Talk Tool, which automatically analyzes classroom audio and provides formative feedback on key aspects of teachers’ classroom discourse (e.g., use of open-ended questions). The tool was designed to promote teacher learning by focusing attention and sense-making on their discourse. We conducted a feedback- response study where five English & Language Art teachers used the Teacher Talk Tool in eight classroom sessions. Teachers completed repeated-measure surveys and semi-structured interviews providing quantitative and qualitative evidence of feedback response. Results indicated that the majority of automated feedback was perceived to be accurate and prompted a high degree of reflection, focusing teachers’ attention on the measured talk constructs. This feedback also led teachers to engage in a process of sense-making, linking the measured talk features to classroom processes and contexts. However, evidence of feedback uptake was more limited. Overall, results contribute to the nascent literature on the efficacy of automated feedback on instructional practice.more » « less
-
null (Ed.)This Research Work in Progress paper presents a case study that demonstrates how a secondary school teacher with a non-STEM background identifies parallels between the engineering design process and music creation to embrace teaching an engineering course for the first time. Multiple interviews and classroom observations were open coded using a two-cycle coding approach to reveal four themes: overcoming imposter syndrome, connections between engineering and music, challenges encountered, and changes in practice. These themes highlight the processes involved in transferring a pedagogical philosophy that can inform future efforts to explore the necessary preconditions for bridging seemingly disparate and unconnected content areas. Further exploration building on these findings will inform efforts to broaden the pool of teachers capable of teaching pre-college engineering classes.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/https://doi.org/10.3102/0013189X18785613
