skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: High School English Teachers Reflect on Their Talk: A Study of Response to Automated Feedback with the Teacher Talk Tool
We present the Teacher Talk Tool, which automatically analyzes classroom audio and provides formative feedback on key aspects of teachers’ classroom discourse (e.g., use of open-ended questions). The tool was designed to promote teacher learning by focusing attention and sense-making on their discourse. We conducted a feedback- response study where five English & Language Art teachers used the Teacher Talk Tool in eight classroom sessions. Teachers completed repeated-measure surveys and semi-structured interviews providing quantitative and qualitative evidence of feedback response. Results indicated that the majority of automated feedback was perceived to be accurate and prompted a high degree of reflection, focusing teachers’ attention on the measured talk constructs. This feedback also led teachers to engage in a process of sense-making, linking the measured talk features to classroom processes and contexts. However, evidence of feedback uptake was more limited. Overall, results contribute to the nascent literature on the efficacy of automated feedback on instructional practice.  more » « less
Award ID(s):
2326169
PAR ID:
10539231
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
https://doi.org/10.1007/s40593-024-00417-x
Date Published:
Journal Name:
International journal of artificial intelligence in education
ISSN:
1560-4306
Subject(s) / Keyword(s):
Instructional observation Teacher learning Classroom discourse Automated feedback
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Using new technology to provide automated feedback on classroom discourse offers a unique opportunity for educators to engage in self-reflection on their teaching, in particular to ensure that the instructional environment is equitable and productive for all students. More information is needed about how teachers experience automated data tools, including what they perceive as relevant and helpful for their everyday teaching. This mixed-methods study explored the perceptions and engagement of 21 math teachers over two years with a big data tool that analyzes classroom recordings and generates information about their discourse practices in near real-time. Findings revealed that teachers perceived the tool as having utility, yet the specific feedback that teachers perceived as most useful changed over time. In addition, teachers who used the tool throughout both years increased their use of talk moves over time, suggesting that they were making changes due to their review of the personalized feedback. These findings speak to promising directions for the development of AI-based, big data tools that help shape teacher learning and instruction, particularly tools that have strong perceived utility. 
    more » « less
  2. null (Ed.)
    One way to support teachers' learning to facilitate the recent reform vision (NRC, 2012) in their classrooms is through professional development (PD). We explored a biology teacher’s (Monica) sensemaking during the PD that focused on facilitating productive science classroom discourse to understand her responses to the PD in terms of teaching science by engaging students in productive talk in science classrooms. Using both video and interview data, we analyzed the process of her sensemaking about facilitating (productive) talk during the PD and the meaning she was making of productive talk. Our analysis indicated that Monica participated in sensemaking mostly about her students' participation in talk. Throughout the PD conversations, she rarely focused on what she could do (or could have done) to facilitate student talk without the PD facilitators' pressing. This is supported by our analysis of the interviews with Monica, which showed that the sense that she was making about productive talk mostly focuses on students' contributions to the talk and their accountability to reasoning, scientific knowledge, and sensemaking. These findings provide implications for facilitating teachers’ sensemaking around new instructional practices and reforms within PD contexts. 
    more » « less
  3. As part of a larger study focused on supporting high school biology teachers' use of productive science talk, this study compares the use of two different observation protocols, the RTOP and the IQA-SOR. Reviewing a year-long data set of video observations collected from classrooms of teachers participating in the larger professional development study, the two validated instruments produced significantly correlated scores of different scales based on the unique structure of each tool. We posit this demonstrates that both instruments can be useful for analyzing classroom instruction intended to emphasize productive science talk. However, the instruments do possess unique structural and theoretical qualities that warrant this study to understand the insights afforded by each. The similarities and differences emerging from each are explored in the presentation and how they impact the analyses. These considerations can be helpful for scholars who research in-service teacher learning as classroom implementation and impact on student learning activities are general outcomes that most professional development research endeavors to explore. Further, considerations of what a particular observation protocols’ foci include will be necessary so that continued research on teacher learning works to make science learning through discourse accessible to all learners. 
    more » « less
  4. null (Ed.)
    Inclusion in mathematics education is strongly tied to discourse rich classrooms, where students ideas play a central role. Talk moves are specific discursive practices that promote inclusive and equitable participation in classroom discussions. This paper describes the development of the TalkMoves application, which provides teachers with detailed feedback on their usage of talk moves based on accountable talk theory. Building on our recent work to automate the classification of teacher talk moves, we have expanded the application to also include feedback on a set of student talk moves. We present results from several deep learning models trained to classify student sentences into student talk moves with performance up to 73% F1. The classroom data used for training these models were collected from multiple sources that were pre-processed and annotated by highly reliable experts. We validated the performance of the model on an out-of-sample dataset which included 166 classroom transcripts collected from teachers piloting the application. 
    more » « less
  5. Within the science education reform movement, there have been long standing calls initiated to attend to equity in the science classroom. These calls are sought to de-settle and advance the broad strokes of “equity for all” into deeper, more meaningful actions, considering the way we view equity and how equitable practices unfold in the classroom. Productive science discourse or productive science talk is just one instructional practice used and discussed which leverages students as sensemakers. This study seeks to better understand productive science talk as a practice of equitation instruction. In examining Ms. Savannah’s practice, a high school biology teacher, two major findings emerged around the use of productive talk: (1) pattern of moves to leverage student ideas and (2) timing of moves to stimulate interest or motivation. These talk moves and timing gave insight into talk as both having the ability to hinder and foster student ideas and provide an initial “on-ramp” for students’ voice to be heard, taken up and have accountability in the classroom. This work continues to sustain a call toward attention to equity and a need to evaluate the equity-aligned practices that are fore-fronted in PDs and workshops. 
    more » « less