skip to main content


Title: Toward an Automatic Speech Classifier for the Teacher
Our system classifies audio from microphones worn by the teacher in order to determine (1) whether the teacher is addressing the whole class or talking to individuals or groups of students. In the latter case, it determines (2) whether the teacher is giving formative feedback, giving corrective feedback, chatting socially, or addressing administrative or workflow concerns. This paper reports the initial accuracy of this system against human coding of middle school math classroom behavior. We also compared audio collected through professional hardware versus more accessible alternatives.  more » « less
Award ID(s):
1840051
NSF-PAR ID:
10185621
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Artificial Intelligence in Education, AIED 2020
Page Range / eLocation ID:
279-284
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Speech and language development are early indicators of overall analytical and learning ability in children. The preschool classroom is a rich language environment for monitoring and ensuring growth in young children by measuring their vocal interactions with both teachers and classmates. Early childhood researchers recognize the importance in analyzing naturalistic vs. controlled lab recordings to measure both quality and quantity of child interactions. Recently, large language model-based speech technologies have performed well on conversational speech recognition. In this regard, we assess performance of such models on the wide dynamic scenario of early childhood classroom settings. This study investigates an alternate Deep Learning-based Teacher-Student learning solution for recognizing adult speech within preschool interactions. Our proposed adapted model achieves the best F1-score for recognizing most frequent 400 words on test sets for both classrooms. Additionally, F1-scores for alternate word groups provides a breakdown of performance across relevant language-based word-categories. The study demonstrates the prospects of addressing educational assessment needs through communication audio stream analysis, while maintaining both security and privacy of all children and adults. The resulting child communication metrics from this study can also be used for broad-based feedback for teachers. 
    more » « less
  2. Researchers have long theorized that characteristics of education systems impact both perceived and experienced corruption in public schools. However, due to insufficient cross-national survey data with measures on corruption in education and unassembled yet publicly available institutional data, there are few empirical tests of this theory. This article provides the rare direct test of the relationship between corruption in European public schools and three education system factors: government expenditure on education, education staff compensation, and teacher workload (pupil–teacher ratio). With a newly constructed harmonized data set for European countries, and controlling for national economic factors and individual characteristics, results of multilevel analyses suggest partial support for the theory that specific institutional characteristics of education systems impact public school corruption. The theorized institutional factors have different effects that depend on whether we examine bribe-giving experience or corruption perception. Results show that bribe-giving experience in public schools of Europe is weakly yet significantly related to education staff compensation. For corruption perception, low levels of government expenditure on education and a lopsided pupil–teacher ratio (too few teachers per student) increase the probability that people view corruption as prevalent. 
    more » « less
  3. When President Obama unveiled his plan to give all students in America the opportunity to learn computer science [1], discussions about computational thinking (CT) began in earnest in many organizations across a wide range of disciplines. However, Jeannette Wing stated the importance of CT for everyone a decade earlier in her landmark essay [2]. In recent years, several people and organizations have posted their own definition of CT, which presents a challenge in being able to assess CT understanding and awareness in people. In an effort to build consensus on how to best assess CT, the authors are developing a web-based tool that will enable CT experts globally to populate, review and rate questions that address various attributes of CT. Teaching Engineering Concepts to Harness Future Innovators and Technologists (TECHFIT) is an NSF-funded project that is examining the impact of the TECHFIT intervention based on the educational program’s delivery context. The CT Assessment System is being developed for TECHFIT as a standard way for teacher participants to gauge CT understanding in their students. It has been designed as a functional, web-based tool that supports management of the CT assessment questions database and giving different levels of access to various stakeholders, including the TECHFIT project team and academicians all over the world. The CT Assessment System includes features to enable authorized users to review, insert, and update a variety of questions in different formats. The level of access to this system is determined by the roles/permissions granted by the administrator. It also enables users to have the ability to rate the questions. The ratings are then aggregated to yield an overall rating value. The CT Assessment system has the capability to provide a clean, authentic and acceptable way to assess CT abilities via a common platform across the world. Attendees of the paper presentation will be invited to sign up and explore this tool to provide feedback for improvement of the tool. 
    more » « less
  4. Speech and language development are early indicators of overall analytical and learning ability in children. The preschool classroom is a rich language environment for monitoring and ensuring growth in young children by measuring their vocal interactions with teachers and classmates. Early childhood researchers are naturally interested in analyzing naturalistic vs. controlled lab recordings to measure both quality and quantity of such interactions. Unfortunately, present-day speech technologies are not capable of addressing the wide dynamic scenario of early childhood classroom settings. Due to the diversity of acoustic events/conditions in such daylong audio streams, automated speaker diarization technology would need to be advanced to address this challenging domain for segmenting audio as well as information extraction. This study investigates an alternate Deep Learning-based diarization solution for segmenting classroom interactions of 3-5 year old children with teachers. In this context, the focus on speech-type diarization which classifies speech segments as being either from adults or children partitioned across multiple classrooms. Our proposed ResNet model achieves a best F1-score of ∼78.0% on data from two classrooms, based on dev and test sets of each classroom. It is utilized with Automatic Speech Recognition-based resegmentation modules to perform child-adult diarization. Additionally, F1-scores are obtained for individual segments with corresponding speaker tags (e.g., adult vs. child), which provide knowledge for educators on child engagement through naturalistic communications. The study demonstrates the prospects of addressing educational assessment needs through communication audio stream analysis, while maintaining both security and privacy of all children and adults. The resulting child communication metrics have been used for broad-based feedback for teachers with the help of visualizations. 
    more » « less
  5. Speech and language development are early indicators of overall analytical and learning ability in children. The preschool classroom is a rich language environment for monitoring and ensuring growth in young children by measuring their vocal interactions with teachers and classmates. Early childhood researchers are naturally interested in analyzing naturalistic vs. controlled lab recordings to measure both quality and quantity of such interactions. Unfortunately, present-day speech technologies are not capable of addressing the wide dynamic scenario of early childhood classroom settings. Due to the diversity of acoustic events/conditions in such daylong audio streams, automated speaker diarization technology would need to be advanced to address this challenging domain for segmenting audio as well as information extraction. This study investigates an alternate Deep Learning-based diarization solution for segmenting classroom interactions of 3-5 year old children with teachers. In this context, the focus on speech-type diarization which classifies speech segments as being either from adults or children partitioned across multiple classrooms. Our proposed ResNet model achieves a best F1-score of ∼71.0% on data from two classrooms, based on dev and test sets of each classroom. Additionally, F1-scores are obtained for individual segments with corresponding speaker tags (e.g., adult vs. child), which provide knowledge for educators on child engagement through naturalistic communications. The study demonstrates the prospects of addressing educational assessment needs through communication audio stream analysis, while maintaining both security and privacy of all children and adults. The resulting child communication metrics have been used for broad-based feedback for teachers with the help of visualizations. 
    more » « less