Search for: All records

Artificial intelligence (AI) can be used to classify instruction-related activities from classroom videos. These AI models, however, are dependent on datasets that are used to train the model to recognize patterns and make predictions. Imbalances in datasets used for training—such as imbalances in the domain of mathematics featured in videos of classroom instruction—may bias a model’s performance, sometimes in unforeseen ways. In this study, we investigate whether an imbalanced training dataset with a disproportionate number of video recordings of lessons focused on Number and Operations and Algebra in elementary mathematics classrooms yielded differences in a model’s performance in other mathematical content domains. We analyze an AI model’s classification of 24 instructional activities and found a notable and unanticipated difference in the model’s performance for one of the mathematical content domains. 

Previous studies have shown that artificial intelligence can be used to classify instruction-related activities in classroom videos. The automated classi- fication of human activities, however, is vulnerable to biases in which the model performs substantially better or worse for different people groups. Although algo- rithmic bias has been highlighted as an important area for research in artificial intelligence in education, there have been few studies that empirically investigate potential bias in instruction-related activity recognition systems. In this paper, we report on an investigation of potential racial and skin tone biases in the automated classification of teachers’ activities in classroom videos. We examine whether a neural network’s classification of teachers’ activities differs with respect to teacher race and skin tone and whether differently balanced training datasets affect the performance of the neural network. Our results indicate that, under ordinary class- room lighting conditions, the neural network performs equally well regardless of teacher race or skin tone. Furthermore, our results suggest the balance of the training dataset with respect to teacher skin tone and race has a small—but not necessarily positive—effect on the neural network’s performance. Our study, how- ever, also suggests the importance of quality lighting for accurate classification of teacher-related instructional activities for teachers of color. We conclude with a discussion of our mixed findings, the limitations of our study, and potential directions for future research.