More Like this

1. Addressing imbalance: Evaluating whether instructional activities classifications differ by mathematics content domains.

   Drimalla, J; Foster, J (November 2025, North_American_Chapter_of_the_International_Group_for_the_Psychology_of_Mathematics_Education)

   Artificial intelligence (AI) can be used to classify instruction-related activities from classroom videos. These AI models, however, are dependent on datasets that are used to train the model to recognize patterns and make predictions. Imbalances in datasets used for training—such as imbalances in the domain of mathematics featured in videos of classroom instruction—may bias a model’s performance, sometimes in unforeseen ways. In this study, we investigate whether an imbalanced training dataset with a disproportionate number of video recordings of lessons focused on Number and Operations and Algebra in elementary mathematics classrooms yielded differences in a model’s performance in other mathematical content domains. We analyze an AI model’s classification of 24 instructional activities and found a notable and unanticipated difference in the model’s performance for one of the mathematical content domains. 

   more » « less
2. Investigating racial and skin tone biases in automated classification of teachers’ activities in classroom videos.

   Drimalla, J; Korban, M; Foster, J (July 2025, Springer Nature Switzerland)

   Previous studies have shown that artificial intelligence can be used to classify instruction-related activities in classroom videos. The automated classi- fication of human activities, however, is vulnerable to biases in which the model performs substantially better or worse for different people groups. Although algo- rithmic bias has been highlighted as an important area for research in artificial intelligence in education, there have been few studies that empirically investigate potential bias in instruction-related activity recognition systems. In this paper, we report on an investigation of potential racial and skin tone biases in the automated classification of teachers’ activities in classroom videos. We examine whether a neural network’s classification of teachers’ activities differs with respect to teacher race and skin tone and whether differently balanced training datasets affect the performance of the neural network. Our results indicate that, under ordinary class- room lighting conditions, the neural network performs equally well regardless of teacher race or skin tone. Furthermore, our results suggest the balance of the training dataset with respect to teacher skin tone and race has a small—but not necessarily positive—effect on the neural network’s performance. Our study, how- ever, also suggests the importance of quality lighting for accurate classification of teacher-related instructional activities for teachers of color. We conclude with a discussion of our mixed findings, the limitations of our study, and potential directions for future research. 

   more » « less
3. ST-360: Spatial–Temporal Filtering-Based Low-Latency 360-Degree Video Analytics Framework

   https://doi.org/10.1145/3694685  

   Li, Jiaxi; Liao, Jingwei; Chen, Bo; Nguyen, Anh; Tiwari, Aditi; Zhou, Qian; Yan, Zhisheng; Nahrstedt, Klara (September 2024, ACM Transactions on Multimedia Computing, Communications, and Applications)

   Recent advances in computer vision algorithms and video streaming technologies have facilitated the development of edge-server-based video analytics systems, enabling them to process sophisticated real-world tasks, such as traffic surveillance and workspace monitoring. Meanwhile, due to their omnidirectional recording capability, 360-degree cameras have been proposed to replace traditional cameras in video analytics systems to offer enhanced situational awareness. Yet, we found that providing an efficient 360-degree video analytics framework is a non-trivial task. Due to the higher resolution and geometric distortion in 360-degree videos, existing video analytics pipelines fail to meet the performance requirements for end-to-end latency and query accuracy. To address these challenges, we introduce the innovative ST-360 framework specifically designed for 360-degree video analytics. This framework features a spatial-temporal filtering algorithm that optimizes both data transmission and computational workloads. Evaluation of the ST-360 framework on a unique dataset of 360-degree first-responders videos reveals that it yields accurate query results with a 50% reduction in end-to-end latency compared to state-of-the-art methods. 

   more » « less
4. Automated classification of activities in classroom videos.

   https://doi.org/10.1016/j.caeai.2024.100207  

   Foster, Jonathan K; Korban, Matthew; Youngs, Peter; Watson, Ginger S; Acton, Scott T (June 2024, Computers and Education: Artificial Intelligence)

   Hwang, Gwo-Jen; Xie, Haoran; Wah, Benjamin; Gasevic, Dragan (Ed.)

   Classroom videos are a common source of data for educational researchers studying classroom interactions as well as a resource for teacher education and professional development. Over the last several decades emerging technologies have been applied to classroom videos to record, transcribe, and analyze classroom interactions. With the rise of machine learning, we report on the development and validation of neural networks to classify instructional activities using video signals, without analyzing speech or audio features, from a large corpus of nearly 250 h of classroom videos from elementary mathematics and English language arts instruction. Results indicated that the neural networks performed fairly-well in detecting instructional activities, at diverse levels of complexity, as compared to human raters. For instance, one neural network achieved over 80% accuracy in detecting four common activity types: whole class activity, small group activity, individual activity, and transition. An issue that was not addressed in this study was whether the fine-grained and agnostic instructional activities detected by the neural networks could scale up to supply information about features of instructional quality. Future applications of these neural networks may enable more efficient cataloguing and analysis of classroom videos at scale and the generation of fine-grained data about the classroom environment to inform potential implications for teaching and learning. 

   more » « less
5. Profiling-free Configuration Adaptation and Latency-Aware Resource Scheduling for Video Analytics

   https://doi.org/10.1109/BigData55660.2022.10020573  

   Zhou, Tian; Wu, Fubao; Gao, Lixin (December 2022, 2022 IEEE International Conference on Big Data (Big Data))

   With increasingly deployed cameras and the rapid advances of Computer Vision, large-scale live video analytics becomes feasible. However, analyzing videos is compute-intensive. In addition, live video analytics needs to be performed in real time. In this paper, we design an edge server system for live video analytics. We propose to perform configuration adaptation without profiling video online. We select configurations with a prediction model based on object movement features. In addition, we reduce the latency through resource orchestration on video analytics servers. The key idea of resource orchestration is to batch inference tasks that use the same CNN model, and schedule tasks based on a priority value that estimates their impact on the total latency. We evaluate our system with two video analytic applications, road traffic monitoring and pose detection. The experimental results show that our profiling-free adaptation reduces the workload by 80% of the state-of-the-art adaptation without lowering the accuracy. The average serving latency is reduced by up to 95% comparing with the profiling-based adaptation. 

   more » « less