More Like this

1. Resource Aware Person Re-identification across Multiple Resolutions

   Wang, Yan; Wang, Lequn; You, Yurong; Zou, Xu; Chen, Vincent; Li, Serena; Huang, Gao; Hariharan, Bharath; Weinberger, Kilian (April 2018, CVPR 2018)

   Not all people are equally easy to identify: color statistics might be enough for some cases while others might re- quire careful reasoning about high- and low-level details. However, prevailing person re-identification(re-ID) meth- ods use one-size-fits-all high-level embeddings from deep convolutional networks for all cases. This might limit their accuracy on difficult examples or makes them needlessly ex- pensive for the easy ones. To remedy this, we present a new person re-ID model that combines effective embeddings built on multiple convolutional network layers, trained with deep-supervision. On traditional re-ID benchmarks, our method improves substantially over the previous state-of- the-art results on all five datasets that we evaluate on. We then propose two new formulations of the person re- ID problem under resource-constraints, and show how our model can be used to effectively trade off accuracy and computation in the presence of resource constraints. 

   more » « less
2. Wardrobe Model for Long Term Re-identification and Appearance Prediction

   https://doi.org/10.1109/AVSS.2018.8639157  

   Lee, Kyung Won; Sankaran, Nishant; Setlur, Srirangaraj; Napp, Nils; Govindaraju, Venu (November 2018, 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS))

   Long-term surveillance applications often involve having to re-identify individuals over several days or weeks. The task is made even more challenging with the lack of sufficient visibility of the subjects faces. We address this problem by modeling the wardrobe of individuals using discriminative features and labels extracted from their clothing information from video sequences. In contrast to previous person re-id works, we exploit that people typically own a limited amount of clothing and that knowing a person's wardrobe can be used as a soft-biometric to distinguish identities. We a) present a new dataset consisting of more than 70,000 images recorded over 30 days of 25 identities; b) model clothing features using CNNs that minimize intra-garments variations while maximizing inter-garments differences; and c) build a reference wardrobe model that captures each persons set of clothes that can be used for re-id. We show that these models open new perspectives to long-term person re-id problem using clothing information. 

   more » « less
3. Optimizing Speaker Diarization for the Classroom: Applications in Timing Student Speech and Distinguishing Teachers from Children

   https://doi.org/10.5281/zenodo.14871875  

   Wang, Jiani; Dudy, Shiran; Hu, Xinlu; Wang, Zhiyong; Southwell, Rosy; Whitehill, Jacob (January 2025, Journal of educational data mining)

   An important dimension of classroom group dynamics & collaboration is how much each person contributes to the discussion. With the goal of distinguishing teachers' speech from children's speech and measuring how much each student speaks, we have investigated how automatic speaker diarization can be built to handle real-world classroom group discussions. We examined key design considerations such as the level of granularity of speaker assignment, speech enhancement techniques, voice activity detection, and embedding assignment methods to find an effective configuration. The best speaker diarization system we found was based on the ECAPA-TDNN speaker embedding model and used Whisper automatic speech recognition to identify speech segments. The diarization error rate (DER) in challenging noisy spontaneous classroom data was around 34%, and the correlations of estimated vs. human annotations of how much each student spoke reached 0.62. The accuracy of distinguishing teachers' speech from children's speech was 69.17%. We evaluated the system for potential accuracy bias across people of different skin tones and genders and found that the accuracy did not show statistically significantly differences across either dimension. Thus, the presented diarization system has potential to benefit educational research and to provide teachers and students with useful feedback to better understand their classroom dynamics. 

   more » « less
4. Developing the systems thinking and computational thinking identification tool

   Bowers, J.; Shin, N.; Brennan, L.; Eidin, E. E.; Stephens, L.; Roderick, S. (January 2022, Proceedings of the 16th International Conference of the Learning Sciences - ICLS 2022)

   Chinn, C.; Tan, E.; Chan, C.; Kali, Y. (Ed.)

   We developed the Systems Thinking (ST) and Computational Thinking (CT) Identification Tool (ID Tool) to identify student involvement in ST and CT as they construct and revise computational models. Our ID Tool builds off the ST and CT Through Modeling Framework, emphasizing the synergistic relationship between ST and CT and demonstrating how both can be supported through computational modeling. This paper describes the process of designing and validating the ID Tool with special emphasis on the observable indicators of testing and debugging computational models. We collected 75 hours of students’ interactions with a computational modeling tool and analyzed them using the ID Tool to characterize students’ use of ST and CT when involved in modeling. The results suggest that the ID Tool has the potential to allow researchers and practitioners to identify student involvement in various aspects of ST and CT as they construct and revise computational models. 

   more » « less
5. ClassID: Enabling Student Behavior Attribution from Ambient Classroom Sensing Systems

   https://doi.org/10.1145/3659586  

   Patidar, Prasoon; Ngoon, Tricia J; Zimmerman, John; Ogan, Amy; Agarwal, Yuvraj (May 2024, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies)

   Ambient classroom sensing systems offer a scalable and non-intrusive way to find connections between instructor actions and student behaviors, creating data that can improve teaching and learning. While these systems effectively provide aggregate data, getting reliable individual student-level information is difficult due to occlusion or movements. Individual data can help in understanding equitable student participation, but it requires identifiable data or individual instrumentation. We propose ClassID, a data attribution method for within a class session and across multiple sessions of a course without these constraints. For within-session, our approach assigns unique identifiers to 98% of students with 95% accuracy. It significantly reduces multiple ID assignments compared to the baseline approach (3 vs. 167) based on our testing on data from 15 classroom sessions. For across-session attributions, our approach, combined with student attendance, shows higher precision than the state-of-the-art approach (85% vs. 44%) on three courses. Finally, we present a set of four use cases to demonstrate how individual behavior attribution can enable a rich set of learning analytics, which is not possible with aggregate data alone. 

   more » « less