skip to main content


Title: Long-term Human Video Activity Quantification of Student Participation
Research on video activity recognition has been primarily focused on differentiating among many diverse activities defined using short video clips. In this paper, we introduce the problem of reliable video activity recognition over long videos to quantify student participation in collaborative learning environments (45 minutes to 2 hours). Video activity recognition in collaborative learning environments contains several unique challenges. We introduce participation maps that identify how and when each student performs each activity to quantify student participation. We present a family of low-parameter 3D ConvNet architectures to detect these activities. We then apply spatial clustering to identify each participant and generate student participation maps using the resulting detections. We demonstrate the effectiveness by training over about 1,000 3-second samples of typing and writing and test our results over ten video sessions of about 10 hours. In terms of activity detection, our methods achieve 80% accuracy for writing and typing that match the recognition performance of TSN, SlowFast, Slowonly, and I3D trained over the same dataset while using 1200x to 1500x fewer parameters. Beyond traditional video activity recognition methods, our video activity participation maps identify how each student participates within each group.  more » « less
Award ID(s):
1949230 1613637 1842220
NSF-PAR ID:
10310078
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
2021 Asilomar Conference on Signals, Systems, and Computers.
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Need/Motivation (e.g., goals, gaps in knowledge) The ESTEEM implemented a STEM building capacity project through students’ early access to a sustainable and innovative STEM Stepping Stones, called Micro-Internships (MI). The goal is to reap key benefits of a full-length internship and undergraduate research experiences in an abbreviated format, including access, success, degree completion, transfer, and recruiting and retaining more Latinx and underrepresented students into the STEM workforce. The MIs are designed with the goals to provide opportunities for students at a community college and HSI, with authentic STEM research and applied learning experiences (ALE), support for appropriate STEM pathway/career, preparation and confidence to succeed in STEM and engage in summer long REUs, and with improved outcomes. The MI projects are accessible early to more students and build momentum to better overcome critical obstacles to success. The MIs are shorter, flexibly scheduled throughout the year, easily accessible, and participation in multiple MI is encouraged. ESTEEM also establishes a sustainable and collaborative model, working with partners from BSCS Science Education, for MI’s mentor, training, compliance, and building capacity, with shared values and practices to maximize the improvement of student outcomes. New Knowledge (e.g., hypothesis, research questions) Research indicates that REU/internship experiences can be particularly powerful for students from Latinx and underrepresented groups in STEM. However, those experiences are difficult to access for many HSI-community college students (85% of our students hold off-campus jobs), and lack of confidence is a barrier for a majority of our students. The gap between those who can and those who cannot is the “internship access gap.” This project is at a central California Community College (CCC) and HSI, the only affordable post-secondary option in a region serving a historically underrepresented population in STEM, including 75% Hispanic, and 87% have not completed college. MI is designed to reduce inequalities inherent in the internship paradigm by providing access to professional and research skills for those underserved students. The MI has been designed to reduce barriers by offering: shorter duration (25 contact hours); flexible timing (one week to once a week over many weeks); open access/large group; and proximal location (on-campus). MI mentors participate in week-long summer workshops and ongoing monthly community of practice with the goal of co-constructing a shared vision, engaging in conversations about pedagogy and learning, and sustaining the MI program going forward. Approach (e.g., objectives/specific aims, research methodologies, and analysis) Research Question and Methodology: We want to know: How does participation in a micro-internship affect students’ interest and confidence to pursue STEM? We used a mixed-methods design triangulating quantitative Likert-style survey data with interpretive coding of open-responses to reveal themes in students’ motivations, attitudes toward STEM, and confidence. Participants: The study sampled students enrolled either part-time or full-time at the community college. Although each MI was classified within STEM, they were open to any interested student in any major. Demographically, participants self-identified as 70% Hispanic/Latinx, 13% Mixed-Race, and 42 female. Instrument: Student surveys were developed from two previously validated instruments that examine the impact of the MI intervention on student interest in STEM careers and pursuing internships/REUs. Also, the pre- and post (every e months to assess longitudinal outcomes) -surveys included relevant open response prompts. The surveys collected students’ demographics; interest, confidence, and motivation in pursuing a career in STEM; perceived obstacles; and past experiences with internships and MIs. 171 students responded to the pre-survey at the time of submission. Outcomes (e.g., preliminary findings, accomplishments to date) Because we just finished year 1, we lack at this time longitudinal data to reveal if student confidence is maintained over time and whether or not students are more likely to (i) enroll in more internships, (ii) transfer to a four-year university, or (iii) shorten the time it takes for degree attainment. For short term outcomes, students significantly Increased their confidence to continue pursuing opportunities to develop within the STEM pipeline, including full-length internships, completing STEM degrees, and applying for jobs in STEM. For example, using a 2-tailed t-test we compared means before and after the MI experience. 15 out of 16 questions that showed improvement in scores were related to student confidence to pursue STEM or perceived enjoyment of a STEM career. Finding from the free-response questions, showed that the majority of students reported enrolling in the MI to gain knowledge and experience. After the MI, 66% of students reported having gained valuable knowledge and experience, and 35% of students spoke about gaining confidence and/or momentum to pursue STEM as a career. Broader Impacts (e.g., the participation of underrepresented minorities in STEM; development of a diverse STEM workforce, enhanced infrastructure for research and education) The ESTEEM project has the potential for a transformational impact on STEM undergraduate education’s access and success for underrepresented and Latinx community college students, as well as for STEM capacity building at Hartnell College, a CCC and HSI, for students, faculty, professionals, and processes that foster research in STEM and education. Through sharing and transfer abilities of the ESTEEM model to similar institutions, the project has the potential to change the way students are served at an early and critical stage of their higher education experience at CCC, where one in every five community college student in the nation attends a CCC, over 67% of CCC students identify themselves with ethnic backgrounds that are not White, and 40 to 50% of University of California and California State University graduates in STEM started at a CCC, thus making it a key leverage point for recruiting and retaining a more diverse STEM workforce. 
    more » « less
  2. null (Ed.)
    COVID-19 has altered the landscape of teaching and learning. For those in in-service teacher education, workshops have been suspended causing programs to adapt their professional development to a virtual space to avoid indefinite postponement or cancellation. This paradigm shift in the way we conduct learning experiences creates several logistical and pedagogical challenges but also presents an important opportunity to conduct research about how learning happens in these new environments. This paper describes the approach we took to conduct research in a series of virtual workshops aimed at teaching rural elementary teachers about engineering practices and how to teach a unit from an engineering curriculum. Our work explores how engineering concepts and practices are socially constructed through interactions with teachers, students, and artifacts. This approach, called interactional ethnography has been used by the authors and others to learn about engineering teaching and learning in precollege classrooms. The approach relies on collecting data during instruction, such as video and audio recordings, interviews, and artifacts such as journal entries and photos of physical designs. Findings are triangulated by analyzing these data sources. This methodology was going to be applied in an in-person engineering education workshop for rural elementary teachers, however the pandemic forced us to conduct the workshops remotely. Teachers, working in pairs, were sent workshop supplies, and worked together during the training series that took place over Zoom over four days for four hours each session. The paper describes how we collected video and audio of teachers and the facilitators both in whole group and in breakout rooms. Class materials and submissions of photos and evaluations were managed using Google Classroom. Teachers took photos of their work and scanned written materials and submitted them all by email. Slide decks were shared by the users and their group responses were collected in real time. Workshop evaluations were collected after each meeting using Google Forms. Evaluation data suggest that the teachers were engaged by the experience, learned significantly about engineering concepts and the knowledge-producing practices of engineers, and feel confident about applying engineering activities in their classrooms. This methodology should be of interest to the membership for three distinct reasons. First, remote instruction is a reality in the near-term but will likely persist in some form. Although many of us prefer to teach in person, remote learning allows us to reach many more participants, including those living in remote and rural areas who cannot easily attend in-person sessions with engineering educators, so it benefits the field to learn how to teach effectively in this way. Second, it describes an emerging approach to engineering education research. Interactional ethnography has been applied in precollege classrooms, but this paper demonstrates how it can also be used in teacher professional development contexts. Third, based on our application of interactional ethnography to an education setting, readers will learn specifically about how to use online collaborative software and how to collect and organize data sources for research purposes. 
    more » « less
  3. Writing is generally recognized as fundamental to the formation and communication of scientific and technical knowledge to peer groups and general audiences. Often, persuasive writing is an essential attribute emphasized by industries and businesses for a successful career in STEM fields. Nevertheless, the current scenario is that students in STEM fields, with their increased demand for more specialized skills in fewer credit hours combined with a lack of emphasis on writing from engineering faculty members, make addressing this need difficult. In addition, students in engineering fields often do not value writing skills and underestimate the amount of writing they will do in their careers. Hence, it is essential to understand and quantify the level of writing skills STEM students exhibit in their technical courses so that mitigation efforts can be designed using commonly available resources to enhance this important skillset among the students, including university writing centers. A research question was posed to study this aspect of technical writing: How do STEM students at institutions conceive of writing and its role in classroom laboratories? This research was conducted at three different universities with students of varied demographics, including one which is designated as a Hispanic-serving institution, via a sequential mixed-methods design. The demography variation among the institutions includes the level of college preparation among students and the mix of ethnicity to see if there are variations among certain groups. Although the sample size is small, the goal was to establish a methodology and a preliminary outcome set that could be used in further research with larger populations. Research data in the form of reports and surveys, were collected from groups of students from four distinct campuses to ascertain the technical writing capability of each group and provide a comparison to better understand the level of intervention required. The quantitative data was collected throughout the academic year through Likert scale surveys as well as rubric-based evaluation of reports. The research design, methodology, and results of the research findings and the proposed mitigation efforts to improve student writing in STEM fields are presented in the paper. The above research was sponsored by the National Science Foundation grant: Collaborative: Research: Writing Assignment Tutor Training in STEM (WATTS), an Interdisciplinary Approach for the Enhancement of Student Writing in STEM fields. 
    more » « less
  4. Writing is generally recognized as fundamental to the formation and communication of scientific and technical knowledge to peer groups and general audiences. Often, persuasive writing is an essential attribute emphasized by industries and businesses for a successful career in STEM fields. Nevertheless, the current scenario is that students in STEM fields, with their increased demand for more specialized skills in fewer credit hours combined with a lack of emphasis on writing from engineering faculty members, make addressing this need difficult. In addition, students in engineering fields often do not value writing skills and underestimate the amount of writing they will do in their careers. Hence, it is essential to understand and quantify the level of writing skills STEM students exhibit in their technical courses so that mitigation efforts can be designed using commonly available resources to enhance this important skillset among the students, including university writing centers. A research question was posed to study this aspect of technical writing: How do STEM students at institutions conceive of writing and its role in classroom laboratories? This research was conducted at three different universities with students of varied demographics, including one which is designated as a Hispanic-serving institution, via a sequential mixed-methods design. The demography variation among the institutions includes the level of college preparation among students and the mix of ethnicity to see if there are variations among certain groups. Although the sample size is small, the goal was to establish a methodology and a preliminary outcome set that could be used in further research with larger populations. Research data in the form of reports and surveys, were collected from groups of students from four distinct campuses to ascertain the technical writing capability of each group and provide a comparison to better understand the level of intervention required. The quantitative data was collected throughout the academic year through Likert scale surveys as well as rubric-based evaluation of reports. The research design, methodology, and results of the research findings and the proposed mitigation efforts to improve student writing in STEM fields are presented in the paper. The above research was sponsored by the National Science Foundation grant: Collaborative: Research: Writing Assignment Tutor Training in STEM (WATTS), an Interdisciplinary Approach for the Enhancement of Student Writing in STEM fields. 
    more » « less
  5. Writing is generally recognized as fundamental to the formation and communication of scientific and technical knowledge to peer groups and general audiences. Often, persuasive writing is an essential attribute emphasized by industries and businesses for a successful career in STEM fields. Nevertheless, the current scenario is that students in STEM fields, with their increased demand for more specialized skills in fewer credit hours combined with a lack of emphasis on writing from engineering faculty members, make addressing this need difficult. In addition, students in engineering fields often do not value writing skills and underestimate the amount of writing they will do in their careers. Hence, it is essential to understand and quantify the level of writing skills STEM students exhibit in their technical courses so that mitigation efforts can be designed using commonly available resources to enhance this important skillset among the students, including university writing centers. A research question was posed to study this aspect of technical writing: How do STEM students at institutions conceive of writing and its role in classroom laboratories? This research was conducted at three different universities with students of varied demographics, including one which is designated as a Hispanic-serving institution, via a sequential mixed-methods design. The demography variation among the institutions includes the level of college preparation among students and the mix of ethnicity to see if there are variations among certain groups. Although the sample size is small, the goal was to establish a methodology and a preliminary outcome set that could be used in further research with larger populations. Research data in the form of reports and surveys, were collected from groups of students from four distinct campuses to ascertain the technical writing capability of each group and provide a comparison to better understand the level of intervention required. The quantitative data was collected throughout the academic year through Likert scale surveys as well as rubric-based evaluation of reports. The research design, methodology, and results of the research findings and the proposed mitigation efforts to improve student writing in STEM fields are presented in the paper. The above research was sponsored by the National Science Foundation grant: Collaborative: Research: Writing Assignment Tutor Training in STEM (WATTS), an Interdisciplinary Approach for the Enhancement of Student Writing in STEM fields. 
    more » « less