skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Instructional Activity Recognition Using A Transformer Network with Multi-Semantic Attention
Instructional activity recognition is an analytical tool for the observation of classroom education. One of the primary challenges in this domain is dealing with the intri- cate and heterogeneous interactions between teachers, students, and instructional objects. To address these complex dynamics, we present an innovative activity recognition pipeline designed explicitly for instructional videos, leveraging a multi-semantic attention mechanism. Our novel pipeline uses a transformer network that incorporates several types of instructional seman- tic attention, including teacher-to-students, students-to-students, teacher-to-object, and students-to-object relationships. This com- prehensive approach allows us to classify various interactive activity labels effectively. The effectiveness of our proposed algo- rithm is demonstrated through its evaluation on our annotated instructional activity dataset.  more » « less
Award ID(s):
2000487 2322993
PAR ID:
10523763
Author(s) / Creator(s):
; ; ;
Editor(s):
Korban, Matthew; Acton, Scott T; Youngs, Peter; Foster, Jonathan
Publisher / Repository:
IEEE
Date Published:
ISBN:
979-8-3503-6011-0
Page Range / eLocation ID:
113 to 116
Subject(s) / Keyword(s):
Not applicable
Format(s):
Medium: X
Location:
Santa Fe, NM, USA
Sponsoring Org:
National Science Foundation
More Like this
  1. VR displays (HMDs) with embedded eye trackers could enable better teacher-guided VR applications since eye tracking could provide insights into student’s activities and behavior patterns. We present several techniques to visualize eye-gaze data of the students to help a teacher gauge student attention level. A teacher could then better guide students to focus on the object of interest in the VR environment if their attention drifts and they get distracted or confused. 
    more » « less
  2. Teachers' efforts to support students, both academically and socially, can play a role in how high school students productively engage with mathematics in the moment. To examine the connection between teacher support and student engagement, we conducted an exploratory mixed-methods study combining data from 20 high school classroom observations with student self-reports taken during the observed activity. Our findings indicate that when teachers provide academic support to their students during a lesson, they are also likely to provide social support. Higher teacher support of both kinds correlates with higher student self-efficacy, as well as social and cognitive engagement. Investigating relationships between observations of teaching and students' self-reports of engagement in-the-moment is a potentially revealing approach for uncovering engaging instructional strategies in secondary mathematics classrooms. 
    more » « less
  3. With the growing prevalence of AI, the need for K-12 AI education is becoming more crucial, which is prompting active research in developing engaging and age-appropriate AI learning activities. Efforts are underway, such as those by the AI4K12 initiative, to establish guidelines for organizing K- 12 AI education; however, effective instructional resources are needed by educators. In this paper, we describe our work to design, develop, and implement an unplugged activity centered on facial recognition technology for middle school students. Facial recognition is integrated into a wide range of applications throughout daily life, which makes it a familiar and engaging tool for students and an effective medium for conveying AI concepts. Our unplugged activity, “Guess Whose Face,” is designed as a board game that focuses on Representation and Reasoning from AI4K12’s 5 Big Ideas in AI. The game is crafted to enable students to develop AI competencies naturally through physical interaction. In the game, one student uses tracing paper to extract facial features from a familiar face shown on a card, such as a cartoon character or celebrity, and then other students try to guess the identity of the hidden face. We discuss details of the game, its iterative refinement, and initial findings from piloting the activity during a summer camp for rural middle school students. 
    more » « less
  4. null (Ed.)
    This work explores epistemological framing dynamics in a middle school biology classroom and how such dynamics shape student engagement and learning opportunities. Our data sources include student and teacher interviews, classroom videos of three multi-day lessons with a focus on argumentation, and work products collected across one academic year. Our analysis reveals that while the teacher made room for students to generate and negotiate ideas, brief but influential moves emphasizing single correct answers undermined students’ sensemaking. These instructional moves, while only occupying a small amount of instructional time, framed students’ sensemaking efforts not as a process to seek the strongest explanation from a number of possibilities, but rather to wait for the correct explanation to be revealed from an authority. 
    more » « less
  5. Recent educational reforms conceptualize science classrooms as spaces where students engage in Science-as-Practice to develop deep understandings of scientific phenomena. When students engage in Science-as-Practice they are constructing explanations, arguing from evidence, and evaluating and communicating information to develop scientific knowledge (NGSS Lead States, 2013). This process of learning requires a focus on productive science talk in which students grapple with and socially negotiate their ideas (Kelly, 2014) through interactions involving talk, joint attention, and shared activity aimed at building, negotiating, and refining new understandings of phenomena and relevant science concepts (Ford, 2015; Michaels & O’Connor, 2012). Productive talk requires the ‘nimble’ involvement of the teacher to help students productively contribute their ideas to the class and use them as resources to drive instructional activities supporting the development and refinement of more sophisticated scientific understandings (Christodoulou & Osborne, 2014; González‐Howard & McNeill, 2020). 
    more » « less