Accurate modeling of student knowledge is essential for large-scale online learning systems that are increasingly used for student training. Knowledge tracing aims to model student knowledge state given the student's sequence of learning activities. Modern Knowledge tracing (KT) is usually formulated as a supervised sequence learning problem to predict students' future practice performance according to their past observed practice scores by summarizing student knowledge state as a set of evolving hidden variables. Because of this formulation, many current KT solutions are not fit for modeling student learning from non-assessed learning activities with no explicit feedback or score observation (e.g., watching video lectures that are not graded). Additionally, these models cannot explicitly represent the dynamics of knowledge transfer among different learning activities, particularly between the assessed (e.g., quizzes) and non-assessed (e.g., video lectures) learning activities. In this paper, we propose Transition-Aware Multi-activity Knowledge Tracing (TAMKOT), which models knowledge transfer between learning materials, in addition to student knowledge, when students transition between and within assessed and non-assessed learning materials. TAMKOT is formulated as a deep recurrent multi-activity learning model that explicitly learns knowledge transfer by activating and learning a set of knowledge transfer matrices, one for each transition type between student activities. Accordingly, our model allows for representing each material type in a different yet transferrable latent space while maintaining student knowledge in a shared space. We evaluate our model on three real-world publicly available datasets and demonstrate TAMKOT's capability in predicting student performance and modeling knowledge transfer.
more »
« less
Graph-enhanced multi-activity knowledge tracing
Knowledge tracing (KT), or modeling student knowledge state given their past activity sequence, is one of the essential tasks in online education systems. Research has demonstrated that students benefit from both assessed (e.g., solving problems, which can be graded) and non-assessed learning activities (e.g., watching video lectures, which cannot be graded), and thus, modeling student knowledge from multiple types of activities with knowledge transfer between them is crucial. However, current approaches to multi-activity knowledge tracing cannot capture coarse-grained between-type associations and are primarily evaluated by predicting student performance on upcoming assessed activities (labeled data). Therefore, they are inadequate in incorporating signals from non-assessed activities (unlabeled data). We propose Graph-enhanced Multi-activity Knowledge Tracing (GMKT) that addresses these challenges by jointly learning a fine-grained recurrent memory-augmented student knowledge model and a coarse-grained graph neural network. In GMKT, we formulate multi-activity knowledge tracing as a semi-supervised sequence learning problem and optimize for accurate student performance and activity type at each time step. We demonstrate the effectiveness of our proposed model by experimenting on three real-world datasets.
more »
« less
- Award ID(s):
- 2047500
- NSF-PAR ID:
- 10434441
- Date Published:
- Journal Name:
- European Conference on Machine Learning and Principles and Practice of Knowledge
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)The state of the art knowledge tracing approaches mostly model student knowledge using their performance in assessed learning resource types, such as quizzes, assignments, and exercises, and ignore the non-assessed learning resources. However, many student activities are non-assessed, such as watching video lectures, participating in a discussion forum, and reading a section of a textbook, all of which potentially contributing to the students' knowledge growth. In this paper, we propose the  first novel deep learning based knowledge tracing model (DMKT) that explicitly model student's knowledge transitions over both assessed and non-assessed learning activities. With DMKT we can discover the underlying latent concepts of each non-assessed and assessed learning material and better predict the student performance in future assessed learning resources. We compare our proposed method with various state of the art knowledge tracing methods on four real-world datasets and show its effectiveness in predicting student performance, representing student knowledge, and discovering the underlying domain model.more » « less
-
Video-language models (VLMs), large models pre-trained on numerous but noisy video-text pairs from the internet, have revolutionized activity recognition through their remarkable generalization and open-vocabulary capabilities. While complex human activities are often hierarchical and compositional, most existing tasks for evaluating VLMs focus only on high-level video understanding, making it difficult to accurately assess and interpret the ability of VLMs to understand complex and fine-grained human activities. Inspired by the recently proposed MOMA framework, we define activity graphs as a single universal representation of human activities that encompasses video understanding at the activity, sub10 activity, and atomic action level. We redefine activity parsing as the overarching task of activity graph generation, requiring understanding human activities across all three levels. To facilitate the evaluation of models on activity parsing, we introduce MOMA-LRG (Multi-Object Multi-Actor Language-Refined Graphs), a large dataset of complex human activities with activity graph annotations that can be readily transformed into natural language sentences. Lastly, we present a model-agnostic and lightweight approach to adapting and evaluating VLMs by incorporating structured knowledge from activity graphs into VLMs, addressing the individual limitations of language and graphical models. We demonstrate a strong performance on activity parsing and few-shot video classification, and our framework is intended to foster future research in the joint modeling of videos, graphs, and language.more » « less
-
We explore the effect of auxiliary labels in improving the classification accuracy of wearable sensor-based human activity recognition (HAR) systems, which are primarily trained with the supervision of the activity labels (e.g. running, walking, jumping). Supplemental meta-data are often available during the data collection process such as body positions of the wearable sensors, subjects' demographic information (e.g. gender, age), and the type of wearable used (e.g. smartphone, smart-watch). This information, while not directly related to the activity classification task, can nonetheless provide auxiliary supervision and has the potential to significantly improve the HAR accuracy by providing extra guidance on how to handle the introduced sample heterogeneity from the change in domains (i.e positions, persons, or sensors), especially in the presence of limited activity labels. However, integrating such meta-data information in the classification pipeline is non-trivial - (i) the complex interaction between the activity and domain label space is hard to capture with a simple multi-task and/or adversarial learning setup, (ii) meta-data and activity labels might not be simultaneously available for all collected samples. To address these issues, we propose a novel framework Conditional Domain Embeddings (CoDEm). From the available unlabeled raw samples and their domain meta-data, we first learn a set of domain embeddings using a contrastive learning methodology to handle inter-domain variability and inter-domain similarity. To classify the activities, CoDEm then learns the label embeddings in a contrastive fashion, conditioned on domain embeddings with a novel attention mechanism, enforcing the model to learn the complex domain-activity relationships. We extensively evaluate CoDEm in three benchmark datasets against a number of multi-task and adversarial learning baselines and achieve state-of-the-art performance in each avenue.more » « less
-
null (Ed.)The Amundsen Sea sector of Antarctica has long been considered the most vulnerable part of the West Antarctic Ice Sheet (WAIS) because of the great water depth at the grounding line and the absence of substantial ice shelves. Glaciers in this configuration are thought to be susceptible to rapid or runaway retreat. Ice flowing into the Amundsen Sea Embayment is undergoing the most rapid changes of any sector of the Antarctic Ice Sheet outside the Antarctic Peninsula, including changes caused by substantial grounding-line retreat over recent decades, as observed from satellite data. Recent models suggest that a threshold leading to the collapse of WAIS in this sector may have been already crossed and that much of the ice sheet could be lost even under relatively moderate greenhouse gas emission scenarios. Drill cores from the Amundsen Sea provide tests of several key questions about controls on ice sheet stability. The cores offer a direct record of glacial history offshore from a drainage basin that receives ice exclusively from the WAIS, which allows clear comparisons between the WAIS history and low-latitude climate records. Today, warm Circumpolar Deep Water (CDW) is impinging onto the Amundsen Sea shelf and causing melting of the underside of the WAIS in most places. Reconstructions of past CDW intrusions can assess the ties between warm water upwelling and large-scale changes in past grounding-line positions. Carrying out these reconstructions offshore from the drainage basin that currently has the most substantial negative mass balance of ice anywhere in Antarctica is thus of prime interest to future predictions. The scientific objectives for this expedition are built on hypotheses about WAIS dynamics and related paleoenvironmental and paleoclimatic conditions. The main objectives are 1. To test the hypothesis that WAIS collapses occurred during the Neogene and Quaternary and, if so, when and under which environmental conditions; 2. To obtain ice-proximal records of ice sheet dynamics in the Amundsen Sea that correlate with global records of ice-volume changes and proxy records for atmospheric and ocean temperatures; 3. To study the stability of a marine-based WAIS margin and how warm deep-water incursions control its position on the shelf; 4. To find evidence for earliest major grounded WAIS advances onto the middle and outer shelf; 5. To test the hypothesis that the first major WAIS growth was related to the uplift of the Marie Byrd Land dome. International Ocean Discovery Program (IODP) Expedition 379 completed two very successful drill sites on the continental rise of the Amundsen Sea. Site U1532 is located on a large sediment drift, now called Resolution Drift, and penetrated to 794 m with 90% recovery. We collected almost-continuous cores from the Pleistocene through the Pliocene and into the late Miocene. At Site U1533, we drilled 383 m (70% recovery) into the more condensed sequence at the lower flank of the same sediment drift. The cores of both sites contain unique records that will enable study of the cyclicity of ice sheet advance and retreat processes as well as bottom-water circulation and water mass changes. In particular, Site U1532 revealed a sequence of Pliocene sediments with an excellent paleomagnetic record for high-resolution climate change studies of the previously sparsely sampled Pacific sector of the West Antarctic margin. Despite the drilling success at these sites, the overall expedition experienced three unexpected difficulties that affected many of the scientific objectives: 1. The extensive sea ice on the continental shelf prevented us from drilling any of the proposed shelf sites. 2. The drill sites on the continental rise were in the path of numerous icebergs of various sizes that frequently forced us to pause drilling or leave the hole entirely as they approached the ship. The overall downtime caused by approaching icebergs was 50% of our time spent on site. 3. An unfortunate injury to a member of the ship's crew cut the expedition short by one week. Recovery of core on the continental rise at Sites U1532 and U1533 cannot be used to precisely indicate the position of ice or retreat of the ice sheet on the shelf. However, these sediments contained in the cores offer a range of clues about past WAIS extent and retreat. At Sites U1532 and U1533, coarse-grained sediments interpreted to be ice-rafted debris (IRD) were identified throughout all recovered time periods. A dominant feature of the cores is recorded by lithofacies cyclicity, which is interpreted to represent relatively warmer periods variably characterized by higher microfossil abundance, greater bioturbation, and higher counts of IRD alternating with colder periods characterized by dominantly gray laminated terrigenous muds. Initial comparison of these cycles to published records from the region suggests that the units interpreted as records of warmer time intervals in the core tie to interglacial periods and the units interpreted as deposits of colder periods tie to glacial periods. The cores from the two drill sites recovered sediments of purely terrigenous origin intercalated or mixed with pelagic or hemipelagic deposits. In particular, Site U1533, which is located near a deep-sea channel originating from the continental slope, contains graded sands and gravel transported downslope from the shelf to the abyssal plain. The channel is likely the path of such sediments transported downslope by turbidity currents or other sediment-gravity flows. The association of lithologic facies at both sites predominantly reflects the interplay of downslope and contouritic sediment supply with occasional input of more pelagic sediment. Despite the lack of cores from the shelf, our records from the continental rise reveal the timing of glacial advances across the shelf and thus the existence of a continent-wide ice sheet in West Antarctica at least during longer time periods since the late Miocene. Cores from both sites contain abundant coarse-grained sediments and clasts of plutonic origin transported either by downslope processes or by ice rafting. If detailed provenance studies confirm our preliminary assessment that the origin of these samples is from the plutonic bedrock of Marie Byrd Land, their thermochronological record will potentially reveal timing and rates of denudation and erosion linked to crustal uplift. The chronostratigraphy of both sites enables the generation of a seismic sequence stratigraphy not only for the Amundsen Sea rise but also for the western Amundsen Sea along the Marie Byrd Land margin through a connecting network of seismic lines.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
