A key challenge in e-learning environments like Intelligent Tutoring Systems (ITSs) is to induce effective pedagogical policies efficiently. While Deep Reinforcement Learning (DRL) often suffers from \textbf{\emph{sample inefficiency}} and \textbf{\emph{reward function}} design difficulty, Apprenticeship Learning (AL) algorithms can overcome them. However, most AL algorithms can not handle heterogeneity as they assume all demonstrations are generated with a homogeneous policy driven by a single reward function. Still, some AL algorithms which consider heterogeneity, often can not generalize to large continuous state space and only work with discrete states. In this paper, we propose an expectation-maximization(EM)-EDM, a general AL framework to induce effective pedagogical policies from given optimal or near-optimal demonstrations, which are assumed to be driven by heterogeneous reward functions. We compare the effectiveness of the policies induced by our proposed EM-EDM against four AL-based baselines and two policies induced by DRL on two different but related tasks that involve pedagogical action prediction. Our overall results showed that, for both tasks, EM-EDM outperforms the four AL baselines across all performance metrics and the two DRL baselines. This suggests that EM-EDM can effectively model complex student pedagogical decision-making processes through the ability to manage a large, continuous state space and adapt to handle diverse and heterogeneous reward functions with very few given demonstrations.
more »
« less
A Generalized Apprenticeship Learning Framework for Modeling Heterogeneous Student Pedagogical Strategies
Introduces EM‑EDM, an AL framework using expectation‑maximization to model heterogeneous student pedagogical strategies across large continuous state spaces. EM‑EDM outperforms four AL baselines and two DRL policies on two pedagogical action prediction tasks.
more »
« less
- Award ID(s):
- 2013502
- PAR ID:
- 10609441
- Publisher / Repository:
- Proceedings of the 17th International Conference on Educational Data Mining / International Educational Data Mining Society
- Date Published:
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Educational Data Mining in Computer Science Education (CSEDM) is an interdisciplinary research community that combines discipline-based computing education research (CER) with educational data-mining (EDM) to advance knowledge in ways that go beyond what either research community could do on its own. The JEDM Special Issue on CSEDM received a total of 12 submissions. Each submission was reviewed by at least three reviewers, who brought expertise from both the EDM and CER communities, as well as one of special issue editors. Ultimately, three papers were accepted, for an acceptance rate of 25%. These three papers cover a variety of important topics in CSEDM research. Edwards et al. discuss the challenges of collecting, sharing and analyzing programming data, and contribute two high-quality CS datasets. Gitinabard et al. contribute new approaches for analyzing data from pairs of students working on programs together, and show how such data can inform classroom instruction. Finally, Zhang et al. contribute a novel model for predicting students' programming performance based on their past performance. Together, these papers showcase the complexities of data, analytics and modeling in the domain of CS, and contribute to our understanding of how students learn in CS classrooms.more » « less
-
We study the convergence properties of the Expectation-Maximization algorithm in the Naive Bayes model. We show that EM can get stuck in regions of slow convergence, even when the features are binary and i.i.d. conditioning on the class label, and even under random (i.e. non worst-case) initialization. In turn, we show that EM can be bootstrapped in a pre-training step that computes a good initialization. From this initialization we show theoretically and experimentally that EM converges exponentially fast to the true model parameters. Our bootstrapping method amounts to running the EM algorithm on appropriately centered iterates of small magnitude, which as we show corresponds to effectively performing power iteration on the covariance matrix of the mixture model, although power iteration is performed under the hood by EM itself. As such, we call our bootstrapping approach “power EM.” Specifically for the case of two binary features, we show global exponentially fast convergence of EM, even without bootstrapping. Finally, as the Naive Bayes model is quite expressive, we show as corollaries of our convergence results that the EM algorithm globally converges to the true model parameters for mixtures of two Gaussians, recovering recent results of Xu et al.’2016 and Daskalakis et al. 2017.more » « less
-
The General Data Protection Regulation (GDPR) in the European Union contains directions on how user data may be collected, stored, and when it must be deleted. As similar legislation is developed around the globe, there is the potential for repercussions across multiple fields of research, including educational data mining (EDM). Over the past two decades, the EDM community has taken consistent steps to protect learner privacy within our research, whilst pursuing goals that will benefit their learning. However, recent privacy legislation may cause our practices to need to change. The right to be forgotten states that users have the right to request that all their data (including deidentified data generated by them) be removed. In this paper, we discuss the potential challenges of this legislation for EDM research, including impacts on Open Science practices, Data Modeling, and Data sharing. We also consider changes to EDM best practices that may aid compliance with this new legislation.more » « less
-
This dataset contains the compact binary populations that were used in the Cosmic Explorer MPSAC White paper1 (submitted to the NSF MPSAC ngGW Subcommittee) and the accompanying technical paper2. Contents: 1. 1-year populations for binary black hole (BBH), binary neutron star (BNS), neutron star-black hole (NSBH), intermediate mass binary black hole (IMBBH), Population III (Pop 3) binary black holes and primordial black hole (PBH) mergers. It also contains the SNRs and measurement errors on intrinsic and extrinsic parameters calculated using gwbench3. 2. 1/4-year sub-population of BNS mergers for which errors on tidal parameters were calculated. 3. An ipython notebook (instructions.ipynb) that shows how the data can be used. References: 1. Evans, Matthew et al. Cosmic Explorer: A Submission to the NSF MPSAC ngGW Subcommittee (2023). arXiv: 2306.13745 [gr-qc]. 2. Gupta, Ish et al. Characterizing Gravitational Wave Detector Networks: From A# to Cosmic Explorer (2023). In preparation. 3. Borhanian, Ssohrab. GWBENCH: a novel Fisher information package for gravitational-wave benchmarking. Class. Quant. Grav. 38, 175014 (2021). arXiv: 2010.15202 [gr-qc].more » « less
An official website of the United States government

