skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Detecting threshold concepts through Bayesian knowledge tracing: examining research skill development in biological sciences at the doctoral level
Threshold concepts are transformative elements of domain knowledge that enable those who attain them to engage domain tasks in a more sophisticated way. Existing research tends to focus on the identification of threshold concepts within undergraduate curricula as challenging concepts that prevent attainment of subsequent content until mastered. Recently, threshold concepts have likewise become a research focus at the level of doctoral studies. However, such research faces several limitations. First, the generalizability of findings in past research has been limited due to the relatively small numbers of participants in available studies. Second, it is not clear which specific skills are contingent upon mastery of identified threshold concepts, making it difficult to identify appropriate times for possible intervention. Third, threshold concepts observed across disciplines may or may not mask important nuances that apply within specific disciplinary contexts. The current study therefore employs a novel Bayesian knowledge tracing (BKT) approach to identify possible threshold concepts using a large data set from the biological sciences. Using rubric-scored samples of doctoral students’ sole-authored scholarly writing, we apply BKT as a strategy to identify potential threshold concepts by examining the ability of performance scores for specific research skills to predict score gains on other research skills. Findings demonstrate the effectiveness of this strategy, as well as convergence between results of the current study and more conventional, qualitative results identifying threshold concepts at the doctoral level.  more » « less
Award ID(s):
1760894
PAR ID:
10327291
Author(s) / Creator(s):
; ; ; ; ;
Date Published:
Journal Name:
Instructional Science
ISSN:
0020-4277
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    Work in Progress - Research Category The purpose of this work in progress paper is to understand the influence of mentoring on the science, technology, engineering, and math (STEM) doctoral student experience. This qualitative case study sought to better understand mentoring relationships between faculty doctoral mentors and STEM doctoral students. This research emphasizes the role of mentoring as part of an intervening strategy for doctoral retention and suggests culturally responsive mentoring as a means to improve the experiences of PhD under-represented minority (URM) students. This study addresses a gap in the literature related to culturally responsive mentoring and the STEM disciplines. The findings were developed from four qualitative research focus group interviews. Focus group interviews were audio recorded, transcribed, and coded by the research team. Data were analyzed using the constant comparative method, an iterative process to extrapolate key words and identify significant patterns [1]. This study uses critical inquiry as a theoretical framework. Our findings revealed that mentoring takes place within a complex environment framed by systems of inequity grounded in race and gender. Three themes were constructed from the data: mentoring as a biased environment, lack of responsiveness to student needs, and relational tensions. This paper briefly examines one theme: mentoring as a biased environment. The data highlight how biased standpoints result in a shift in the learning experience. Bias may be based on race, gender, or age, and may be implicit or explicit. Within this environment doctoral students are challenged to navigate spaces such as classrooms and laboratories that can be wrought with difficulties springing from gender and race.This paper is relevant to mentoring and STEM as it acknowledges that mentoring is a heavily nuanced practice with important cultural implications relative to PhD STEM students and faculty. 
    more » « less
  2. Personalized learning environments requiring the elicitation of a student’s knowledge state have inspired researchers to propose distinct models to understand that knowledge state. Recently, the spotlight has shone on comparisons between traditional, interpretable models such as Bayesian Knowledge Tracing (BKT) and complex, opaque neural network models such as Deep Knowledge Tracing (DKT). Although DKT appears to be a powerful predictive model, little effort has been expended to dissect the source of its strength. We begin with the observation that DKT differs from BKT along three dimensions: (1) DKT is a neural network with many free parameters, whereas BKT is a probabilistic model with few free parameters; (2) a single instance of DKT is used to model all skills in a domain, whereas a separate instance of BKT is constructed for each skill; and (3) the input to DKT interlaces practice from multiple skills, whereas the input to BKT is separated by skill. We tease apart these three dimensions by constructing versions of DKT which are trained on single skills and which are trained on sequences separated by skill. Exploration of three data sets reveals that dimensions (1) and (3) are critical; dimension (2) is not. Our investigation gives us insight into the structural regularities in the data that DKT is able to exploit but that BKT cannot. 
    more » « less
  3. null (Ed.)
    Recent student knowledge modeling algorithms such as Deep Knowledge Tracing (DKT) and Dynamic Key-Value Memory Networks (DKVMN) have been shown to produce accurate predictions of problem correctness within the same learning system. However, these algorithms do not attempt to directly infer student knowledge. In this paper we present an extension to these algorithms to also infer knowledge. We apply this extension to DKT and DKVMN, resulting in knowledge estimates that correlate better with a posttest than knowledge estimates from Bayesian Knowledge Tracing (BKT), an algorithm designed to infer knowledge, and another classic algorithm, Performance Factors Analysis (PFA). We also apply our extension to correctness predictions from BKT and PFA, finding that knowledge estimates produced with it correlate better with the posttest than BKT and PFA’s standard knowledge estimates. These findings are significant since the primary aim of education is to prepare students for later experiences outside of the immediate learning activity. 
    more » « less
  4. Studies on graduate education have shown that underrepresented minorities finish PhDs in engineering at lesser rates and longer timeframes than their majority counterparts. While multiple interventions have been designed for students considering their decision to apply for graduate school or students completing their doctoral journey, few focus on the transition into those doctoral programs. To prepare minoritized doctoral students for this transition to the Ph. D., we developed and researched the Rising Doctoral Institute (RDI). The RDI is a four-day summer workshop for incoming doctoral students who identify as underrepresented in engineering and intend to begin graduate school in the Fall semester. This paper aims to discuss the process through which we developed the RDI and our initial research findings. We conclude with our plan to disseminate these workshops across multiple US institutions using a change-theory informed dissemination model. 
    more » « less
  5. Studies on graduate education have shown that underrepresented minorities finish PhDs in engineering at lesser rates and longer timeframes than their majority counterparts. While multiple interventions have been designed for students considering their decision to apply for graduate school or students completing their doctoral journey, few focus on the transition into those doctoral programs. To prepare minoritized doctoral students for this transition to the Ph.D., we developed and researched the Rising Doctoral Institute (RDI). The RDI is a four-day summer workshop for incoming doctoral students who identify as underrepresented in engineering and intend to begin graduate school in the Fall semester. This paper aims to discuss the process through which we developed the RDI and our initial research findings. We conclude with our plan to disseminate these workshops across multiple US institutions using a change-theory informed dissemination model. 
    more » « less