skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: A Trajectory-Clustering Framework for Assessing AI-Based Adaptive Interventions in Undergraduate STEM Learning
In this paper, we present a framework for quantifying the impact of interventions on the full trajectories of students' experiences. The interventions are given periodically based on student performance forecasting from an artificial intelligence (AI) model. We performed a small-scale randomized controlled trial for evaluating the impact of the AI-based intervention system on the undergraduate students of a science, technology, engineering, and mathematics (STEM) course. Intervention messaging content was based on machine learning forecasting models trained on data collected from the students in the same course over the preceding 3 years. Trial results show that the intervention produced a statistically significant increase in the proportion of students that achieved a passing grade. By applying the trajectory-analysis framework we find that the intervention impacts the stories of some types of students more than others, and use this to define new ways of identifying students who are most likely to benefit. Together these outcomes point to the potential and promise of just-in-time interventions for STEM learning and the need for larger fully-powered randomized controlled trials.  more » « less
Award ID(s):
2142558
PAR ID:
10499291
Author(s) / Creator(s):
;
Publisher / Repository:
American Society for Engineering Education
Date Published:
Journal Name:
2023 ASEE Annual Conference & Exposition
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; (, PLOS ONE)
    Carvalho, Paulo F. (Ed.)
    We present results from a small-scale randomized controlled trial that evaluates the impact of just-in-time interventions on the academic outcomes of N = 65 undergraduate students in a STEM course. Intervention messaging content was based on machine learning forecasting models of data collected from 537 students in the same course over the preceding 3 years. Trial results show that the intervention produced a statistically significant increase in the proportion of students that achieved a passing grade. The outcomes point to the potential and promise of just-in-time interventions for STEM learning and the need for larger fully-powered randomized controlled trials. 
    more » « less
  2. ; (, AAC&U)
    Many campuses have utilized linked-course communities, wherein students take two or more courses with the same peers, in an effort to enhance learning and build community among students. Previous research on the effectiveness of linked-course communities have utilized quasi-experimental designs that are subject to selection bias – the communities may be effective due to which students volunteered to join the communities. This research session reports findings from a randomized controlled trial that eliminated selection bias by randomly assigning first-time first-year College of Science and Math students to linked-course communities or a control group. Findings demonstrate that students in the communities earned higher GPAs and more STEM credits in their first semester than students in the control group. Session participants will identify a curricular intervention and consider the feasibility of conducting a randomized controlled trial to assess effectiveness. 
    more » « less
  3. ; ; ; ; (, CBE—Life Sciences Education)
    Dolan, Erin L. (Ed.)
    Mindset interventions, which shift students’ beliefs about classroom experiences, have shown promise for promoting diversity in science, technology, engineering, and mathematics (STEM). Psychologists have emphasized the importance of customizing these interventions to specific courses, but there is not yet a protocol for doing so. We developed a protocol for creating customized “peer-modeled” mindset interventions that elicit advice from former students in videotaped interviews. In intervention activities, clips from these interviews, in which the former students’ stories model the changes in thinking about challenge and struggle that helped them succeed in a specific course, are provided to incoming life sciences students. Using this protocol, we developed a customized intervention for three sections of Introductory Biology I at a large university and tested it in a randomized controlled trial ( N = 917). The intervention shifted students’ attributions for struggle in the class away from a lack of potential to succeed and toward the need to develop a better approach to studying. The intervention also improved students’ approaches to studying and sense of belonging and had promising effects on performance and persistence in biology. Effects were pronounced among first-generation college students and underrepresented racial/ethnic minority students, who have been historically underrepresented in the STEM fields. 
    more » « less
  4. ; ; (, L@S '23: Proceedings of the Tenth ACM Conference on Learning @ Scale)
    In past work, time management interventions involving prompts, alerts, and planning tools have successfully nudged students in online courses, leading to higher engagement and improved performance. However, few studies have investigated the effectiveness of these interventions over time, understanding if the effectiveness maintains or changes based on dosage (i.e., how often an intervention is provided). In the current study, we conducted a randomized controlled trial to test if the effect of a time management intervention changes over repeated use. Students at an online computer science course were randomly assigned to receive interventions based on two schedules (i.e., high-dosage vs. low-dosage). We ran a two-way mixed ANOVA, comparing students' assignment start time and performance across several weeks. Unexpectedly, we did not find a significant main effect from the use of the intervention, nor was there an interaction effect between the use of the intervention and week of the course. 
    more » « less
  5. ; ; ; ; ; ; (, AAC&U Conference: Transforming STEM Higher Education)
    Four writing-intensive, inquiry-based, three-credit seminars were created to serve as the hub for linked learning communities for first-year students in STEM. Based on United Nations Sustainable Development Goals (UN SDGs), the seminars engaged students in socially-relevant modeling, lab work, and public presentations. The seminars were designed to foster a communal view of science and mathematics, both in terms of the importance of collaboration to STEM success and the application of STEM to real-world problems. Course structures and sample materials will be shared, along with preliminary analyses from a randomized controlled trial comparing students in the seminars to a control group of peers. In fall 2021,students who participated in the seminars reported increased awareness of the UN SDGs, valued team work more highly, and earned more credits and higher grades than control group students. Supported by NSF2020765, these seminars are part of a study of the effectiveness of learning communities. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email