skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: The Effect of an Intelligent Tutor on Performance on Specific Posttest Problems
This paper drills deeper into the documented effects of the Cognitive Tutor Algebra I and ASSISTments intelligent tutoring systems by estimating their effects on specific problems. We start by describing a multilevel Rasch-type model that facilitates testing for differences in the effects between problems and precise problem-specific effect estimation without the need for multiple comparisons corrections. We find that the effects of both intelligent tutors vary between problems– the effects are positive for some, negative for others, and undeterminable for the rest. Next we explore hypotheses explaining why effects might be larger for some problems than for others. In the case of ASSISTments, there is no evidence that problems that are more closely related to students’ work in the tutor displayed larger treatment effects.  more » « less
Award ID(s):
1840771
PAR ID:
10275074
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
Educational Data Mining EDM 2021
Page Range / eLocation ID:
246
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; (, LAK 2024 (submitted, in review))
    We present a conversational AI tutor (CAIT) for the purpose of aiding students on middle school math problems. CAIT was created utilizing the CLASS framework, and it is an LLM fine-tuned on Vicuna using a conversational dataset created by prompting ChatGPT using problems and explanations in ASSISTments. CAIT is trained to generate scaffolding questions, provide hints, and correct mistakes on math problems. We find that CAIT identifies 60% of correct answers as correct, generates effective sub-problems 33% of the time, and has a positive sentiment 72% of the time, with the remaining 28% of interactions being neutral. This paper discusses the hurdles to further implementation of CAIT into ASSISTments, namely improved accuracy and efficacy of sub-problems, and establishes CAIT as a proof of concept that the CLASS framework can be applied to create an effective mathematics tutorbot. 
    more » « less
  2. ; ; ; ; ; ; ; ; ; (, Proceedings of the National Academy of Sciences)
    Human decision making is plagued by systematic errors that can have devastating consequences. Previous research has found that such errors can be partly prevented by teaching people decision strategies that would allow them to make better choices in specific situations. Three bottlenecks of this approach are our limited knowledge of effective decision strategies, the limited transfer of learning beyond the trained task, and the challenge of efficiently teaching good decision strategies to a large number of people. We introduce a general approach to solving these problems that leverages artificial intelligence to discover and teach optimal decision strategies. As a proof of concept, we developed an intelligent tutor that teaches people the automatically discovered optimal heuristic for environments where immediate rewards do not predict long-term outcomes. We found that practice with our intelligent tutor was more effective than conventional approaches to improving human decision making. The benefits of training with our cognitive tutor transferred to a more challenging task and were retained over time. Our general approach to improving human decision making by developing intelligent tutors also proved successful for another environment with a very different reward structure. These findings suggest that leveraging artificial intelligence to discover and teach optimal cognitive strategies is a promising approach to improving human judgment and decision making. 
    more » « less
  3. ; ; (, AIED 2023)
    Math performance continues to be an important focus for improve- ment. Many districts adopted educational technology programs to support student learning and teacher instruction. The ASSISTments program provides feedback to students as they solve homework problems and automatically prepares reports for teachers about student performance on daily assignments. During the 2018- 19 and 2019-20 school years, WestEd led a large-scale randomized controlled trial to replicate the effects of ASSISTments in 63 schools in North Carolina in the US. 32 treatment schools implemented ASSISTments in 7th-grade math class- rooms. Recently, we conducted a follow-up analysis to measure the long-term effects of ASSISTments on student performance one year after the intervention, when the students were in 8th grade. The initial results suggested that implement- ing ASSISTments in 7th grade improved students’ performance in 8th grade and minority students benefited more from the intervention. 
    more » « less
  4. ; ; ; (, Artificial Intelligence in Education 2020)
    Cognitive control and rule learning are two important mechanisms that explain how goals influence behavior and how knowledge is acquired. These mechanisms are studied heavily in cognitive science literature within highly controlled tasks to understand human cognition. Although they are closely linked to the student behaviors that are often studied within intelligent tutoring systems (ITS), their direct effects on learning have not been explored. Understanding these underlying cognitive mechanisms of beneficial and harmful student behaviors can provide deeper insight into detecting such behaviors and improve predictive models of student learning. In this paper, we present a thinkaloud study where we asked students to narrate their thought processes while solving probability problems in ASSISTments. Students are randomly assigned to one of two conditions that are designed to induce the two modes of cognitive control based on the Dual Mechanisms of Control framework. We also observe how the students go through the phases of rule learning as defined in a rule learning paradigm. We discuss the effects of these different mechanisms on learning, and how the information they provide can be used in student modeling. 
    more » « less
  5. ; ; ; (, International Conference of Artificial Intelligence in Education)
    Cognitive control and rule learning are two important mechanisms that explain how goals influence behavior and how knowledge is acquired. These mechanisms are studied heavily in cognitive science literature within highly controlled tasks to understand human cognition. Although they are closely linked to the student behaviors that are often studied within intelligent tutoring systems (ITS), their direct effects on learning have not been explored. Understanding these underlying cognitive mechanisms of beneficial and harmful student behaviors can provide deeper insight into detecting such behaviors and improve predictive models of student learning. In this paper, we present a thinkaloud study where we asked students to narrate their thought processes while solving probability problems in ASSISTments. Students are randomly assigned to one of two conditions that are designed to induce the two modes of cognitive control based on the Dual Mechanisms of Control framework. We also observe how the students go through the phases of rule learning as defined in a rule learning paradigm. We discuss the effects of these different mechanisms on learning, and how the information they provide can be used in student modeling. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email