skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Single Template vs. Multiple Templates: Examining the Effects of Problem Format on Performance
Classroom and lab-based research have shown the advantages of exposing students to a variety of problems with format differences between them, compared to giving students problem sets with a single problem format. In this paper, we investigate whether this approach can be effectively deployed in an intelligent tutoring system, which affords the opportunity to automatically generate and adapt problem content for practice and assessment purposes. We conducted a randomized controlled trial to compare students who practiced problems based on a single template to students who practiced problems based on multiple templates within the same intelligent tutoring system. No conclusive evidence was found for differences in the two conditions on students’ post-test performance and hint request behavior. However, students who saw multiple templates spent more time answering practice items compared to students who solved problems of a single structure, making the same degree of progress but taking longer to do so.  more » « less
Award ID(s):
1931523
PAR ID:
10191146
Author(s) / Creator(s):
; ; ; ; ; ;
Date Published:
Journal Name:
International Conference on the Learning Sciences
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; (, Proceedings of the International Conference on Artificial Intelligence in Education)
    Olney, AM; Chounta, IA; Liu, Z; Santos, OC; Bittencourt, II (Ed.)
    This work investigates how tutoring discourse interacts with students’ proximal knowledge to explain and predict students’ learning outcomes. Our work is conducted in the context of high-dosage human tutoring where 9th-grade students attended small group tutorials and individually practiced problems on an Intelligent Tutoring System (ITS). We analyzed whether tutors’ talk moves and students’ performance on the ITS predicted scores on math learning assessments. We trained Random Forest Classifiers (RFCs) to distinguish high and low assessment scores based on tutor talk moves, student’s ITS performance metrics, and their combination. A decision tree was extracted from each RFC to yield an interpretable model. We found AUCs of 0.63 for talk moves, 0.66 for ITS, and 0.77 for their combination, suggesting interactivity among the two feature sources. Specifically, the best decision tree emerged from combining the tutor talk moves that encouraged rigorous thinking and students’ ITS mastery. In essence, tutor talk that encouraged mathematical reasoning predicted achievement for students who demonstrated high mastery on the ITS, whereas tutors’ revoicing of students’ mathematical ideas and contributions was predictive for students with low ITS mastery. Implications for practice are discussed. 
    more » « less
  2. ; ; (, Artificial Intelligence in Education. AIED 2024. Lecture Notes in Computer Science)
    Olney, AM; Chounta, IA; Liu, Z; Santos; OC; Bittencourt, II (Ed.)
    This work investigates how tutoring discourse interacts with students’ proximal knowledge to explain and predict students’ learning outcomes. Our work is conducted in the context of high-dosage human tutoring where 9th-grade students (N = 1080) attended small group tutorials and individually practiced problems on an Intelligent Tutoring System (ITS). We analyzed whether tutors’ talk moves and students’ performance on the ITS predicted scores on math learning assessments. We trained Random Forest Classifiers (RFCs) to distinguish high and low assessment scores based on tutor talk moves, student’s ITS performance metrics, and their combination. A decision tree was extracted from each RFC to yield an interpretable model. We found AUCs of 0.63 for talk moves, 0.66 for ITS, and 0.77 for their combination, suggesting interactivity among the two feature sources. Specifically, the best decision tree emerged from combining the tutor talk moves that encouraged rigorous thinking and students’ ITS mastery. In essence, tutor talk that encouraged mathematical reasoning predicted achievement for students who demonstrated high mastery on the ITS, whereas tutors’ revoicing of students’ mathematical ideas and contributions was predictive for students with low ITS mastery. Implications for practice are discussed. 
    more » « less
  3. ; ; (, Springer Cham)
    Olney, A M; Chounta, I A; Liu, Z; Santos, O C; Bittencourt, I I (Ed.)
    This work investigates how tutoring discourse interacts with students’ proximal knowledge to explain and predict students’ learning outcomes. Our work is conducted in the context of high-dosage human tutoring where 9th-grade students attended small group tutorials and individually practiced problems on an Intelligent Tutoring System (ITS). We analyzed whether tutors’ talk moves and students’ performance on the ITS predicted scores on math learning assessments. We trained Random Forest Classifiers (RFCs) to distinguish high and low assessment scores based on tutor talk moves, student’s ITS performance metrics, and their combination. A decision tree was extracted from each RFC to yield an interpretable model. We found AUCs of 0.63 for talk moves, 0.66 for ITS, and 0.77 for their combination, suggesting interactivity among the two feature sources. Specifically, the best decision tree emerged from combining the tutor talk moves that encouraged rigorous thinking and students’ ITS mastery. In essence, tutor talk that encouraged mathematical reasoning predicted achievement for students who demonstrated high mastery on the ITS, whereas tutors’ revoicing of students’ mathematical ideas and contributions was predictive for students with low ITS mastery. Implications for practice are discussed. 
    more » « less
  4. ; ; ; (, Educational Data Mining EDM 2021)
    null (Ed.)
    This paper drills deeper into the documented effects of the Cognitive Tutor Algebra I and ASSISTments intelligent tutoring systems by estimating their effects on specific problems. We start by describing a multilevel Rasch-type model that facilitates testing for differences in the effects between problems and precise problem-specific effect estimation without the need for multiple comparisons corrections. We find that the effects of both intelligent tutors vary between problems– the effects are positive for some, negative for others, and undeterminable for the rest. Next we explore hypotheses explaining why effects might be larger for some problems than for others. In the case of ASSISTments, there is no evidence that problems that are more closely related to students’ work in the tutor displayed larger treatment effects. 
    more » « less
  5. ; ; ; ; (, Proceedings of the 44th Annual Conference of the Cognitive Science Society)
    Culbertson, J.; Perfors, A.; Rabagliati, H.; Ramenzoni, V. (Ed.)
    One pedagogical technique that promotes conceptual understanding in mathematics learners is self-explanation integrated with worked examples (e.g.,Rittle-Johnson et al., 2017). In this work, we implemented self-explanations with worked examples (correct and erroneous) in a software-based Intelligent Tutoring System (ITS) for learning algebra. We developed an approach to eliciting self-explanations in which the ITS guided students to select explanations that were conceptually rich in nature. Students who used the ITS with self-explanations scored higher on a posttest that included items tapping both conceptual and procedural knowledge than did students who used a version of the ITS that included only traditional problem-solving practice. This study replicates previous findings that self-explanation and worked examples in an ITS can foster algebra learning (Booth et al., 2013). Further, this study extends prior work to show that guiding students towards conceptual explanations is beneficial. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email