skip to main content

Title: A Comparison of Hints vs. Scaffolding in a MOOC with Adult Learners.
Scaffolding and providing feedback on problem-solving activities during online learning has consistently been shown to improve performance in younger learners. However, less is known about the impacts of feedback strategies on adult learners. This paper investigates how two computer-based support strategies, hints and required scaffolding questions, contribute to performance and behavior in an edX MOOC with integrated assignments from ASSISTments, a web-based platform that implements diverse student supports. Results from a sample of 188 adult learners indicated that those given scaffolds benefited less from ASSISTments support and were more likely to request the correct answer from the system.  more » « less
Award ID(s):
1931523 1724889 1931419
Author(s) / Creator(s):
; ; ; ;
Roll, I.; McNamara, D.; Sosnovsky, S.; Luckin, R.; Dimitrova, V.
Date Published:
Journal Name:
Artificial Intelligence in Education
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Background

    Providing adaptive scaffolds to help learners develop effective self‐regulated learning (SRL) behaviours has been an important goal for intelligent learning environments. Adaptive scaffolding is especially important in open‐ended learning environments (OELE), where novice learners often face difficulties in completing their learning tasks.


    This paper presents a systematic framework for adaptive scaffolding in Betty's Brain, a learning‐by‐teaching OELE for middle school science, where students construct a causal model to teach a virtual agent, generically named Betty. We evaluate the adaptive scaffolding framework and discuss its implications on the development of more effective scaffolds for SRL in OELEs.


    We detect key cognitive/metacognitiveinflection points, that is, moments where students' behaviours and performance change during learning, often suggesting an inability to apply effective learning strategies. At inflection points, Mr. Davis (a mentor agent in Betty's Brain) or Betty (the teachable agent) provides context‐specific conversational feedback, focusing on strategies to help the student become a more productive learner, or encouragement to support positive emotions. We conduct a classroom study with 98 middle schoolers to analyse the impact of adaptive scaffolds on students' learning behaviours and performance. We analyse how students with differential pre‐to‐post learning outcomes receive and use the scaffolds to support their subsequent learning process in Betty's Brain.

    Results and Conclusions

    Adaptive scaffolding produced mixed results, with some scaffolds (viz., strategic hints that supported debugging and assessment of causal models) being generally more useful to students than others (viz., encouragement prompts). Additionally, there were differences in how students with high versus low learning outcomes responded to some hints, as suggested by the differences in their learning behaviours and performance in the intervals after scaffolding. Overall, our findings suggest how adaptive scaffolding in OELEs like Betty's Brain can be further improved to better support SRL behaviours and narrow the learning outcomes gap between high and low performing students.


    This paper contributes to our understanding and impact of adaptive scaffolding in OELEs. The results of our study indicate that successful scaffolding has to combine context‐sensitive inflection points with conversational feedback that is tailored to the students' current proficiency levels and needs. Also, our conceptual framework can be used to design adaptive scaffolds that help students develop and apply SRL behaviours in other computer‐based learning environments.

    more » « less
  2. Abstract

    Intelligent tutoring systems (ITSs) incorporate pedagogical agents (PAs) to scaffold learners' self‐regulated learning (SRL) via prompts and feedback to promote learners' monitoring and regulation of their cognitive, affective, metacognitive and motivational processes to achieve their (sub)goals. This study examines PAs' effectiveness in scaffolding and teaching SRL during learning with MetaTutor, an ITS on the human circulatory system. Undergraduates (N = 118) were randomly assigned to a condition:Control Condition(i.e. learners could only self‐initiate SRL strategies) andPrompt and Feedback Condition(i.e. PAs prompted learners to engage in SRL). Learners' log‐file data captured when strategies were used, the initiator of the strategy (i.e. learner and PA), and the relevance of instructional content pages in relation to learner subgoals. While results showed that PAs were effective scaffolders of SRL in which they prompted learners to engage in SRL strategies more when content was relevant towards their subgoals and as time on page and task increased, there were mixed findings about the effectiveness of PAs as teachers of SRL. Findings show how production rules guiding PA prompts can improve their scaffolding and teaching of SRL across the learning task – through contextualizing SRL strategies to the instructional content and in relation to the relevance of the content to learners' subgoals.Practitioner notesWhat is already known about this topic

    Most learners struggle to efficiently and effectively use self‐regulated learning (SRL) strategies to attain goals and subgoals.

    There is a need for SRL to be scaffolded for learners to manage multiple goals and subgoals while learning about complex STEM topics.

    Intelligent tutoring systems (ITSs) typically incorporate pedagogical agents (PAs) to prompt learners to engage in SRL strategy and provide feedback.

    There are mixed findings on the effectiveness of PAs in scaffolding learners' SRL.

    What this paper adds

    We consider PAs not only scaffolders but also teachers of SRL.

    Results showed that while PAs encouraged the use of SRL strategies when the content was relevant to subgoals, they did not discourage the use of SRL strategies when the content was not relevant.

    Results for this study were mixed in their support of PAs as teachers of SRL.

    Learners increasingly depended on PAs to prompt SRL strategies as time on task progressed.

    Implications for practice and/or policy

    PAs are effective scaffolders of SRL with more research needed to understand their role as teachers of SRL.

    PA scaffolding is more essential as time on task progresses.

    When deploying specific cognitive and metacognitive SRL strategies, the relevance of the content to learners' subgoals should be taken into account.

    more » « less
  3. Because Chinese reading and writing systems are not phonetic, Mandarin Chinese learners must construct six-way mental connections in order to learn new words, linking characters, meanings, and sounds. Little research has focused on the difficulties inherent to each specific component involved in this process, especially within digital learning environments. The present work examines Chinese word acquisition within ASSISTments, an online learning platform traditionally known for mathematics education. Students were randomly assigned to one of three conditions in which researchers manipulated a learning assignment to exclude one of three bi-directional connections thought to be required for Chinese language acquisition (i.e., sound-meaning and meaning-sound). Researchers then examined whether students’ performance differed significantly when the learning assignment lacked sound-character, character-meaning, or meaning-sound connection pairs, and whether certain problem types were more difficult for students than others. Assessment of problems by component type (i.e., characters, meanings, and sounds) revealed support for the relative ease of problems that provided sounds, with students exhibiting higher accuracy with fewer attempts and less need for system feedback when sounds were included. However, analysis revealed no significant differences in word acquisition by condition, as evidenced by next-day post-test scores or pre- to post-test gain scores. Implications and suggestions for future work are discussed. 
    more » « less
  4. Abstract  
    more » « less
  5. null (Ed.)
    This poster showcases the progress of students who are receiving scholarships from the National Science Foundation S-STEM project: A Pathway to Completion for Pursuing Engineering and Engineering Technology Degrees. Thus far, 20 academically high-achieving students who demonstrate financial need have participated in the project. Thirty-six scholarships have been awarded to date, in which a maximum of twelve scholarships are awarded per semester; some students have received scholarships multiple times. Students are from electrical engineering, computer engineering, mechanical engineering, civil engineering, civil engineering technology, and modeling and simulation majors. As part of this S-STEM project, students also receive academic support, mentorship related to the development of professional workforce skills, career search skills, and opportunities to participate in industry-related field trips. Role models, many of whom are practicing engineers with STEM degrees and are military veterans, serve as presenters and share their personal career pathways and answer students’ questions in the required one-hour weekly seminar. Although the students participating in this project meet the strenuous academic criteria set by the project (3.0/4.0), many of the students struggle financially, due to having expended their G.I. benefits, which can impede their academic performance and graduation. While many student success programs focus on freshman and sophomore students, what makes this project unique is its focus on enabling student success at the junior and senior years. This project provides a portfolio of different activities for the more mature student, e.g. financial aid through scholarships, community-based learning opportunities, and academic success strategies that enable stronger retention and student completion rates. Project activities are tailored to veterans and adult learners as this group of students is particularly vulnerable given their need to simultaneously juggle academic, family, and financial obligations. 
    more » « less