Self-explanation is a learning strategy in which learners attempt to make sense of what they learn by generating explanations to themselves (Chi et al.,1989;Rittle-Johnson et al., 2017). A number of studies have provided evidence for the effectiveness of self-explanation across domains (Ainsworth & Loizou, 2003;Bisra et al., 2018). From a cognitive perspective, self-explanation helps learners integrate to-be-learned information with their prior knowledge, leading to deeper understanding of the content (Bisra et al., 2018). For example, in the context of problem solving in mathematics, learners may be asked to provide reasoning for their solved steps in order to deepen their conceptual understanding of the procedures. Although self-explanation activities may take different forms (e.g., explaining worked examples, explaining while solving problems, and explaining text passages), they share the core principle of supporting deeper understanding through connecting new content with existing knowledge.
The demonstrated effectiveness of self-explanation does not guarantee that effective self-explanation activities are easily designed. Self-explanation can be a demanding task for learners. It has been reported that scaffolding self-explanation activities facilitates learning (Rittle-Johnson et al., 2017). Prior studies have designed and tested various types of scaffolded self-explanation, such as presenting menu-based, multiple-choice explanations (Aleven & Koedinger, 2002;Berthold et al., 2011;Rau et al., 2015), providing training on self-explanation (Hodds et al., 2014), using visual representations (Ainsworth & Loizou, 2003;Nagashima, Bartel et al., 2020), using contrasting cases (Sidney et al., 2015), and providing feedback on self-explanation (Aleven & Koedinger, 2002).
All of these types of self-explanation support have been shown to be effective. Yet, there are still challenges in how best to design optimal scaffolding support for self-explanation. A first challenge lies in how to design scaffolded self-explanation to promote both conceptual and procedural knowledge. Acquiring both conceptual and procedural knowledge is fundamental to learning (Rittle-Johnson & Alibali, 1999); however, studies on scaffolded self-explanation have typically shown it to be effective for enhancing either conceptual knowledge or procedural knowledge, but not both (Berthold et al., 2011;Nagashima, Bartel et al., 2020;Rau et al., 2015, but see Aleven & Koedinger, 2002). Rittle-Johnson et al. (2017) explain that this disassociation may be due to the unique characteristics of specific forms of scaffolding. Self-explanation scaffolding designed to focus on one aspect of content may hinder learners' focus on other important aspects. For example, asking students to select a correct conceptual explanation from among a list of similar explanations in a multiple-choice format would encourage learners to focus on conceptual understanding of the content, but it would not give an opportunity for learners to develop their procedural skills (e.g., problem-solving skills).
A second challenge is how to design scaffolded self-explanation that enhances problem-solving performance when combined with, or embedded in, problem-solving activities. Self-explanation can be timeconsuming, and because self-explanation requires learners to engage in additional cognitive activities, learners who receive self-explanation support may solve fewer problems in a limited amount of time compared to solving problems without self-explanation support. If scaffolded appropriately during self-explanation, learners' performance on the target task would improve. This would result in efficient learning (i.e., learners with selfexplanation achieve similar learning gains with fewer problems or less time spent compared to those without self-explanation). Most prior studies of self-explanation do not report measures of the problem-solving performance and efficiency of learning with self-explanation, such as time spent on the task (Bisra et al., 2018; but see Aleven & Koedinger, 2002). In sum, there are persistent design challenges in how to design effective and efficient self-explanation that supports both learning and performance.
To approach these challenges, we designed self-explanation support for a web-based educational software called an Intelligent Tutoring System (ITS) for algebra problem solving (Long & Aleven, 2014). In our design, selfexplanation is interleaved with problem solving; learners are asked to explain the next strategic problem-solving step in the form of a diagram before doing the same step in symbols (Figures 123). They receive feedback from the ITS both on their explanation and their step using mathematical symbols. We designed the self-explanation support following several evidence-based principles from cognitive psychology, educational psychology, instructional design, and the learning sciences, which we describe below. Visual representations designed to support students' conceptual understanding Research has shown that visual representations can support conceptual understanding (Rau, 2017). Visual representations can depict information that is difficult to express through verbal means and can make important information salient. We chose a visual representation called tape diagrams, which are commonly used in algebra classrooms in countries such as Japan, Singapore, and the United States (Booth & Koedinger, 2012;Chu et al., 2017;Murata, 2008). Prior studies using tape diagrams in algebra problem solving show that tape diagrams help students gain conceptual understanding and avoid conceptual errors (Chu et al., 2017;Nagashima, Bartel et al., 2020). In particular, our own prior experiment found that diagrammatic self-explanation (in which students, after each equation-solving step, are asked to select, from three options, a diagram that corresponds to the step) helped learners gain conceptual knowledge in algebra (Nagashima, Bartel et al., 2020). In the present study, students are similarly asked to choose tape diagrams as a way to explain their steps, following the principle of anticipatory self-explanation (Bisra et al., 2018;Renkl, 1997), as explained next.
Anticipatory self-explanation to support understanding of problem-solving strategies Anticipatory self-explanation is a type of self-explanation in which learners generate inferences about future steps. Previously, Renkl (1997) found that, when prompted to talk aloud while studying worked examples that provided solutions step-by-step, many successful self-explainers predicted solutions in advance. In algebra problem solving, such anticipatory self-explanation, rather than post-hoc self-explanation, can potentially support inference generation about strategic problem-solving steps (e.g., "what would be a good next step for the equation, 3x + 2 = 8?"). If students consider the mathematical symbols as the target representation to learn, engaging in step-level anticipatory self-explanation could help students understand strategic next steps, which would improve both understanding of strategic solution steps and problem-solving performance. On the other hand, post-hoc self-explanation might not be particularly effective for helping students take strategic problemsolving steps.
The use of contrasting cases is an established instructional strategy in which learners are presented with contrasting examples that differ in meaningful conceptual aspects (Schwartz et al., 2011). Contrasting cases help learners notice meaningful differences. This instructional strategy is typically used with prompts for selfexplanation, to encourage learners to cognitively and constructively engage with the cases (Sidney et al., 2015).
In the self-explanation support used in the current study, three options of tape diagrams are displayed, which differ in one conceptual aspect and one strategy-related aspect. For example, in Figure 2 the tutor displays three diagrams that represent a correct and strategic next step (diagram on the left), an incorrect option (diagram on the right, in which the subtraction is done on only one side of the equation) and an option that is correct but not strategic (diagram in the middle, in which 2x was added to both sides, which does not get the learner closer to the solution). This set of options allows learners to distinguish, not only between correct and incorrect steps, but also between correct and strategic steps and correct but not strategic steps. In problem states in which two correct and strategic steps are available (e.g., subtracting 2x from 8x = 2x + 6 or dividing both sides by 2), the ITS shows those two options and one incorrect option. Engaging with contrasting cases prior to practicing the target problem-solving skill with symbols might be particularly meaningful, because students would be able to follow the selected diagram option when entering the solution step with symbols and thereby learn to use correct and strategic steps.
In the present study, we investigate the effectiveness of scaffolded self-explanation support on learning and performance. We hypothesize that (H1) the anticipatory diagrammatic self-explanation will promote students' conceptual understanding, enhance procedural skills, and help students learn formal algebraic strategies. We also hypothesize that (H2) the anticipatory diagrammatic self-explanation will enhance performance during problem solving in the ITS; students with the support will perform better on learning process measures (e.g., fewer hint requests and fewer incorrect attempts per step) while solving symbolic problem-solving steps, and they will solve a similar number of problems as students who do not receive the scaffolded self-explanation support.
We conducted an in vivo experiment (i.e., a randomized controlled experiment in a real classroom context) at two private schools in the United States. Participants included 55 6 th graders and 54 7 th graders across nine class sections taught by four teachers. The experiment was conducted in October 2020, when both schools adopted a hybrid teaching mode in which the majority of students (n = 102) attended study sessions in-person and the rest of the students attended remotely (n = 7). Teachers reported that they had never focused their instruction on tape diagrams, although they indicated that some students might have seen tape diagrams in their learning materials.
Intelligent Tutoring System for equation solving In addition to the anticipatory diagrammatic self-explanation ITS described above, we used a version that did not include tape diagrams (Figure 4) (Long & Aleven, 2014). In this No-Diagram ITS, students learn to solve equations step-by-step, but without diagrammatic self-explanation steps. All other features (e.g., step-level feedback messages and hints) are the same as in the version with tape diagrams. Both ITS versions had four different types of equations, which were chosen in consultation with the teachers (Table 1). We only used equations with positive numbers since tape diagrams were not found useful for representing negative numbers (Nagashima, Yang et al., 2020). Most of the participants in this study, per teachers' report, had seen or practiced Levels 1 and 2 problems, but had not learned Levels 3 and 4 problems.
We developed web-based pretest and posttest assessments to assess students' conceptual and procedural knowledge of basic algebra. The tests contained several items drawn from our previous work (Nagashima, Bartel et al., 2020) as well as new items. The conceptual knowledge items consisted of eight multiple-choice questions and one open-ended question, which assessed a wide range of conceptual knowledge constructs, including like terms, inverse operations, isolating variables, and the concept of keeping both sides of an equation equal. We also included four problem-solving items (e.g., "solve for x: 3x + 2 = 8"), including two items that were similar to those included in the ITS and two transfer items involving negative numbers. We developed two isomorphic versions of the test that varied only with respect to the specific numbers used in the items;
participants received one form as pretest and the other as posttest (with versions counterbalanced across subjects).
The study took place during two regular mathematics classes. The classes were virtually connected to the experimenters and remote learners through a video conferencing system.
One 6 th grader was absent for the second day and excluded from the analysis; therefore, we analyzed data from the remaining 108 students, namely, 54 6 th -graders (28 Diagram, 26 No-Diagram) and 54 7 th -graders ( 27Diagram, 27 No-Diagram). Open-ended items were coded for whether student answers were correct or incorrect by two researchers (Cohen's kappa = .91). Table 2 presents raw pretest and posttest performance on conceptual knowledge (CK) and procedural knowledge (PK) items. The maximum scores were 9 and 4, respectively. We first tested hypothesis H1 (benefits of anticipatory diagrammatic self-explanation with respect to learning outcomes). We analyzed the data using hierarchical linear modeling (HLM) because the study was conducted in nine classes taught by four teachers at two schools. According to both AIC and BIC, a two-level model showed the best fit, in which students (level 1) were nested in classes (level 2). The inclusion of teachers (level 3) and schools (level 4) did not improve the model fit. We ran two HLMs with posttest scores on CK and PK as dependent variables, type of ITS assigned as the independent variable, and pretest scores (either CK or PK given the dependent variable) as a covariate. For both CK and PK, there was no significant effect of the Diagram/No-Diagram condition (CK: t(99.3) = -1.030, p = .31, PK: t(99.4) = -0.292, p = .77). We also ran two additional HLMs, regressing pretest-posttest gains for CK and PK (dependent variables) on type of ITS. There was a significant gain from pretest to posttest for CK (t(108) = 2.778, p < .01) but not for PK (t(106) = 1.153, p = .26), and no significant effect of ITS type. This suggests that students in both ITS conditions improved in conceptual knowledge but not in procedural knowledge.
We then analyzed the strategies that students used to solve the problem-solving items on the pretest and posttest. We adopted a coding scheme by Koedinger et al. (2008), which identified both formal (algebraic) and informal (non-algebraic) ways of solving equations (Cohen's kappa = .73; Table 3). We were primarily interested in the Algebra strategy because the goal of the ITS was to help students learn the formal algebraic strategy. We performed the strategy coding independent of the correctness coding used to calculate students' test scores. On the pretest, 11 students in the Diagram condition and 17 students in the No-Diagram condition used the Algebraic strategy on one or more problem-solving items. More students did so on the posttest; 26 students in the Diagram condition and 23 students in the No-Diagram condition used the Algebraic strategy. We used McNemar's test to compare the frequency of use of the Algebra strategy at pretest and posttest for each condition. The increase in frequency was significant (p < .01) for students in the Diagram condition but was not significant (p = .11) for students in the No-Diagram condition. This pattern also held when we limited the analysis to problems involving negative numbers (transfer problems); there was a pretest-posttest increase of only 1 student in the No-Diagram condition, but 12 students in the Diagram condition (p < .01). These findings suggest that, although students who learned with anticipatory diagrammatic self-explanation did not have greater gains on tests of conceptual and procedural knowledge, they were more likely to learn the formal algebraic strategy and to apply it to problems with no diagram support, even for problem types that they did not practice in the ITS (H1 partially supported). Log data analysis on students' learning processes
Next, we tested hypothesis H2 (benefits of anticipatory diagrammatic self-explanation with respect to learning processes), using log data from the ITS. Specifically, we looked at "learning curves", which plot students' performance within the ITS over time (Rivers et al., 2016). Figure 5 depicts learning curves for the two conditions. The y-axis shows the error rate on steps in tutor problems, averaged across students and skills, and the x-axis shows the sequence of opportunities for practicing each skill. Learning curve analysis assumes that learning occurs when a curve starts with a relatively high initial error rate and gradually goes down as students practice the target skills. The curves are fit to student performance data using the Additive Factors Model (AFM), a specialized form of logistic regression (Rivers et al., 2016). In our study, students practiced a variety of equation-solving skills (e.g., subtracting variable terms). We expected that students who learned with diagrammatic self-explanation support would perform better in the ITS than their peers who did not receive the support (H2). On the symbolic problem-solving steps in the ITS (i.e., excluding the performance on the selfexplanation steps, which only occurred in the Diagram condition), students in the Diagram condition had a lower error rate than students in the No-Diagram condition. Figure 5 shows learning curves averaged across all symbolic equation-solving skills students in both conditions practiced. Students in the Diagram condition made fewer errors than those in the No-Diagram condition, especially on the earlier opportunities. Both groups improved as they solved more problems (i.e., both curves show a gradual decline). After much practice, the No Diagram condition eventually lowered their error rate to the same level as the Diagram condition.
In parallel to the trend observed in the learning curves, we found that, when restricting the analysis to symbolic steps only (i.e., excluding diagrammatic self-explanation steps), students who received the selfexplanation support trended toward using fewer hints (t(89.52) = -1.812, p = .07) and spent significantly less time on each symbolic problem-solving step (t(99.51) = -2.238, p = .03) than students who did not receive the self-explanation support (Table 4). The average number of problems solved in the ITS during the (fixed amount of) available time did not differ significantly across conditions, (t(99.30) = -0.528, p = .60) (Table 4).
In summary, the students in both conditions practiced a similar number of problems in the ITS in a similar amount of time overall, and the anticipatory diagrammatic self-explanation helped students spend less time and ask for fewer hints on symbolic steps (H2 partially supported). In addition, the learning curves indicate that students in both conditions learned equation-solving skills eventually, but the students in the Diagram condition learned them faster and had a smoother experience, with fewer errors.
Self-explanation has been shown to support student learning in various domains, but it is not easy to design appropriately-scaffolded self-explanation activities. Our study investigated the effectiveness of anticipatory diagrammatic self-explanation as a proposed approach to enhancing both learning and performance. We found that anticipatory diagrammatic self-explanation embedded in an Intelligent Tutoring System (ITS) helped students learn to apply a formal, algebraic problem-solving strategy to problems outside the ITS and to transfer problems involving negative numbers (H1). Anticipatory diagrammatic self-explanation also supported student performance within the ITS, measured by lower learning curves, less frequent use of hints, and less time spent on each symbolic equation-solving step (H2). Anticipatory self-explanation did not lead to differences in posttest scores, contrary to H1, but it helped students learn more efficiently; students learned the formal algebraic strategy while solving a similar number of problems with less time and fewer errors and hint requests, and they achieved similar gains on conceptual knowledge (H2).
We attribute these findings to the design and learning principles used in supporting anticipatory diagrammatic self-explanation. Specifically, we reason that the process of selecting the next correct-andstrategic problem-solving step, depicted diagrammatically, helped students perform better and faster on the corresponding step with symbols. On steps with symbols, students had a diagrammatic representation of the step available to them on the screen. They could refer to this representation as they sought to express the step using mathematical symbols. Engaging in this cognitive process may have helped students understand step-level formal strategies in a visual form (e.g., visually seeing that constant terms are taken out from both sides of an equation). Comparing and contrasting the different tape diagrams may have supported students in selecting steps that were both correct and strategic, and it may have helped them avoid using informal strategies, such as guessing. It may be, as well, that the better performance resulting from the anticipatory diagrams gave students a bit more confidence to take on the challenge of moving towards formal algebra.
An intriguing question is why the ITS with anticipatory diagrammatic self-explanation did not lead to greater gains in conceptual and procedural knowledge than the ITS with no diagram support. Regarding procedural knowledge, students did not make gains from pretest to posttest in either condition. Further, there was no difference in solving equations correctly between the conditions at post-test, even though students with diagrams exhibited greater use of formal problem-solving strategies. It is possible that students in the Diagram condition might need more practice in correctly applying the formal strategy they acquired in the ITS without the help of diagrams. In other words, it seems that students in the Diagram condition developed further towards formal use of algebra than their counterparts in the No-Diagram condition, but not yet to the extent that the use of the more challenging formal strategies paid off in terms of improved correctness. Regarding conceptual knowledge, it might be that the anticipatory use of diagrams in the ITS focused students primarily on strategic issues, as the diagrams were used in planning problem-solving steps. It may be that students need to engage in "principle-based explanation" (Renkl, 1997) to facilitate acquisition of conceptual knowledge (e.g., verbally explaining why the selected diagram is correct and strategic). It might also be that students with varying levels of prior knowledge benefit from diagrammatic self-explanation differently (Booth & Koedinger, 2012). Future studies should examine the effects of anticipatory diagrammatic self-explanation with students having varying degrees of prior knowledge and experience in algebra.
Our study has several limitations. First, the study was conducted with one specific type of diagrams, tape diagrams, and it focused on one specific task domain, equation solving in algebra. To understand how the results could generalize across domains and types of visual representations, more research is needed to examine the effects of anticipatory diagrammatic self-explanations. Also, it is possible that students were not very motivated to work on the posttest, especially given that the study was conducted remotely during the COVID-19 pandemic. This may have contributed to the absence of pretest-posttest gains in procedural knowledge, even though the learning curves suggest that learning occurred.
In summary, we designed a novel self-explanation scaffolding support for students in middle-school algebra, namely, anticipatory diagrammatic self-explanation. We investigated the effectiveness of this support embedded in an Intelligent Tutoring System, in a classroom study. We found that anticipatory diagrammatic self-explanation helped students learn formal algebraic strategies and perform better on problem solving, while making similar conceptual gains as students who did not receive the support. Our study contributes to the theoretical and practical understanding of how visual representations, contrasting cases, and anticipatory selfexplanation can be integrated into scaffolding support that helps students learn and perform effectively and efficiently.