An official website of the United States government
We conducted a controlled study to investigate whether having students choose the concept on which to solve each practice problem in an adaptive tutor helped improve learning. We analyzed data from an adaptive tutor used by introductory programming students over three semesters. The tutor presented code-tracing problems, used pretest-practice-post-test protocol, and presented line-by-line explanation of the correct solution as feedback. We found that choice did not in-crease the amount of learning or pace of learning. But, it resulted in greater improvement in score on learned concepts, and the effect size was medium.
more »
« less
Providing the Option to Skip Feedback – A Reproducibility Study
Would providing choice lead to improved learning with a tutor? We had conducted and reported a controlled study earlier, wherein, introductory programing students were given the choice of skipping the line-by-line feedback provided after each incorrect answer in a tutor on if/if-else statements. Contrary to expectations, the study found that the choice to skip feedback did not lead to greater learning. We tried to reproduce these results using two tutors on if/if-else and switch statements, and with a larger subject pool. We found that whereas choice did not lead to greater learning on if/if-else tutor in this reproducibility study either, it resulted in decreased learning on switch tutor. We hypothesize that skipping feedback is indeed detrimental to learning. But, inter-relationships among the concepts covered by a tutor and the transfer of learning facilitated by these relationships compensate for the negative effect of skipping line-by-line feedback. We also found contradictory results between the two studies which highlight the need for reproducibility studies in empirical research.
more »
« less
- Award ID(s):
- 1432190
- PAR ID:
- 10097974
- Date Published:
- Journal Name:
- Intelligent Tutoring Systems
- Page Range / eLocation ID:
- 180-185
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
A study was conducted to reproduce the results of an earlier study on the effectiveness of visualization for learning expression evaluation in a problem-solving software tutor on arithmetic expressions. In the current reproducibility study, data was collected from a software tutor on assignment expressions over six semesters. ANOVA analysis of the amount and speed of learning was conducted with treatment, sex and racial groups as fixed factors. Results include that visualization helped the students learn significantly more concepts, whether the students needed to use the tutor or benefited from using the tutor. However, it only benefited the less-prepared students. It did not help the students learn faster. It benefited both the sexes and traditionally represented as well as underrepresented groups. The current study confirmed almost all the results from the previous study, albeit for a harder topic. One reason why visualization was found to be effective in both these studies may be that the same visualization scheme was used by the students to both view feedback and construct their answers.more » « less
-
Do students retain the programming concepts they have learned using software tutors over the long term? In order to answer this question, we analyzed the data collected by a software tutor on selection statements. We used the data of the students who used the tutor more than once to see whether they had retained for the second session what they had learned during the first session. We found that students retained over 71% of selection concepts that they had learned during the first session. The more problems students solved during the first session, the greater the percentage of retention. Even when students already knew a concept and did not benefit from using the tutor, a small percentage of concepts were for-gotten from the first session to the next, corresponding to transience of learning. Transience of learning varied with concepts. We list confounding factors of the study.more » « less
-
Reported here are the findings of a comparative study on the effects of using a Socratic Intelligent Tutoring System for source code comprehension and learning computer programming. The result shows there are significant differences between the two groups where students who used Socratic Tutor ITS improved their knowledge by 45% in term of learning gain, developed a better understanding of concepts such as nested if-else and for loop, and improved their confidence level by 13%. Furthermore, the result of the Pearson product-moment correlation coefficient shows a positive correlation (r = 0.68) between feedback from the ITS and learning gain.more » « less
-
Reported here are the findings of a comparative study on the effects of using a Socratic Intelligent Tutoring System for source code comprehension and learning computer programming. The result shows there are significant differences between the two groups where students who used Socratic Tutor ITS improved their knowledge by 45% in term of learning gain, developed a better understanding of concepts such as nested if-else and for loop, and improved their confidence level by 13%. Furthermore, the result of the Pearson product-moment correlation coefficient shows a positive correlation (r = 0.68) between feedback from the ITS and learning gain.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Conference Paper:
- The DOI is not currently available.
