More Like this

1. Introducing desirable difficulty in engineering mathematics with spaced retrieval practice

   Bego, Campbell R. ; Ralston, Patricia A. ; Lyle, Keith B. ; Immekus, Jason C. ( July 2021 , Proceedings of the 128th ASEE Annual Conference and Exposition)

   null (Ed.)

   This study examined the difficulty introduced by spaced retrieval practice in Calculus I for undergraduate engineering students. Spaced retrieval practice is an instructional technique in which students engage in multiple recall exercises on the same topic with intermittent temporal delays in between. Spacing out retrieval practice increases the difficulty of the exercises, reducing student performance on them. However, empirical research indicates that spaced retrieval practice is associated with improvements in students’ long-term memory for the retrieved information. The short-term costs and long-term benefits of spaced retrieval practice is an example of desirable difficulty, when more difficult exercises during the early stages of learning result in longer-lasting memory [1]. With support from the National Science Foundation (NSF), we sought to address: Does spacing decrease performance on retrieval practice exercises in an engineering mathematics course? Results showed that student performance was significantly lower for questions in the spaced condition than questions in the massed condition, indicating that we successfully increased the difficulty of the questions by spacing them out over time. Future work will assess final quiz performance to determine whether spacing improved long-term course performance, i.e., whether the difficulty imposed by spacing was desirable. 

   more » « less
2. Single-paper meta-analyses of the effects of spaced retrieval practice in nine introductory STEM courses: is the glass half full or half empty?

   https://doi.org/10.1186/s40594-024-00468-5  

   Bego, Campbell R. ; Lyle, Keith B. ; Ralston, Patricia A. S. ; Immekus, Jason C. ; Chastain, Raymond J. ; Haynes, Lora D. ; Hoyt, Lenore K. ; Pigg, Rachel M. ; Rabin, Shira D. ; Scobee, Matthew W. ; et al ( February 2024 , International Journal of STEM Education)

   Undergraduate STEM instructors want to help students learn and retain knowledge for their future courses and careers. One promising evidence-based technique that is thought to increase long-term memory is spaced retrieval practice, or repeated testing over time. The beneficial effect of spacing has repeatedly been demonstrated in the laboratory as well as in undergraduate mathematics courses, but its generalizability across diverse STEM courses is unknown. We investigated the effect of spaced retrieval practice in nine introductory STEM courses. Retrieval practice opportunities were embedded in bi-weekly quizzes, either massed on a single quiz or spaced over multiple quizzes. Student performance on practice opportunities and a criterial test at the end of each course were examined as a function of massed or spaced practice. We also conducted a single-paper meta-analysis on criterial test scores to assess the generalizability of the effectiveness of spaced retrieval practice across introductory STEM courses.

   Significant positive effects of spacing on the criterial test were found in only two courses (Calculus I for Engineers and Chemistry for Health Professionals), although small positive effect sizes were observed in two other courses (General Chemistry and Diversity of Life). Meta-analyses revealed a significant spacing effect when all courses were included, but not when calculus was excluded. The generalizability of the spacing effect across STEM courses therefore remains unclear.

   Although we could not clearly determine the generalizability of the benefits of spacing in STEM courses, our findings indicate that spaced retrieval practice could be a low-cost method of improving student performance in at least some STEM courses. More work is needed to determine when, how, and for whom spaced retrieval practice is most beneficial. The effect of spacing in classroom settings may depend on some design features such as the nature of retrieval practice activities (multiple-choice versus short answer) and/or feedback settings, as well as student actions (e.g., whether they look at feedback or study outside of practice opportunities). The evidence is promising, and further pragmatic research is encouraged.

    

   more » « less
3. Using SQL to query the difficulty imposed by spaced retrieval in engineering mathematics

   https://doi.org/10.1109/FIE49875.2021.9637171  

   Boyd, Jeremy ; Bego, Campbell ; Garcia, Osvaldo ; Ralston, Patricia ; Immekus, Jason ; Lyle Keith ( January 2021 , 2021 IEEE Frontiers in Education Conference (FIE))

   This WIP Research Paper investigates the temporal nature of the difficulty imposed by spacing. Spaced retrieval practice is an evidence-based strategy for improving memory and consists of asking multiple questions on a topic with intermittent delays. Spacing is often thought to impose difficulty by making questions harder to answer. However, this difficulty may be desirable, since spacing ultimately improves memory. In this paper, we (a) outline an implementation of spaced retrieval practice in an engineering mathematics classroom, (b) describe the development of an SQL database to organize and manage the large and complex dataset, and (c) discuss a brief but interesting dive into the rich data we have collected. Statistical analyses revealed that, when three questions targeting the same topic are spaced over multiple quizzes, versus being massed on a single quiz, students are less likely to answer the first and second questions correctly. Spacing does not affect students' ability to answer the third questions. This suggests that spacing may impose difficulty when students are first learning to perform mathematical operations, rather than when they are trying to retrieve memories of how to perform those operations. 

   more » « less
4. Spaced Retrieval Practice Imposes Desirable Difficulty in Calculus Learning

   https://doi.org/10.1007/s10648-022-09677-2  

   Lyle, Keith B. ; Bego, Campbell R. ; Ralston, Patricia A. S. ; Immekus, Jason C. ( April 2022 , Educational Psychology Review)

   After being taught how to perform a new mathematical operation, students are often given several practice problems in a single set, such as a homework assignment or quiz (i.e., massed practice). An alternative approach is to distribute problems across multiple homeworks or quizzes, increasing the temporal interval between practice (i.e., spaced practice). Spaced practice has been shown to increase the long-term retention of various types of mathematics knowledge. Less clear is whether spacingdecreasesperformance during practice, with some studies indicating that it does and others indicating it does not. To increase clarity, we tested whether spacing produces long-term retention gains, but short-term practice costs, in a calculus course. On practice quizzes, students worked problems on various learning objectives in either massed fashion (3 problems on a single quiz) or spaced fashion (3 problems across 3 quizzes). Spacing increased retention of learning objectives on an end-of-semester test but reduced performance on the practice quizzes. The reduction in practice performance was nuanced: Spacing reduced performance only on the first two quiz questions, leaving performance on the third question unaffected. We interpret these findings as evidence that spacing led to more protracted, but ultimately more robust, learning. We, therefore, conclude that spacing imposes a desirable form of difficulty in calculus learning.

    

   more » « less
5. Adaptive vs. fixed spacing of learning items: Evidence from studies of learning and transfer in chemistry education

   Mettler, E. ; El-Ashmawy, A. K. ; Massey, C. M. ; & Kellman, P. J. ( July 2020 , Proceedings of the Annual Conference of the Cognitive Science Society)

   Denison, S. ; Mack, M. ; Xu, Y. ; Armstrong, B. C. (Ed.)

   Spacing presentations of learning items across time improves memory relative to massed schedules of practice – the well-known spacing effect. Spaced practice can be further enhanced by adaptively scheduling the presentation of learning items to deliver customized spacing intervals for individual items and learners. ARTS - Adaptive Response-time-based Sequencing (Mettler, Massey, & Kellman 2016) determines spacing dynamically in relation to each learner’s ongoing speed and accuracy in interactive learning trials. We demonstrate the effectiveness of ARTS when applied to chemistry nomenclature in community college chemistry courses by comparing adaptive schedules to fixed schedules consisting of continuously expanding spacing intervals. Adaptive spacing enhanced the efficiency and durability of learning, with learning gains persisting after a two-week delay and generalizing to a standardized assessment of chemistry knowledge after 2-3 months. Two additional experiments confirmed and extended these results in both laboratory and community college settings. 

   more » « less