Search for: All records

Accuracy on assessments is commonly studied in education research but response time (RT) is relatively less investigated even though decades of research in cognitive sciences indicate that time can be an important dimension for understanding student learning. To better understand RT and the potentially important relations between accuracy and RT in physics education, we conducted an exploratory investigation by collecting and analyzing both accuracy and RT data on physics-relevant math skills on low-stakes pre and posttests as well as course exam scores in algebra-based and calculus-based introductory physics courses over two semesters for a total of $N=1936$participants. Overall, we found a high level of variation in response times revealing weak but consistent patterns of associations between RT and accuracy on skills and exam scores. First, we found a nonlinear relationship between RT and accuracy on the pretest and on the post-test, which may indicate a variety of strategies and engagement among students on these participation-credit-only tests. Second, the results indicate that while RT alone does not predict course grade, when controlling for accuracy on pre or posttest math skills, students with lower RT on these skills are more likely to get better grades. Therefore, both pre or posttest accuracy and speed predicted course grades, though accuracy explained a substantial amount of variance ( $\sim 35\%$) while pretest RT explained a much smaller amount of variance ( $\sim 1\%$). Third, controlling for both pretest accuracy and pretest RT, we found that students who sped up from pre to posttest were likely to get higher exam scores; however, students who slowed down were on average likely to have a higher post-test score. Fourth, since systemic inequities in STEM education have been documented via measured mean differences between some demographic groups for exam scores and accuracy on math skills, we compared RTs by sex, race, first-generation status, and citizenship to potentially gain more insight into these inequities. We found no consistent or conclusive evidence of demographic differences, though in multiple comparisons, Black, Hispanic, Native American, and Pacific Islander students had larger RTs on average, and in one comparison they were slightly faster. We found that RT was not a mediator of demographic differences in physics grades, though, as expected, accuracy on math skills was a mediator. We briefly discuss how our results relate to various cognitive models such as cognitive ability, speed-accuracy trade-offs, fluency and cognitive load, dual-process theories, and student psychological factors like self-efficacy, anxiety, and motivation. We argue that, based on which (if any) of the above mechanisms are at play, valuing speed in physics may have benefits, such as improving fluency to reduce cognitive load and drawbacks, such as unintentionally using speed as a proxy for achievement or inducing excessive stress that may interfere with performance and student well-being. 

We conducted two studies to investigate the extent to which brief, spaced, mastery practice on skills relevant to introductory physics affects student performance. The first study investigated the effect of practice of “specific” physics skills, each one relevant to only one or a few items on the course exam. This study employed a quasiexperimental design with 766 students assigned to “intervention” or “control” conditions by lecture section sharing common exams. Results of the first study indicate significant improvement in the performance for only some of the exam items relevant to the specific skills practiced. We also observed between-section performance differences on other exam items not relevant to training, which may be due to specific prior quiz items from individual instructors. The second study investigated the effect of practice on the “general” skill of algebra relevant to introductory physics, a skill which was relevant to most of the exam items. This study employed a similar quasiexperimental design with 363 students assigned to treatment or control conditions, and we also administered a reliable pre- and post-test assessment of the algebra skills that was iteratively developed for this project. Results from the second study indicate that 75% of students had high accuracy on the algebra pretest. Students in the control condition who scored low on the pretest gained about 0.7 standard deviations on the post-test, presumably from engagement with the course alone, and students in the algebra practice condition had statistically similar gains, indicating no observed effect of algebra practice on algebra pre- to post-test gains. In contrast, we find some potential evidence that the algebra practice improved final exam performance for students with high pretest scores and did not benefit students with low pretest scores, although this result is inconclusive: the point estimate of the effect size was 0.24 for high pretest scoring students, but the 95% confidence interval [ $-0.01$, 0.48] slightly overlapped with zero. Further, we find a statistically significant positive effect of algebra practice on exam items that have higher algebraic complexity and no effect for items with low complexity. One possible explanation for the added benefit of algebra practice for high-scoring students is fluency in algebra skills may have improved. Overall, our observations provide some evidence that spaced, mastery practice is beneficial for exam performance for specific and general skills, and that students who are better prepared in algebra may be especially benefitting from mastery practice in relevant algebra skills in terms of improved final exam performance. 

We have investigated the temporal patterns of algebra (N ¼ 606) and calculus (N ¼ 507) introductory physics students practicing multiple basic physics topics several times throughout the semester using an online mastery homework application called science, technology, engineering, and mathematics (STEM) fluency aimed at improving basic physics skills. For all skill practice categories, we observed an increase in measures of student accuracy, such as a decrease in the number of questions attempted to reach mastery, and a decrease in response time per question, resulting in an overall decrease in the total time spent on the assignments. The findings in this study show that there are several factors that impact a student’s performance and evolution on the mastery assignments throughout the semester. For example, using linear mixed modeling, we report that students with lower math preparation for the physics class start with lower accuracy and slower response times on the mastery assignments than students with higher math preparation. However, by the end of the semester, the less prepared students reach similar performance levels to their more prepared classmates on the mastery assignments. This suggests that STEM fluency is a useful tool for instructors to implement to refresh student’s basic math skills. Additionally, gender and procrastination habits impact the effectiveness and progression of the student’s response time and accuracy on the STEM fluency assignments throughout the semester. We find that women initially answer more questions in the same amount of time as men before reaching mastery. As the semester progresses and students practice the categories more, this performance gap diminishes between males and females. In addition, we find that students who procrastinate (those who wait until the final few hours to complete the assignments) are spending more time on the assignments despite answering a similar number of questions as compared to students who do not procrastinate. We also find that student mindset (growth vs fixed mindset) was not related to a student’s progress on the online mastery assignments. Finally, we find that STEM fluency practice improves performance beyond the effects of other components of instruction, such as lectures, group-work recitations, and homework assignments.