More Like this

1. Evolution of response time and accuracy on online mastery practice assignments for introductory physics students

   https://doi.org/10.1103/PhysRevPhysEducRes.19.020111  

   Nieberding, Megan; Heckler, Andrew F. (August 2023, Physical Review Physics Education Research)

   We have investigated the temporal patterns of algebra (N ¼ 606) and calculus (N ¼ 507) introductory physics students practicing multiple basic physics topics several times throughout the semester using an online mastery homework application called science, technology, engineering, and mathematics (STEM) fluency aimed at improving basic physics skills. For all skill practice categories, we observed an increase in measures of student accuracy, such as a decrease in the number of questions attempted to reach mastery, and a decrease in response time per question, resulting in an overall decrease in the total time spent on the assignments. The findings in this study show that there are several factors that impact a student’s performance and evolution on the mastery assignments throughout the semester. For example, using linear mixed modeling, we report that students with lower math preparation for the physics class start with lower accuracy and slower response times on the mastery assignments than students with higher math preparation. However, by the end of the semester, the less prepared students reach similar performance levels to their more prepared classmates on the mastery assignments. This suggests that STEM fluency is a useful tool for instructors to implement to refresh student’s basic math skills. Additionally, gender and procrastination habits impact the effectiveness and progression of the student’s response time and accuracy on the STEM fluency assignments throughout the semester. We find that women initially answer more questions in the same amount of time as men before reaching mastery. As the semester progresses and students practice the categories more, this performance gap diminishes between males and females. In addition, we find that students who procrastinate (those who wait until the final few hours to complete the assignments) are spending more time on the assignments despite answering a similar number of questions as compared to students who do not procrastinate. We also find that student mindset (growth vs fixed mindset) was not related to a student’s progress on the online mastery assignments. Finally, we find that STEM fluency practice improves performance beyond the effects of other components of instruction, such as lectures, group-work recitations, and homework assignments. 

   more » « less
2. Algebra and other relevant physics skills: The effectiveness of mastery practice on skills accuracy and exam grade in introductory physics

   https://doi.org/10.1103/PhysRevPhysEducRes.21.010121  

   Liu, Qiaoyi; Moni_Prakash, Harish; Heckler, Andrew F (March 2025, Physical Review Physics Education Research)

   NA (Ed.)

   We conducted two studies to investigate the extent to which brief, spaced, mastery practice on skills relevant to introductory physics affects student performance. The first study investigated the effect of practice of “specific” physics skills, each one relevant to only one or a few items on the course exam. This study employed a quasiexperimental design with 766 students assigned to “intervention” or “control” conditions by lecture section sharing common exams. Results of the first study indicate significant improvement in the performance for only some of the exam items relevant to the specific skills practiced. We also observed between-section performance differences on other exam items not relevant to training, which may be due to specific prior quiz items from individual instructors. The second study investigated the effect of practice on the “general” skill of algebra relevant to introductory physics, a skill which was relevant to most of the exam items. This study employed a similar quasiexperimental design with 363 students assigned to treatment or control conditions, and we also administered a reliable pre- and post-test assessment of the algebra skills that was iteratively developed for this project. Results from the second study indicate that 75% of students had high accuracy on the algebra pretest. Students in the control condition who scored low on the pretest gained about 0.7 standard deviations on the post-test, presumably from engagement with the course alone, and students in the algebra practice condition had statistically similar gains, indicating no observed effect of algebra practice on algebra pre- to post-test gains. In contrast, we find some potential evidence that the algebra practice improved final exam performance for students with high pretest scores and did not benefit students with low pretest scores, although this result is inconclusive: the point estimate of the effect size was 0.24 for high pretest scoring students, but the 95% confidence interval [ $-0.01$, 0.48] slightly overlapped with zero. Further, we find a statistically significant positive effect of algebra practice on exam items that have higher algebraic complexity and no effect for items with low complexity. One possible explanation for the added benefit of algebra practice for high-scoring students is fluency in algebra skills may have improved. Overall, our observations provide some evidence that spaced, mastery practice is beneficial for exam performance for specific and general skills, and that students who are better prepared in algebra may be especially benefitting from mastery practice in relevant algebra skills in terms of improved final exam performance. 

   more » « less
3. Response time on math skills and its association with math skills accuracy and physics course grades

   https://doi.org/10.1103/PhysRevPhysEducRes.21.010144  

   Moni_Prakash, Harish; Heckler, Andrew F (April 2025, Physical Review Physics Education Research)

   Accuracy on assessments is commonly studied in education research but response time (RT) is relatively less investigated even though decades of research in cognitive sciences indicate that time can be an important dimension for understanding student learning. To better understand RT and the potentially important relations between accuracy and RT in physics education, we conducted an exploratory investigation by collecting and analyzing both accuracy and RT data on physics-relevant math skills on low-stakes pre and posttests as well as course exam scores in algebra-based and calculus-based introductory physics courses over two semesters for a total of $N=1936$participants. Overall, we found a high level of variation in response times revealing weak but consistent patterns of associations between RT and accuracy on skills and exam scores. First, we found a nonlinear relationship between RT and accuracy on the pretest and on the post-test, which may indicate a variety of strategies and engagement among students on these participation-credit-only tests. Second, the results indicate that while RT alone does not predict course grade, when controlling for accuracy on pre or posttest math skills, students with lower RT on these skills are more likely to get better grades. Therefore, both pre or posttest accuracy and speed predicted course grades, though accuracy explained a substantial amount of variance ( $\sim 35\%$) while pretest RT explained a much smaller amount of variance ( $\sim 1\%$). Third, controlling for both pretest accuracy and pretest RT, we found that students who sped up from pre to posttest were likely to get higher exam scores; however, students who slowed down were on average likely to have a higher post-test score. Fourth, since systemic inequities in STEM education have been documented via measured mean differences between some demographic groups for exam scores and accuracy on math skills, we compared RTs by sex, race, first-generation status, and citizenship to potentially gain more insight into these inequities. We found no consistent or conclusive evidence of demographic differences, though in multiple comparisons, Black, Hispanic, Native American, and Pacific Islander students had larger RTs on average, and in one comparison they were slightly faster. We found that RT was not a mediator of demographic differences in physics grades, though, as expected, accuracy on math skills was a mediator. We briefly discuss how our results relate to various cognitive models such as cognitive ability, speed-accuracy trade-offs, fluency and cognitive load, dual-process theories, and student psychological factors like self-efficacy, anxiety, and motivation. We argue that, based on which (if any) of the above mechanisms are at play, valuing speed in physics may have benefits, such as improving fluency to reduce cognitive load and drawbacks, such as unintentionally using speed as a proxy for achievement or inducing excessive stress that may interfere with performance and student well-being. 

   more » « less
4. (2023). Assessment of a Particle Sedimentation Hands-On Learning Tool with Application in Blood Cell Separations. Education for Chemical Engineers, On-Line, 28-40. doi:https://doi.org/10.1016/j.ece.2023.07.001

   Kaiphanliam, K. M. (July 2023, Education for chemical engineers)

   Chemical engineers frequently contribute to the advancement of the medical field; however, medical applications are often only covered in elective courses. To introduce medical applications into the core curriculum, we implemented a hands-on learning tool that portrays blood separation principles through microbead settling in a core third-year chemical engineering separations class. Test scores from twenty-six students show significant growth at p < 0.001 from Pretest to Posttest I at average values of 41 % and 68 %, respectively. Posttest II scores reveal a significantly higher average score of 84 % for students who sat through lecture before the hands-on experiment in comparison to 75 % for students who first had the hands-on experiment then lecture with statistical significance of p = 0.046 and a moderate Cohen’s d effect size of 0.442. Students report positive, lasting impressions from the guided-learning worksheet and hands-on learning experience on their feedback surveys and one-on-one interviews. Retention assessments from four students six months post-intervention reveal retention of concepts with an average test score of 74 %. These outcomes suggest hands-on learning tools are most impactful on conceptual and motivational gains when supplemented with pre-experiment lectures and quality complementary learning materials. 

   more » « less
5. From Sleep Patterns to Heart Rhythms: Predicting Atrial Fibrillation from Overnight Polysomnograms

   https://doi.org/10.1101/2024.06.04.24308444  

   Koscova, Zuzana; Rad, Ali Bahrami; Nasiri, Samaneh; Reyna, Matthew A; Sameni, Reza; Trotti, Lynn M; Sun, Haoqi; Turley, Niels; Stone, Katie L; Thomas, Robert J; et al (June 2024, medRxiv)

   Abstract BackgroundAtrial fibrillation (AF) is often asymptomatic and thus under-observed. Given the high risks of stroke and heart failure among patients with AF, early prediction and effective management are crucial. Importantly, obstructive sleep apnea is highly prevalent among AF patients (60–90%); therefore, electrocardiogram (ECG) analysis from polysomnography (PSG), a standard diagnostic tool for subjects with suspected sleep apnea, presents a unique opportunity for the early prediction of AF. Our goal is to identify individuals at a high risk of developing AF in the future from a single-lead ECG recorded during standard PSGs. MethodsWe analyzed 18,782 single-lead ECG recordings from 13,609 subjects at Massachusetts General Hospital, identifying AF presence using ICD-9/10 codes in medical records. Our dataset comprises 15,913 recordings without a medical record for AF and 2,056 recordings from patients who were first diagnosed with AF between 1 day to 15 years after the PSG recording. The PSG data were partitioned into training, validation, and test cohorts. In the first phase, a signal quality index (SQI) was calculated in 30-second windows and those with SQI<0.95 were removed. From each remaining window, 150 hand-crafted features were extracted from time, frequency, time-frequency domains, and phase-space reconstructions of the ECG. A compilation of 12 statistical features summarized these window-specific features per recording, resulting in 1,800 features. We then updated a pre-trained deep neural network and data from the PhysioNet Challenge 2021 using transfer-learning to discriminate between recordings with and without AF using the same Challenge data. The model was applied to the PSG ECGs in 16-second windows to generate the probability of AF for each window. From the resultant probability sequence, 13 statistical features were extracted. Subsequently, we trained a shallow neural network to predict future AF using the extracted ECG and probability features. ResultsOn the test set, our model demonstrated a sensitivity of 0.67, specificity of 0.81, and precision of 0.3 for predicting AF. Further, survival analysis for AF outcomes, using the log-rank test, revealed a hazard ratio of 8.36 (p-value of 1.93 × 10⁻⁵²). ConclusionsOur proposed ECG analysis method, utilizing overnight PSG data, shows promise in AF prediction despite a modest precision indicating the presence of false positive cases. This approach could potentially enable low-cost screening and proactive treatment for high-risk patients. Ongoing refinement, such as integrating additional physiological parameters could significantly reduce false positives, enhancing its clinical utility and accuracy. 

   more » « less