An official website of the United States government
Preparing for high-stakes exams in introductory physics courses is generally a self-regulated activity. Compared to other exam reviewing strategies, doing practice exams has been shown to help students recognize gaps in their knowledge, encourage active practicing, and produce long-term retention. However, many students, particularly students who are struggling with the course material, are not guided by research-based study strategies and do not use practice exams effectively. Using data collected from a fully online course in Spring 2021, this study examines two interventions aimed at improving student selfregulated studying behaviors and enhancing student metacognition during exam preparation. We found that a modified format of online practice exams with one attempt per question and delayed feedback, increases the accuracy of feedback about student readiness for exams but does not change the accuracy of their predicted exam scores or studying behaviors. Additionally, an added mock exam one week before the actual exam impacts students’ intentions for studying but does not impact actual study behaviors or facilitate metacognition. These results suggest that interventions designed to improve exam preparation likely need to include explicit instruction on study strategies and student beliefs about learning.
more »
« less
A Quantitative Analysis of When Students Choose to Grade Questions on Computerized Exams with Multiple Attempts
In this paper, we study a computerized exam system that allows students to attempt the same question multiple times. This system permits students either to receive feedback on their submitted answer immediately or to defer the feedback and grade questions in bulk. An analysis of student behavior in three courses across two semesters found similar student behaviors across courses and student groups. We found that only a small minority of students used the deferred feedback option. A clustering analysis that considered both when students chose to receive feedback and either to immediately retry incorrect problems or to attempt other unfinished problems identified four main student strategies. These strategies were correlated to statistically significant differences in exam scores, but it was not clear if some strategies improved outcomes or if stronger students tended to prefer certain strategies.
more »
« less
- Award ID(s):
- 1915257
- PAR ID:
- 10200296
- Date Published:
- Journal Name:
- Proceedings of the Seventh ACM Conference on Learning @ Scale
- Page Range / eLocation ID:
- 329 to 332
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Henderson, Charles (Ed.)Preparing for high-stakes exams in introductory physics courses is generally a self-regulated activity. Compared to other exam reviewing strategies, doing practice exams has been shown to help students recognize gaps in their knowledge, encourage active practicing, and produce long-term retention. However, many students, particularly students who are struggling with the course material, are not guided by research-based study strategies and do not use practice exams effectively. Using data collected from a fully online course in Spring 2021, this study examines two interventions aimed at improving student selfregulated studying behaviors and enhancing student metacognition during exam preparation. We found that a modified format of online practice exams with one attempt per question and delayed feedback, increases the accuracy of feedback about student readiness for exams but does not change the accuracy of their predicted exam scores or studying behaviors. Additionally, an added mock exam one week before the actual exam impacts students’ intentions for studying but does not impact actual study behaviors or facilitate metacognition. These results suggest that interventions designed to improve exam preparation likely need to include explicit instruction on study strategies and student beliefs about learning.more » « less
-
Students often find biology courses to be very difficult and isolating, particularly if they identify as part of a group that has been historically excluded from STEM. Some of this anxiety and isolation comes from high-stakes exams. We decided to use the collaborative structure of two-stage exams to try to overcome the isolation of assessment. In two-stage exams, students take an individual exam, and then immediately get into groups and take the exam again, discussing the questions and the rationale behind the answers. Their exam scores are a combination of the two attempts. Our move to emergency online learning because of the COVID-19 pandemic forced us to try our two-stage exams online. In this Teaching Tools and Strategies essay, we discuss our process of offering two-stage exams online at two different institutions: a two-year Community College and four-year Research University. We share feedback from the students and discuss our iterative improvements to two-stage exam use.more » « less
-
Introductory engineering courses at large universities often number over a hundred students, while online classes can have even larger enrollments, significantly constraining instructors’ ability to provide feedback on homework, including the free-body diagrams (FBDs). Most online homework systems do not provide feedback on FBDs if the systems even allow the submission, and instructors often lack time or resources to provide this. A few systems have been developed that use a menu-based system allowing students to creative FBDs. There is a growing concern amongst engineering educators that student lacks critical sketching skills and the ability to idealize a real-world system as a free body diagram (FBD). A sketch-recognition based tutoring system, Mechanix, allows learners to hand-draw solutions just as they would with pencil and paper, while also providing iterative real-time personalized feedback. Sketch recognition algorithms use artificial intelligence to identify the shapes, their relationships, and other features of the sketched student drawing. Other AI algorithms then determine if and why a student’s work is incorrect, enabling the tutoring system to return immediate and iterative personalized feedback facilitating student learning that is otherwise not possible in large classes. Preliminary results using Mechanix, a sketch-based statics tutoring system built at Texas A&M University suggest that a sketch-based tutoring system increases homework motivation in struggling students and is as effective as paper-and-pencil-based homework for teaching method of joints truss analysis. The current project implements Mechanix at five different universities obtaining Pre/Post Concept Inventory, homework, and exam scores. It is compared against either the university's current online system or paper-based homework. Focus groups provide further insight into the students’ perceptions about the impact of Mechanix on their learning.more » « less
-
Introductory engineering courses within large universities often have annual enrollments exceeding several hundreds of students, while online classes have even larger enrollments. It is challenging to achieve differentiated instruction in classrooms with class sizes and student diversity of such great magnitude. In such classes, professors assess whether students have mastered a concept through multiple-choice questions, marking answers as right or wrong with little feedback, or using online text-only systems. However, in these scenarios the feedback is of a mostly binary nature (right or wrong) with limited constructive feedback to scaffold learning. A growing concern among engineering educators is that students are losing both the critical skill of sketched diagrams and the ability to take a real system and reduce it to an accurate but simplified free-body diagram (FBD). A sketch-recognition based tutoring system, called Mechanix, allows students to hand-draw solutions just as they would with pencil and paper, while also providing iterative real-time personalized feedback. Sketch recognition algorithms use artificial intelligence to identify the shapes, their relationships, and other features of the sketched student drawing. Other AI algorithms then determine if and why a student’s work is incorrect, enabling the tutoring system to return immediate and iterative personalized feedback facilitating student learning that is otherwise not possible in large classes. To observe the effectiveness of this system, it has been implemented into various courses at three universities, with two additional universities planning to use the system within the next year. Student knowledge is measured using Concept Inventories based in both Physics and Statics, common exam questions, and assignments turned in for class. Preliminary results using Mechanix, a sketch-based statics tutoring system built at Texas A&M University, suggest that a sketch-based tutoring system increases homework motivation in struggling students and is as effective as paper-and-pencil-based homework for teaching method of joints truss analysis. In focus groups, students believed the system enhanced their learning and increased engagement. Keywords: sketch recognition; intelligent user interfaces; physics education; engineering educationmore » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3386527.3406740
