Search for: All records

This paper presents the design and analysis of a pilot problem set deployed to engineering students to assess their retention of physics knowledge at the start of a statics course. The problem set was developed using the NSF-IUSE (grant #2315492) Learning Map project (LMap) and piloted in the spring and fall of 2024. The LMap process is rooted in the Analysis, Design, Development, Implementation, and Evaluation (ADDIE) model [1] and Backward Design [2,3], extending these principles to course sequences to align learning outcomes, assessments, and instructional practices. The primary motivation for this problem set (Statics Knowledge Inventory, SKI) was to evaluate students' understanding and retention of physics concepts at the beginning of a statics course. The SKI includes a combination of multiple-choice questions (MCQ) and procedural problems, filling a gap in widely-used concept inventories for physics and statics, such as the Force Concept Inventory (FCI) and Statics Concept Inventory (SCI), which evaluate learning gains within a course, rather than knowledge retention across courses. Using the LMap analysis and instructor consultations, we identified overlapping concepts and topics between Physics and Statics courses, referred to here as “interdependent learning outcomes” (ILOs). The problem set includes 15 questions—eight MCQs and seven procedural problems. Unlike most concept inventories, procedural problems were added to provide insight into students’ problem-solving approach and conceptual understanding. These problems require students to perform calculations, demonstrate their work, and assess their conceptual understanding of key topics, and allow the instructors to assess essential prerequisite skills like drawing free-body diagrams (FBDs), computing forces and moments, and performing basic vector calculation and unit conversions. Problems were selected and adapted from physics and statics textbooks, supplemented by instructor-designed questions to ensure full coverage of the ILOs. We used the revised 2D Bloom’s Taxonomy [4] and a 3D representation of it [5] to classify each problem within a 6x4 matrix (six cognitive processes x four knowledge dimensions). This classification provided instructors and students with a clear understanding of the cognitive level required for each problem. Additionally, we measured students’ perceived confidence and difficulty in each problem using two questions on a 3-point Likert scale. The first iteration of the problem set was administered to 19 students in the spring 2024 statics course. After analyzing their performance, we identified areas for improvement and revised the problem set, removing repetitive MCQs and restructuring the procedural problems into scaffolded, multi-part questions with associated rubrics for evaluation. The revised version, consisting of five MCQs and six procedural problems, was deployed to 136 students in the fall 2024 statics course. A randomly selected subset of student answers from the second iteration was graded and analyzed to compare with the first. This analysis will inform future efforts to evaluate knowledge retention and transfer in key skills across sequential courses. In collaboration with research teams developing concept inventories for mechanics courses, we aim to integrate these procedural problems into future inventories. 

This project supports the success of undergraduate engineering students through coordinated design of curricula across STEM course sequences. The Analysis, Design, Development, Implementation, Evaluation (ADDIE) framework and backward design are being used to develop guides for instructors to align learning outcomes, assessments, and instructional materials in a physics – engineering mechanics course sequence. The approach relies on the analysis of student learning outcomes in each course, identification of interdependent learning outcomes, and development of skills hierarchies in the form of visual learning maps. The learning maps are used to illustrate the knowledge required and built upon throughout the course sequence. This study will assess the effectiveness of a course redesign intervention, which uses visual learning maps and backward design concepts, to guide instructors within a common course sequence to align learning outcomes and assessments. If successful, the intervention is expected to improve students’ primary learning and knowledge retention, as well as persistence and success in the degree. The study will compare academic performance among Mechanical Engineering B.S., Environmental Engineering B.S., and Civil Engineering B.S. students who begin a Physics for Engineers – Statics – Dynamics course prior to the intervention (control) and after the intervention (treatment). During control and treatment terms, students’ primary learning in individual courses will be assessed using established concept inventories. Retention of knowledge from pre-requisite courses will be tracked using pre-identified problem sets (quizzes, exams) specifically associated with interdependent learning outcomes in the Statics and Dynamics courses. Students’ primary learning and knowledge retention in the sequence will be related to longer term student success outcomes, including retention and graduation. The poster will show the results of the research team’s first year of work, including an analysis of current course materials, learning maps for each course, identification of interdependent learning outcomes, example guiding materials and templates for instructors, and preliminary student performance data from the control cohort.