An official website of the United States government
This paper presents the design and analysis of a pilot problem set deployed to engineering students to assess their retention of physics knowledge at the start of a statics course. The problem set was developed using the NSF-IUSE (grant #2315492) Learning Map project (LMap) and piloted in the spring and fall of 2024. The LMap process is rooted in the Analysis, Design, Development, Implementation, and Evaluation (ADDIE) model [1] and Backward Design [2,3], extending these principles to course sequences to align learning outcomes, assessments, and instructional practices. The primary motivation for this problem set (Statics Knowledge Inventory, SKI) was to evaluate students' understanding and retention of physics concepts at the beginning of a statics course. The SKI includes a combination of multiple-choice questions (MCQ) and procedural problems, filling a gap in widely-used concept inventories for physics and statics, such as the Force Concept Inventory (FCI) and Statics Concept Inventory (SCI), which evaluate learning gains within a course, rather than knowledge retention across courses. Using the LMap analysis and instructor consultations, we identified overlapping concepts and topics between Physics and Statics courses, referred to here as “interdependent learning outcomes” (ILOs). The problem set includes 15 questions—eight MCQs and seven procedural problems. Unlike most concept inventories, procedural problems were added to provide insight into students’ problem-solving approach and conceptual understanding. These problems require students to perform calculations, demonstrate their work, and assess their conceptual understanding of key topics, and allow the instructors to assess essential prerequisite skills like drawing free-body diagrams (FBDs), computing forces and moments, and performing basic vector calculation and unit conversions. Problems were selected and adapted from physics and statics textbooks, supplemented by instructor-designed questions to ensure full coverage of the ILOs. We used the revised 2D Bloom’s Taxonomy [4] and a 3D representation of it [5] to classify each problem within a 6x4 matrix (six cognitive processes x four knowledge dimensions). This classification provided instructors and students with a clear understanding of the cognitive level required for each problem. Additionally, we measured students’ perceived confidence and difficulty in each problem using two questions on a 3-point Likert scale. The first iteration of the problem set was administered to 19 students in the spring 2024 statics course. After analyzing their performance, we identified areas for improvement and revised the problem set, removing repetitive MCQs and restructuring the procedural problems into scaffolded, multi-part questions with associated rubrics for evaluation. The revised version, consisting of five MCQs and six procedural problems, was deployed to 136 students in the fall 2024 statics course. A randomly selected subset of student answers from the second iteration was graded and analyzed to compare with the first. This analysis will inform future efforts to evaluate knowledge retention and transfer in key skills across sequential courses. In collaboration with research teams developing concept inventories for mechanics courses, we aim to integrate these procedural problems into future inventories.
more »
« less
Learning Map Framework to Align Instruction and Improve Student Learning in a Physics-Engineering Mechanics Course Sequence
This project supports the success of undergraduate engineering students through coordinated design of curricula across STEM course sequences. The Analysis, Design, Development, Implementation, Evaluation (ADDIE) framework and backward design are being used to develop guides for instructors to align learning outcomes, assessments, and instructional materials in a physics – engineering mechanics course sequence. The approach relies on the analysis of student learning outcomes in each course, identification of interdependent learning outcomes, and development of skills hierarchies in the form of visual learning maps. The learning maps are used to illustrate the knowledge required and built upon throughout the course sequence. This study will assess the effectiveness of a course redesign intervention, which uses visual learning maps and backward design concepts, to guide instructors within a common course sequence to align learning outcomes and assessments. If successful, the intervention is expected to improve students’ primary learning and knowledge retention, as well as persistence and success in the degree. The study will compare academic performance among Mechanical Engineering B.S., Environmental Engineering B.S., and Civil Engineering B.S. students who begin a Physics for Engineers – Statics – Dynamics course prior to the intervention (control) and after the intervention (treatment). During control and treatment terms, students’ primary learning in individual courses will be assessed using established concept inventories. Retention of knowledge from pre-requisite courses will be tracked using pre-identified problem sets (quizzes, exams) specifically associated with interdependent learning outcomes in the Statics and Dynamics courses. Students’ primary learning and knowledge retention in the sequence will be related to longer term student success outcomes, including retention and graduation. The poster will show the results of the research team’s first year of work, including an analysis of current course materials, learning maps for each course, identification of interdependent learning outcomes, example guiding materials and templates for instructors, and preliminary student performance data from the control cohort.
more »
« less
- Award ID(s):
- 2315492
- PAR ID:
- 10537181
- Publisher / Repository:
- ASEE Peer
- Date Published:
- Format(s):
- Medium: X
- Location:
- Portland OR
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Academic bridge courses are implemented to impact students’ academic success by revising fundamental concepts and skills necessary to successfully complete discipline-specific courses. The bridge courses are often short (one to three weeks) and highly dense in content (commonly mathematics or math-related applications). With the support of the NSF-funded (DUE - Division of Undergraduate Education) STEM Center at Sam Houston State University (SHSU), we designed a course for upcoming engineering majors (i.e., first-year students and transfer students) that consists of a two-week-long pre-semester course organized into two main sessions. The first sessions (delivered in the mornings) were synchronous activities focused on strengthening student academic preparedness and socio-academic integration and fostering networking leading to a strong STEM learning community. The second sessions (delivered in the afternoons) were asynchronous activities focused on discipline-specific content knowledge in engineering. The engineering concepts were organized via eight learning modules covering basic math operations, applied trigonometry, functions in engineering, applied physics, introduction to statics and Microsoft Excel, and engineering economics and its applied decision. All materials in the course were designed by engineering faculty (from the chair of the department to assistant professors and lecturers in engineering) and one educational research faculty (from the department of chemistry). The course design process started with a literature review on engineering bridge courses to understand prior work, followed by surveying current engineering faculty to propose goals for the course. The designed team met weekly after setting the course goals over two semesters. The design process was initiated with backward design principles (i.e., start with the course goals, then the assessments, end with the learning activities) and continued with ongoing revision. The work herein presents this new engineering bridge course’s goals, strategy, and design process. Preliminary student outcomes will be discussed based on the course’s first implementation during summer 2021.more » « less
-
null (Ed.)The University of Alabama is exploring Learning in Advance (LIA) courses to introduce engineering students to concepts and correct common misconceptions prior to encountering the complex theories and concepts in three different gateway courses. These gateway courses are circuit analysis, statics, and data-structures/algorithms. The courses were identified based on analysis of institutional data. Data indicated that greater than 90% of UA students who succeed in the three courses went on to complete their undergraduate degree. Yet, each course has individually high rates of failure and/or withdrawals. The objective and intended learning outcomes of each of the three courses is to provide students with knowledge of key concepts that will strengthen the student’s critical thinking skills and establish a strong technical foundation. In this work an overview of the LIA courses is provided along with summaries of collected student feedback and the plans for future assessment to track the effectiveness of this intervention to improve student outcomes in the gateway courses.more » « less
-
Previous research has demonstrated a link between prior knowledge and student success in engineering courses. However, while course-to-course relations exist, researchers have paid insufficient attention to internal course performance development. This study aims to address this gap—designed to quantify and thus extract meaningful insights—by examining a fundamental engineering course, Statics, from three perspectives: (1) progressive learning reflected in performance retention throughout the course; (2) critical topics and their influence on students’ performance progression; and (3) student active participation as a surrogate measure of progressive learning. By analyzing data collected from 222 students over five semesters, this study draws insights on student in-course progressive learning. The results show that early learning had significant implications in building a foundation in progressive learning throughout the semester. Additionally, insufficient knowledge on certain topics can hinder student learning progression more than others, which eventually leads to course failure. Finally, student participation is a pathway to enhance learning and achieve excellent course performance. The presented analysis approach provides educators with a mechanism for diagnosing and devising strategies to address conceptual lapses for STEM (science, technology, engineering, and mathematics) courses, especially where progressive learning is essential.more » « less
-
null (Ed.)Bridge courses are often created to provide participants with remediation instruction on discipline-specific content knowledge, like chemistry and mathematics, before enrollment in regular (semester-long) courses. The bridge courses are then designed to impact student’s academic success in the short-term. Also, as a consequence of the bridge course experience, it is often expected that students’ dropout rates on those regular courses will decrease. However, the bridge courses are often short (ten or fewer days) and packed with content, thus creating challenges for helping students sustain their learning gains over time. With the support of the NSF funded (DUE - Division Of Undergraduate Education) STEM Center at Sam Houston State University, we are designing a course for entering chemistry students that consists of a one-week pre-semester intensive bridge component, which then flows into a one-month co-curricular support component at the beginning of the semester. The primary goals of the bridge component of the course are to strengthen student academic preparedness, calibrated-self-efficacy, and to foster networking leading to a strong learning community. The goal of the co-curricular extension is to help students sustain and build upon the learning gains of the initial bridge component. We plan to extend the co-curricular portion of the course in future years. A key measure of success will be improved participant course grades in the introductory chemistry courses for majors. Our design process has been centered on weekly meetings that alternate between literature review and course design. The design process was initiated with backward design principles and continues with ongoing revision. The goals, design strategy, and design process of this new course will be presented along with the achieved student outcomes during the implementation of the past summer 2020.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
