An official website of the United States government
This paper presents the design and analysis of a pilot problem set deployed to engineering students to assess their retention of physics knowledge at the start of a statics course. The problem set was developed using the NSF-IUSE (grant #2315492) Learning Map project (LMap) and piloted in the spring and fall of 2024. The LMap process is rooted in the Analysis, Design, Development, Implementation, and Evaluation (ADDIE) model [1] and Backward Design [2,3], extending these principles to course sequences to align learning outcomes, assessments, and instructional practices. The primary motivation for this problem set (Statics Knowledge Inventory, SKI) was to evaluate students' understanding and retention of physics concepts at the beginning of a statics course. The SKI includes a combination of multiple-choice questions (MCQ) and procedural problems, filling a gap in widely-used concept inventories for physics and statics, such as the Force Concept Inventory (FCI) and Statics Concept Inventory (SCI), which evaluate learning gains within a course, rather than knowledge retention across courses. Using the LMap analysis and instructor consultations, we identified overlapping concepts and topics between Physics and Statics courses, referred to here as “interdependent learning outcomes” (ILOs). The problem set includes 15 questions—eight MCQs and seven procedural problems. Unlike most concept inventories, procedural problems were added to provide insight into students’ problem-solving approach and conceptual understanding. These problems require students to perform calculations, demonstrate their work, and assess their conceptual understanding of key topics, and allow the instructors to assess essential prerequisite skills like drawing free-body diagrams (FBDs), computing forces and moments, and performing basic vector calculation and unit conversions. Problems were selected and adapted from physics and statics textbooks, supplemented by instructor-designed questions to ensure full coverage of the ILOs. We used the revised 2D Bloom’s Taxonomy [4] and a 3D representation of it [5] to classify each problem within a 6x4 matrix (six cognitive processes x four knowledge dimensions). This classification provided instructors and students with a clear understanding of the cognitive level required for each problem. Additionally, we measured students’ perceived confidence and difficulty in each problem using two questions on a 3-point Likert scale. The first iteration of the problem set was administered to 19 students in the spring 2024 statics course. After analyzing their performance, we identified areas for improvement and revised the problem set, removing repetitive MCQs and restructuring the procedural problems into scaffolded, multi-part questions with associated rubrics for evaluation. The revised version, consisting of five MCQs and six procedural problems, was deployed to 136 students in the fall 2024 statics course. A randomly selected subset of student answers from the second iteration was graded and analyzed to compare with the first. This analysis will inform future efforts to evaluate knowledge retention and transfer in key skills across sequential courses. In collaboration with research teams developing concept inventories for mechanics courses, we aim to integrate these procedural problems into future inventories.
more »
« less
When the light bulb turns on: motivation and collaboration spark the creation of ideas for theoretical physicists
In this project, we sought to uncover the cognitive processes and skills that are involved in completing a theoretical physics project. Theoretical physics is often portrayed as a field requiring individual genius and can seem inaccessible to undergraduate students, as well as the public. We drew upon the foundations of Cognitive Task Analysis and completed semi-structured interviews with eleven theoretical physics faculty members from several different research institutions who specialized in subfields including quantum optics, biophysics, computational astrophysics, and string theory. We analyzed the processes and skills of these physicists, focusing on an analysis of idea origin, which is typically the first cognitive process within a project, and how it was connected to collaboration and motivation. We used concept maps to organize these codes and portray the factors that influence the creation of project ideas. We found that motivation and collaboration are fundamental determinants of project ideas and their origins, which contradicts the "lone genius" stereotype. These findings on cognitive processes and skills can help us understand how to better prepare students to do theoretical physics research. Finally, the information gathered during this project may be useful for improving the public understanding of theoretical physics, dispelling the belief that the field requires "genius," and making it accessible to more students.
more »
« less
- Award ID(s):
- 1846321
- PAR ID:
- 10323332
- Date Published:
- Journal Name:
- Physics Education Research Conference 2021
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Frank, Brian W.; Jones, Dyan L.; Ryan, Qing X. (Ed.)Many of the activities and cognitive processes that physicists use while solving problems are "invisible" to students, which can hinder their acquisition of important expert-like skills. Whereas the detailed calculations performed by researchers are often published in journals and textbooks, other activities such as those undertaken while planning how to approach a problem are rarely discussed in published research. Hence, these activities are especially hidden from students. To better understand how physicists solve problems in their professional research, we leveraged the framework of cognitive task analysis to conduct semi-structured interviews with theoretical physicists (N=11). Here we elucidate the role of planning and preliminary analysis in theorists' work. Theorists described using a variety of activities in order to decide if their project was doable while also generating possible solution paths. These actions included doing cursory calculations, reflecting on previous knowledge, gaining intuition and understanding by studying prior work, and reproducing previous results. We found that theorists typically did not pursue projects unless they had a clear idea of what the outcome of their project would be, or at least knew that they would be able to make progress on the problem. Thus, this preliminary design and analysis phase was highly important for theorists despite being largely hidden from students. We conclude by suggesting potential ways to incorporate our findings into the classroom to give students more numerous opportunities to engage in these expert-like practices.more » « less
-
How to Support Career Technical Instructors to Develop Students' Professional Skills: Research BriefNA (Ed.)Project GOALS (Greater Opportunities to Advance Lifelong Success), an NSF Advanced Technological Education targeted research project, brought together researchers and community college educators from 2020 through 2024 to co-develop, test, package, and distribute resources for developing technicians' professional skills. Through this work, the team discovered the barriers that hinder both instructors and students from connecting around professional skills development. To address these barriers, the Project GOALS team developed an instructional framework grounded in research that integrates focused low-stakes activities into classes as students work toward their technical certificates. Based on our research, we believe Project GOALS provides ways for students to students' career readiness. In this research brief, the team describes how the Project GOALS co-development collaboration revealed the supports that technical instructors need to build students' professional awareness and reflection habits. Through qualitative analysis, Project GOALS researchers discovered the challenges that prevent many instructors from sharing their honest assessments of students' professional skills. The brief describes findings and recommends ways that community colleges can provide the assistance and resources that instructors need to develop students' professional skills.more » « less
-
Recent trends in the cybersecurity workforce have recognized that effective solutions for complex problems require collective efforts from individuals with diverse sets of knowledge, skills, and abilities. Therefore, the growing need to train students in team collaboration skills propelled educators in computer science and engineering to adopt team-based pedagogical strategies. Team-based pedagogy has shown success in enhancing students' knowledge in course subjects and their motivation in learning. However, it is limited in offering concrete frameworks specifically focusing on how to teach team collaboration skills. As part of an interdisciplinary effort, we draw on Transactive Memory Systems Theory-a communication theory that explains how individuals in groups learn who knows what and organize who does what-in developing a Team Knowledge Sharing Assignment as a tool for student teams to structure their team collaboration processes. This paper reports a result of a case study in designing and facilitating the assignment for cybersecurity students enrolled in a scholarship program. Students' evaluations and the instructor's assessment reveal that the assignment made a positive impact on students' team collaboration skills by helping them successfully identify their team members' expertise and capitalize on their team's knowledge resources when delegating functional roles. Based on this case study, we offer practical suggestions on how the assignment could be used for various classes or cybersecurity projects and how instructors could maximize its benefits.more » « less
-
Exploring social and cognitive engagement in small groups through a community of learners (CoL) lensA variety of research studies reveal the advantages of actively engaging students in the learning process through collaborative work in the classroom. However, the complex nature of the learning environment in large college general chemistry courses makes it challenging to identify the different factors that affect students’ cognitive and social engagement while working on in-class tasks. To provide insights into this area, we took a closer look at students’ conversations during in-class activities to characterize typical discourse patterns and expressed chemical thinking in representative student groups in samples collected in five different learning environments across four universities. For this purpose, we adapted and applied a ‘Community of Learners’ (CoL) theoretical perspective to characterize group activity through the analysis of student discourse. Within a CoL perspective, the extent to which a group functions as a community of learners is analyzed along five dimensions including Community of Discourse (CoD), Legitimization of Differences (LoD), Building on Ideas (BoI), Reflective Learning (RL), and Community of Practice (CoP). Our findings make explicit the complexity of analyzing student engagement in large active learning environments where a multitude of variables can affect group work. These include, among others, group size and composition, the cognitive level of the tasks, the types of cognitive processes used to complete tasks, and the motivation and willingness of students to substantively engage in disciplinary reasoning. Our results point to important considerations in the design and implementation of active learning environments that engage more students with chemical ideas at higher levels of reasoning.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1119/perc.2021.pr.Griston
