An official website of the United States government
Research based assessments have a productive and storied history in PER. While useful for conducting research on student learning, their utility is limited for instructors interested in improving their own courses. We have developed a new assessment design process that leverages three-dimensional learning, evidence-centered design, and self-regulated learning to deliver actionable feedback to instructors about supporting their students' learning. We are using this approach to design the Thermal and Statistical Physics Assessment (TaSPA), which also allows instructors to choose learning goals that align with their teaching. Perhaps more importantly, this system will be completely automated when it is completed, making the assessment scalable with minimal burden on instructors and researchers. This work represents an advancement in how we assess physics learning at a large scale and how the PER community can better support physics instructors and students.
more »
« less
Patterns in Elementary-Age Student Responses to Personalized & Generic Code Comprehension Questions
The CS community has struggled to assess student learning at the K-8 level, with techniques ranging from one-on-one interviews to written assessments. While scalable, automated techniques exist for analyzing student code, a scalable method for assessing student comprehension of their own code has remained elusive. This study is a first step in bridging the gap between the knowledge gained from interviews and the time efficiency and scalability of written assessments and automated analysis. The goal of this study is to understand how student answers on various types of questions differ depending on whether they are being asked about their own code or generic code. We find that while there were no statistically-significant differences in overall scores, questions about generic and personalized code of comparable complexity are far from equivalent. Our qualitative analyses revealed interesting patterns in student responses, inviting further research into this assessment technique. In particular, students answered differently from students with generic code when presented with individual blocks from their code taken out of context and placed into different code snippets, and students answered in a way that demonstrates a functional, instead of structural, understanding on Explain in Plain English (EiPE) questions.
more »
« less
- Award ID(s):
- 1660871
- PAR ID:
- 10158898
- Date Published:
- Journal Name:
- SIGCSE '20: Proceedings of the 51st ACM Technical Symposium on Computer Science Education
- Page Range / eLocation ID:
- 514 to 520
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The career paths of PhD scientists often deviate from their doctoral theses. As a result, the need to integrate student-centered career and professional development training is important to meet the needs of doctoral students. Qualifying exams (QEs) represent a significant milestone in progression toward graduation within most PhD Programs in the United States. These exams are commonly administered 2–3 years into a PhD program following the completion of coursework, with the primary objective of evaluating whether the candidate possesses the necessary knowledge and skills to progress with their dissertation research. To enhance the value of QEs and intentionally align them with the diverse career trajectories of our students, we explored the inclusion of student-centered assessments in a track with a Pharmaceutical Sciences PhD program. In this PhD program, one component of QEs is a series of monthly, written cumulative exams focused on recent scientific literature in the faculty and students’ discipline. To create a student-centered QE, the student and a faculty member collaborated to develop personalized assessments focused on career exploration and in alignment with individual student’s career goals. All students enrolled in the PhD track (n = 8) were invited to participate in a survey about their experience with the redesigned QE. A combination of Likert scale and short answer questions were collected; quantitative items were analyzed with descriptive statistics and qualitative items with thematic coding. A subset of survey participants (n = 5) participated in a focus group regarding their experience with both the Traditional Model QE and the redesigned Pilot Model QE. Two faculty interviews were conducted regarding the design, content, procedures, and evaluation of student QEs. The study design and analysis were grounded in the cognitive apprenticeship framework, with a focus on how the QEs were situated within the four domains of this framework: content, methods, sequencing, and sociology. Results revealed that this student-centered QE approach was perceived to be more aligned with student career aspirations and to have a high interest level and value for students without placing a substantial additional burden on participants. This suggests that it is a feasible mechanism for integrating student-centered assessment into QEs.more » « less
-
We evaluated the impact of the current COVID-19 pandemic and systemic racism on Underrepresented Minority (URM) students pursuing higher education in the STEM fields. Given the ongoing pandemic and the wave of protests in response to a series of police brutalities and systemic racism, URM students were thrown into uncharted territory. We reached out to a group of Black and Latino students who were already engaged in STEM. We began surveys and interviews by asking participants how they were and how their family and communities were doing. Next, participants answered questions about academic progress, challenges, and what support would be helpful. Our framework was based on a mixed-methods approach that draws on the work of Michael Patton (Qualitative Research & Evaluation Methods: Integrating Theory and Practice, 2014) and Veronica Thomas (American Journal of Evaluation, 2016, 38 (1), 7–28). Qualitative data from interviews were collected to capture perceptions, experiences, and recommendations of the study participants. Survey data were collected to reach as many students as possible and to provide numerical self-assessments of student experience, progression, and obstacles. All qualitative data were coded thematically using Atlas. ti, with the goal of illuminating emerging themes, and quantitative data were reviewed using descriptive statistics. Themes emerging from both data sets were compared, contrasted, and integrated in order to develop consistent findings that would enhance URM student perseverance and persistence in the face of confounding adversities. This study shows that ILSAMP COVID-19 Study participants maintained a commitment to pursuing a career in STEM. The findings of this study also indicate that the participants are stressed by their immediate circumstances and by the ongoing racism of U.S. society. These students ask for additional financial, academic, and networking support during the disruptions caused by the pandemic. More specifically, students request continued advising and connection with STEM professionals who can help them envision and enact a pathway to their own careers in STEM during this tumultuous period. The study validates the importance of key elements of the national LSAMP model as reported by Clewell et al. (Revitalizing the Nation’s Talent Pool in STEM, 2006). These are: academic integration, social integration, and professional integration. In addition, it identifies several other factors that are key to student success, including interventions that directly address racial trauma and economic hardship.more » « less
-
Abstract We investigated the difficulties that physics students in upper-level undergraduate quantum mechanics and graduate students after quantum and statistical mechanics core courses have with the Fermi energy, the Fermi–Dirac distribution and total electronic energy of a free electron gas after they had learned relevant concepts in their respective courses. These difficulties were probed by administering written conceptual and quantitative questions to undergraduate students and asking some undergraduate and graduate students to answer those questions while thinking aloud in one-on-one individual interviews. We find that advanced students had many common difficulties with these concepts after traditional lecture-based instruction. Engaging with a sequence of clicker questions improved student performance, but there remains room for improvement in their understanding of these challenging concepts.more » « less
-
Abstract Understanding and communicating uncertainty is a key skill needed in the practice of science. However, there has been little research on the instruction of uncertainty in undergraduate science education. Our team designed a module within an online geoscience field course which focused on explicit instruction around uncertainty and provided students with an uncertainty rating scale to record and communicate their uncertainty with a common language. Students then explored a complex, real-world geological problem about which expert scientists had previously made competing claims through geologic maps. Provided with data, expert uncertainty ratings, and the previous claims, students made new geologic maps of their own and presented arguments about their claims in written form. We analyzed these reports along with assessments of uncertainty. Most students explicitly requested geologists’ uncertainty judgments in a post-course assessment when asked why scientists might differ in their conclusions and/or utilized the rating scale unprompted in their written arguments. Through the examination of both pre- and post-course assessments of uncertainty and students’ course-based assessments, we argue that explicit instruction around uncertainty can be introduced during undergraduate coursework and could facilitate geoscience novices developing into practicing geoscientists.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3328778.3366833
