Despite rapid growth of quantum information science (QIS) workforce development initiatives, perceived lack of agreement among faculty on core content has made prior research-based curriculum and assessment development initiatives difficult to scale. To identify areas of consensus on content coverage, we report findings from a survey of N=63 instructors teaching introductory QIS courses at US institutions of higher learning. We identify a subset of content items common across a large fraction (≥ 80%) of introductory QIS courses that are potentially amenable to research-based curriculum development, with an emphasis on foundational skills in mathematics, physics, and engineering. As a further guide for curriculum development, we also examine differences in content coverage by level (undergraduate/graduate) and discipline. Finally, we briefly discuss the implications of our findings for the development of a research-based QIS assessment at the postsecondary level.
Utilizing Evidence-Centered Design to Develop Assessments: A High School Introductory Computer Science Course
Evidence-centered design (ECD) is an assessment framework tailored to provide structure and rigor to the assessment development process, and also to generate evidence of assessment validity by tightly coupling assessment tasks with focal knowledge, skills, and abilities (FKSAs). This framework is particularly well-suited to FKSAs that are complex and multi-part (Mislevy and Haertel, 2006), as is the case with much of the focal content within the computer science (CS) domain. This paper presents an applied case of ECD used to guide assessment development in the context of a redesigned introductory CS curriculum. In order to measure student learning of CS skills and content taught through the curriculum, knowledge assessments were written and piloted. The use of ECD provided an organizational framework for assessment development efforts, offering assessment developers a clear set of steps with accompanying documentation and decision points, as well as providing robust validity evidence for the assessment. The description of an application of ECD for assessment development within the context of an introductory CS course illustrates its utility and effectiveness, and also provides a guide for researchers carrying out related work.
more »
« less
- Award ID(s):
- 1639946
- PAR ID:
- 10290098
- Date Published:
- Journal Name:
- Frontiers in Education
- Volume:
- 6
- ISSN:
- 2504-284X
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
Abstract -
more » « less
Abstract This study explores student agency in the context of a culturally authentic computer science (CS) curriculum implemented in an introductory CS course in two high schools. Drawing on focus group and interview data, the study utilizes qualitative research methods to examine how students exercise critical agency as they engage in the course and how the curriculum supports student agency. Findings suggest three ways in which the curriculum served as a context for student agency: (1) gaining CS knowledge and skills that students then apply to address real-world needs and problems, (2) creating opportunities to “try-on” or improvise new identities and/or envision “future selves” in CS, and (3) engaging in personally relevant project work that leverages assets students brought to their experience with the curriculum. Implications for CS education research and practice are discussed.
-
Nehm, Ross (Ed.)To excel in modern science, technology, engineering, and mathematics careers, biology majors need a range of transferable skills, yet competency development is often a relatively underdeveloped facet of the undergraduate curriculum. We have elaborated the Vision and Change core competency framework into a resource called the BioSkills Guide, a set of measurable learning outcomes that can be more readily implemented by faculty. Following an iterative review process including more than 200 educators, we gathered evidence of the BioSkills Guide’s content validity using a national survey of more than 400 educators. Rates of respondent support were high (74.3–99.6%) across the 77 outcomes in the final draft. Our national sample during the development and validation phases included college biology educators representing more than 250 institutions, including 73 community colleges, and a range of course levels and biology subdisciplines. Comparison of the BioSkills Guide with other science competency frameworks reveals significant overlap but some gaps and ambiguities. These differences may reflect areas where understandings of competencies are still evolving in the undergraduate biology community, warranting future research. We envision the BioSkills Guide supporting a variety of applications in undergraduate biology, including backward design of individual lessons and courses, competency assessment development, and curriculum mapping and planning.more » « less
-
Physics instructors and education researchers use research-based assessments (RBAs) to evaluate students' preparation for physics courses. This preparation can cover a wide range of constructs including mathematics and physics content. Using separate mathematics and physics RBAs consumes course time. We are developing a new RBA for introductory mechanics as an online test using both computerized adaptive testing and cognitive diagnostic models. This design allows the adaptive RBA to assess mathematics and physics content knowledge within a single assessment. In this article, we used an evidence-centered design framework to inform the extent to which our models of skills students develop in physics courses fit the data from three mathematics RBAs. Our dataset came from the LASSO platform and includes 3,491 responses from the Calculus Concept Assessment, Calculus Concept Inventory, and Pre-calculus Concept Assessment. Our model included five skills: apply vectors, conceptual relationships, algebra, visualizations, and calculus. The "deterministic inputs, noisy 'and' gate'' (DINA) analyses demonstrated a good fit for the five skills. The classification accuracies for the skills were satisfactory. Including items from the three mathematics RBAs in the item bank for the adaptive RBA will provide a flexible assessment of these skills across mathematics and physics content areas that can adapt to instructors' needs.more » « less
-
null (Ed.)This paper discusses the results of replicating and extending a study performed by Cooper et al. examining the relationship between students’ spatial skills and their success in learning to program. Whereas Cooper et al. worked with high school students participating in a summer program, we worked with college students taking an introductory computing course. Like Cooper et al.’s study, we saw a correlation between a student’s spatial skills and their success in learning computing. More significantly, we saw that after applying an intervention to teach spatial skills, students demonstrated improved performance both on a standard spatial skills assessment as well as on a CS content instrument. We also saw a correlation between students’ enjoyment in computing and improved performance both on a standard spatial skills assessment and on a CS content instrument, a result not observed by Cooper et al.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.3389/feduc.2021.695376
