Recent calls for reform in K‐12 science education and the National Academy of Engineering's Grand Challenges for Engineering in the 21st Century emphasize improving science teaching, students' engagement, and learning. In this study, we designed and implemented a curriculum unit for sixth‐grade students (
Conventional assessment analysis of student results, referred to as rubric‐based assessments (RBA), has emphasized numeric scores as the primary way of communicating information to teachers about their students’ learning. In this light, rethinking and reflecting on not only how scores are generated but also what analyses are done with them to inform classroom practices is of utmost importance. Informed by Systemic Functional Linguistics and Latent Dirichlet Allocation analyses, this study utilizes an innovative bilingual (Spanish–English) constructed response assessment of science and language practices for middle and high school students to perform a multilayered analysis of student responses. We explore multiple ways of looking at students’ performance through their written assessments and discuss features of student responses that are made visible through these analyses. Findings from this study suggest that science educators would benefit from a multidimensional model which deploys complementary ways in which we can interpret student performance. This understanding leads us to think that researchers and developers in the field of assessment need to promote approaches that analyze student science performance as a multilayered phenomenon.
more » « less- PAR ID:
- 10457872
- Publisher / Repository:
- Wiley Blackwell (John Wiley & Sons)
- Date Published:
- Journal Name:
- Journal of Research in Science Teaching
- Volume:
- 57
- Issue:
- 6
- ISSN:
- 0022-4308
- Page Range / eLocation ID:
- p. 856-878
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
Abstract i = 1305). The curriculum unit integrated science and engineering content and practices to teach ecology, water pollution, and engineering design. We investigated the designed integrated STEM unit's effectiveness in students' science learning outcomes on pre‐, post‐, and delayed post‐assessments. We collected pre‐and post‐assessment data of students' science learning outcomes for both the baseline group (taught via existing district‐adopted curriculum) and an intervention group (taught with integrated life science and engineering curriculum). We used a quasi‐experimental research design and examined differences between baseline and intervention groups. We used ANCOVA to explore differences in students' learning in baseline and intervention groups. Furthermore, for students in the intervention group, we conducted repeated‐measures ANOVA to investigate knowledge retention. Our analyses also accounted for students' gender and People of Color (POC) status. We conducted multiple regression analyses to explore the relationship between students' gender, POC status, and their learning outcomes. The results indicated that the intervention group students performed significantly better than the students in the baseline group. The repeated measures ANOVA showed that students in the intervention group retained science knowledge after 8 weeks of instruction. Finally, the regression analysis for the baseline group showed that gender and POC status were not significant predictors of their post‐assessment scores. However, POC status was a significant predictor of post‐assessment scores and knowledge retention for the intervention group. Overall, this study provides valuable findings on how an integrated STEM curriculum designed with engineering design and practices improves students' science learning outcomes. -
Involving students in scientific modeling practice is one of the most effective approaches to achieving the next generation science education learning goals. Given the complexity and multirepresentational features of scientific models, scoring student-developed models is time- and cost-intensive, remaining one of the most challenging assessment practices for science education. More importantly, teachers who rely on timely feedback to plan and adjust instruction are reluctant to use modeling tasks because they could not provide timely feedback to learners. This study utilized machine learn- ing (ML), the most advanced artificial intelligence (AI), to develop an approach to automatically score student- drawn models and their written descriptions of those models. We developed six modeling assessment tasks for middle school students that integrate disciplinary core ideas and crosscutting concepts with the modeling practice. For each task, we asked students to draw a model and write a description of that model, which gave students with diverse backgrounds an opportunity to represent their understanding in multiple ways. We then collected student responses to the six tasks and had human experts score a subset of those responses. We used the human-scored student responses to develop ML algorithmic models (AMs) and to train the computer. Validation using new data suggests that the machine-assigned scores achieved robust agreements with human consent scores. Qualitative analysis of student-drawn models further revealed five characteristics that might impact machine scoring accuracy: Alternative expression, confusing label, inconsistent size, inconsistent position, and redundant information. We argue that these five characteristics should be considered when developing machine-scorable modeling tasks.more » « less
-
The CS community has struggled to assess student learning at the K-8 level, with techniques ranging from one-on-one interviews to written assessments. While scalable, automated techniques exist for analyzing student code, a scalable method for assessing student comprehension of their own code has remained elusive. This study is a first step in bridging the gap between the knowledge gained from interviews and the time efficiency and scalability of written assessments and automated analysis. The goal of this study is to understand how student answers on various types of questions differ depending on whether they are being asked about their own code or generic code. We find that while there were no statistically-significant differences in overall scores, questions about generic and personalized code of comparable complexity are far from equivalent. Our qualitative analyses revealed interesting patterns in student responses, inviting further research into this assessment technique. In particular, students answered differently from students with generic code when presented with individual blocks from their code taken out of context and placed into different code snippets, and students answered in a way that demonstrates a functional, instead of structural, understanding on Explain in Plain English (EiPE) questions.more » « less
-
more » « less
Abstract This study explores the role of unconventional forms of classroom assessments in expanding minoritized students' opportunities to learn (OTL) in high school physics classrooms. In this research + practice partnership project, high school physics teachers and researchers co‐designed a unit about momentum to expand minoritized students' meaningful OTL. Specifically, the unit was designed to (a) expand what it means to learn and be good at science using unconventional forms of assessment, (b) facilitate students to leverage everyday experiences, concerns, and home languages to do science, and (c) support teachers to facilitate meaningful dialogical interactions. The analysis focused on examining minoritized students' OTLs mediated by intentionally designed, curriculum‐embedded, unconventional forms of assessments. The participants were a total of 76 students in 11th or 12th grade. Data were gathered in the form of student assessment tasks, a science identity survey, and interviews. Data analysis entailed: (a) statistical analysis of student performance measured by conventional and unconventional assessments and (b) qualitative analysis of two Latinx students' experiences with the co‐designed curriculum and assessments. The findings suggest that the use of unconventional forms of curriculum‐embedded assessment can increase minoritized students' OTL
if the assessment facilitates minoritized students to personally and deeply relate themselves to academic tasks. -
more » « less
Crafting Engaging Science Environments is a high school chemistry and physics project-based learning intervention that meets Next Generation Science Standards performance expectations. It was administered to a diverse group of over 4,000 students in a randomized control trial in California and Michigan. Results show that treatment students, on average, performed 0.20 standard deviations higher than control students on an independently developed summative science assessment. Mediation analyses show an indirect path between teacher- and student-reported participation in modeling practices and science achievement. Exploratory analyses indicate positive treatment effects for enhancing college ambitions. Overall, results show that improving secondary school science learning is achievable with a coherent system comprising teacher and student learning experiences, professional learning, and formative unit assessments that support students in “doing” science.
