skip to main content


Title: Developing and validating Next Generation Science Standards ‐aligned learning progression to track three‐dimensional learning of electrical interactions in high school physical science
Abstract

The Framework for K‐12 science education (TheFramework) and Next Generation Science Standards (NGSS) emphasize the usefulness of learning progressions in helping align curriculum, instruction, and assessment to organize the learning process. TheFrameworkdefines three dimensions of science as the basis of theoretical learning progressions described in the document and used to develop NGSS. The three dimensions include disciplinary core ideas, scientific and engineering practices, and crosscutting concepts. TheFrameworkdefines three‐dimensional learning (3D learning) as integrating scientific and engineering practices, crosscutting concepts, and disciplinary core ideas to make sense of phenomena. Three‐dimensional learning leads to the development of a deep, useable understanding of big ideas that students can apply to explain phenomena and solve real‐life problems. While theFrameworkdescribes the theoretical basis of 3D learning, and NGSS outlines possible theoretical learning progressions for the three dimensions across grades, we currently have very limited empirical evidence to show that a learning progression for 3D learning can be developed and validated in practice. In this paper, we demonstrate the feasibility of developing a 3D learning progression (3D LP) supported by qualitative and quantitative validity evidence. We first present a hypothetical 3D LP aligned to a previously designed NGSS‐based curriculum. We further present multiple sources of validity evidence for the hypothetical 3D LP, including interview analysis and item response theory (IRT) analysis to show validity evidence for the 3D LP. Finally, we demonstrate the feasibility of using the assessment tool designed to probe levels of the 3D LP for assigning 3D LP levels to individual student answers, which is essential for the practical applicability of any LP. This work demonstrates the usefulness of validated 3D LP for organizing the learning process in the NGSS classroom, which is essential for the successful implementation of NGSS.

 
more » « less
NSF-PAR ID:
10450017
Author(s) / Creator(s):
 ;  ;  
Publisher / Repository:
Wiley Blackwell (John Wiley & Sons)
Date Published:
Journal Name:
Journal of Research in Science Teaching
Volume:
58
Issue:
4
ISSN:
0022-4308
Page Range / eLocation ID:
p. 589-618
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. The Next Generation Science Standards (NGSS) reflect an ambitious vision for science education where students investigate phenomena or solve problems through using and applying disciplinary core ideas in concert with science and engineering practices and crosscutting concepts. Because the NGSS are so different from prior standards, the need for high-quality curriculum materials is especially great. As new curricula go to scale, it will be important to conduct evidence-based research on their efficacy. We conducted a randomized experiment to examine the efficacy of a widely available NGSS-designed middle school curriculum for improving seventh grade students’ learning in physical science. A hierarchical linear modeling approach was applied to analyze student learning outcomes as measured by an NGSS-aligned assessment. Initial findings demonstrate evidence of promise of the curriculum materials for supporting three-dimensional teaching and learning. The findings provide support for further research on NGSS-designed materials at other grade levels and within other science domains. 
    more » « less
  2. The Next Generation Science Standards (NGSS) emphasize integrating three dimensions of science learning: disciplinary core ideas, cross-cutting concepts, and science and engineering practices. In this study, we develop formative assessments that measure student understanding of the integration of these three dimensions along with automated scoring methods that distinguish among them. The formative assessments allow students to express their emerging ideas while also capturing progress in integrating core ideas, cross-cutting concepts, and practices. We describe how item and rubric design can work in concert with an automated scoring system to independently score science explanations from multiple perspectives. We describe item design considerations and provide validity evidence for the automated scores. 
    more » « less
  3. null (Ed.)
    This research paper describes the development of an assessment instrument for use with middle school students that provides insight into students’ interpretive understanding by looking at early indicators of developing expertise in students’ responses to solution generation, reflection, and concept demonstration tasks. We begin by detailing a synthetic assessment model that served as the theoretical basis for assessing specific thinking skills. We then describe our process of developing test items by working with a Teacher Design Team (TDT) of instructors in our partner school system to set guidelines that would better orient the assessment in that context and working within the framework of standards and disciplinary core ideas enumerated in the Next Generation Science Standards (NGSS). We next specify our process of refining the assessment from 17 items across three separate item pools to a final total of three open-response items. We then provide evidence for the validity and reliability of the assessment instrument from the standards of (1) content, (2) meaningfulness, (3) generalizability, and (4) instructional sensitivity. As part of the discussion from the standards of generalizability and instructional sensitivity, we detail a study carried out in our partner school system in the fall of 2019. The instrument was administered to students in treatment (n= 201) and non- treatment (n = 246) groups, wherein the former participated in a two-to-three- week, NGSS-aligned experimental instructional unit introducing the principles of engineering design that focused on engaging students using the Imaginative Education teaching approach. The latter group were taught using the district’s existing engineering design curriculum. Results from statistical analysis of student responses showed that the interrater reliability of the scoring procedures were good-to-excellent, with intra-class correlation coefficients ranging between .72 and .95. To gauge the instructional sensitivity of the assessment instrument, a series of non-parametric comparative analyses (independent two-group Mann- Whitney tests) were carried out. These found statistically significant differences between treatment and non-treatment student responses related to the outcomes of fluency and elaboration, but not reflection. 
    more » « less
  4. null (Ed.)
    This research paper describes the development of an assessment instrument for use with middle school students that provides insight into students’ interpretive understanding by looking at early indicators of developing expertise in students’ responses to solution generation, reflection, and concept demonstration tasks. We begin by detailing a synthetic assessment model that served as the theoretical basis for assessing specific thinking skills. We then describe our process of developing test items by working with a Teacher Design Team (TDT) of instructors in our partner school system to set guidelines that would better orient the assessment in that context and working within the framework of standards and disciplinary core ideas enumerated in the Next Generation Science Standards (NGSS). We next specify our process of refining the assessment from 17 items across three separate item pools to a final total of three open-response items. We then provide evidence for the validity and reliability of the assessment instrument from the standards of (1) content, (2) meaningfulness, (3) generalizability, and (4) instructional sensitivity. As part of the discussion from the standards of generalizability and instructional sensitivity, we detail a study carried out in our partner school system in the fall of 2019. The instrument was administered to students in treatment (n= 201) and non-treatment (n = 246) groups, wherein the former participated in a two-to-three-week, NGSS-aligned experimental instructional unit introducing the principles of engineering design that focused on engaging students using the Imaginative Education teaching approach. The latter group were taught using the district’s existing engineering design curriculum. Results from statistical analysis of student responses showed that the interrater reliability of the scoring procedures were good-to-excellent, with intra-class correlation coefficients ranging between .72 and .95. To gauge the instructional sensitivity of the assessment instrument, a series of non-parametric comparative analyses (independent two-group Mann-Whitney tests) were carried out. These found statistically significant differences between treatment and non-treatment student responses related to the outcomes of fluency and elaboration, but not reflection. 
    more » « less
  5. Abstract

    Recent instructional reforms in science education aim to change the way students engage in learning in the discipline, as they describe that students are to engage with disciplinary core ideas, crosscutting concepts, and the practices of science to make sense of phenomena (NRC, 2012). For such sensemaking to become a reality, there is a need to understand the ways in which students' thinking can be maintained throughout the trajectory of science lessons. Past research in this area tends to foreground either the curriculum or teachers' practices. We propose a more comprehensive view of science instruction, one that requires attention to teachers' practice, the instructional task, and students' engagement. In this study, by examining the implementation of the same lesson across three different classrooms, our analysis of classroom videos and artifacts of students' work revealed how the interaction of teachers' practices, students' intellectual engagement, and a cognitively demanding task together support rigorous instruction. Our analyses shed light on their interaction that shapes opportunities for students' thinking and sensemaking throughout the trajectory of a science lesson. The findings provide implications for ways to promote rigorous opportunities for students' learning in science classrooms.

     
    more » « less