More Like this

1. Automated scoring of science explanations for multiple NGSS dimensions and knowledge integration

   Riordan, B.; Wiley, K.; Chen, J. K.; Bradford, A.; Bichler, S.; Mulholland, M.; Gerard, L. F. (April 2020, Annual Meeting of the American Educational Research Association (AERA))

   The Next Generation Science Standards (NGSS) emphasize integrating three dimensions of science learning: disciplinary core ideas, cross-cutting concepts, and science and engineering practices. In this study, we develop formative assessments that measure student understanding of the integration of these three dimensions along with automated scoring methods that distinguish among them. The formative assessments allow students to express their emerging ideas while also capturing progress in integrating core ideas, cross-cutting concepts, and practices. We describe how item and rubric design can work in concert with an automated scoring system to independently score science explanations from multiple perspectives. We describe item design considerations and provide validity evidence for the automated scores. 

   more » « less
2. Delivering Formative Assessment and Reflecting on Student Learning as Components of Elementary Teacher Professional Learning

   Summers, Ryan; Lee, Min_Jung; Inouye, Martha; Hammack, Rebekah; Macias, Meghan (January 2025, Association for Science Teacher Education (ASTE))

   Teachers in small communities may be geographically isolated and have smaller collegial networks. Consequently, teachers in these settings may have limited exposure to contemporary strategies for engaging learners in science and engineering as suggested in the Next Generation Science Standards (NGSS). Thus, we provided a 5-day online PL experience and a year-long of modest supports (e.g., online professional learning community) to over 150 rural teachers from four states (CA, MT, ND, WY) to bridge the access gap and to enhance their instructional capabilities in teaching NGSS-aligned science and engineering lessons. Considering that the quality of the questions posed in a formative assessment impacts the quality of student thinking and what it reveals, we provided a formative assessment task, “Planning a Park” developed by Stanford NGSS Assessment Project (SNAP) and SCALE Science at WestEd, to participating teachers to implement in their classrooms. Teachers received online professional learning opportunities about the task before and after administering it in their classrooms. To understand their experiences with the task, we collected multiple data sources for triangulation, such as surveys about teachers’ preparedness to implement science lessons, teachers’ self-reported observations while delivering the task, their reflections about students’ performance, examples of student responses to the task, and interview responses from a sub-sample of teachers. As an initial analysis, we employed a descriptive coding process to capture teachers’ diverse experiences with the SCALE task (Saldaña, 2021). In this session, we will report rural teachers’ experiences with the formative assessment task that was provided as part of a year of modest supports. We believe this study will support the science education community, especially individuals preparing teachers to teach science and researchers on assessment, by sharing the benefits of implementing a formative assessment task during inservice teachers’ professional learning. 

   more » « less
3. Implementing NGSS-designed Curriculum Materials: Promising Results from an Efficacy Study

   Feng, Mingyu; Murphy, Robert; Harris, Christopher J.; Rutstein, Daisy; Loper, Suzanna (April 2022, Paper presented at 2022 AERA Annual Meeting)

   The Next Generation Science Standards (NGSS) reflect an ambitious vision for science education where students investigate phenomena or solve problems through using and applying disciplinary core ideas in concert with science and engineering practices and crosscutting concepts. Because the NGSS are so different from prior standards, the need for high-quality curriculum materials is especially great. As new curricula go to scale, it will be important to conduct evidence-based research on their efficacy. We conducted a randomized experiment to examine the efficacy of a widely available NGSS-designed middle school curriculum for improving seventh grade students’ learning in physical science. A hierarchical linear modeling approach was applied to analyze student learning outcomes as measured by an NGSS-aligned assessment. Initial findings demonstrate evidence of promise of the curriculum materials for supporting three-dimensional teaching and learning. The findings provide support for further research on NGSS-designed materials at other grade levels and within other science domains. 

   more » « less
4. An empirical investigation of neural methods for content scoring of science explanations

   https://doi.org/10.18653/v1/2020.bea-1.13  

   Riordan, Brian; Bichler, Sarah; Bradford, Allison; King Chen, Jennifer; Wiley, Korah; Gerard, Libby; C. Linn, Marcia (January 2020, Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications)

   With the widespread adoption of the Next Generation Science Standards (NGSS), science teachers and online learning environments face the challenge of evaluating students' integration of different dimensions of science learning. Recent advances in representation learning in natural language processing have proven effective across many natural language processing tasks, but a rigorous evaluation of the relative merits of these methods for scoring complex constructed response formative assessments has not previously been carried out. We present a detailed empirical investigation of feature-based, recurrent neural network, and pre-trained transformer models on scoring content in real-world formative assessment data. We demonstrate that recent neural methods can rival or exceed the performance of feature-based methods. We also provide evidence that different classes of neural models take advantage of different learning cues, and pre-trained transformer models may be more robust to spurious, dataset-specific learning cues, better reflecting scoring rubrics. 

   more » « less
5. An empirical investigation of neural methods for content scoring of science explanations.

   Riordan, B.; Bichler, S.; Bradford, A.; King Chen, J.; Wiley, K.; Gerard, L.; Linn, M. (July 2020, Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications)

   null (Ed.)

   With the widespread adoption of the Next Generation Science Standards (NGSS), science teachers and online learning environments face the challenge of evaluating students’ integration of different dimensions of science learning. Recent advances in representation learning in natural language processing have proven effective across many natural language processing tasks, but a rigorous evaluation of the relative merits of these methods for scoring complex constructed response formative assessments has not previously been carried out. We present a detailed empirical investigation of feature-based, recurrent neural network, and pre-trained transformer models on scoring content in real-world formative assessment data. We demonstrate that recent neural methods can rival or exceed the performance of feature-based methods. We also provide evidence that different classes of neural models take advantage of different learning cues, and pre-trained transformer models may be more robust to spurious, dataset-specific learning cues, better reflecting scoring rubrics. 

   more » « less