Search for: All records

We use natural language processing (NLP) to train an automated scoring model to assess students’ reasoning on how to slow climate change. We use the insights from scoring over 1000 explanations to design a knowledge integration intervention and test it in three classrooms. The intervention supported students to distinguish relevant evidence, improving connections between ideas in a revised explanation. We discuss next steps for using the NLP model to support teachers and students in classrooms. 

We used Natural Language Processing (NLP) to design an adaptive computer dialogue that engages students in a conversation to reflect on and revise their written explanations of a science dilemma. We study the accuracy of the NLP idea detection. We analyze how 98 12-13 year-olds interacted with the dialogue as a part of a Diagnostic Inventory. We study students’ initial and revised science explanations along with their logged responses to the dialogue. The dialogue led to a high rate of student revision compared to prior studies of adaptive guidance. The adaptive prompt encouraged students to reflect on prior experiences, to consider new variables, and to raise scientific questions. Students incorporated these new ideas when revising their initial explanations. We discuss how these adaptive dialogues can strengthen science instruction. 

With the widespread adoption of the Next Generation Science Standards (NGSS), science teachers and online learning environments face the challenge of evaluating students’ integration of different dimensions of science learning. Recent advances in representation learning in natural language processing have proven effective across many natural language processing tasks, but a rigorous evaluation of the relative merits of these methods for scoring complex constructed response formative assessments has not previously been carried out. We present a detailed empirical investigation of feature-based, recurrent neural network, and pre-trained transformer models on scoring content in real-world formative assessment data. We demonstrate that recent neural methods can rival or exceed the performance of feature-based methods. We also provide evidence that different classes of neural models take advantage of different learning cues, and pre-trained transformer models may be more robust to spurious, dataset-specific learning cues, better reflecting scoring rubrics. 

Teachers are currently facing a major instructional challenge, namely, supporting students to meet the three dimensional learning goals of the Next Generation Science Standards without adequate curriculum materials to do so. In this paper, we report the design and outcomes of professional development activities that support teachers and researchers to collaboratively customize Web-based Inquiry Science Environment (WISE) units to help students develop coherent science knowledge. The WISE units and professional development activities were developed using the Knowledge Integration (KI) Framework. We show that the KI framework functioned as an effective scaffold to support teachers in modifying their teaching practice to make curriculum customizations that are evidence-based and aligned with a theory of learning. We discuss how our study results informed the design of an online curriculum customization and implementation interface and offer the principles of the KI framework as design principles for the development of other collaborative professional development endeavors. 

With the increasing use of online interactive environments for science and engineering education in grades K-12, there is a growing need for detailed automatic analysis of student explanations of ideas and reasoning. With the widespread adoption of the Next Generation Science Standards (NGSS), an important goal is identifying the alignment of student ideas with NGSS-defined dimensions of proficiency. We develop a set of constructed response formative assessment items that call for students to express and integrate ideas across multiple dimensions of the NGSS and explore the effectiveness of state-of-the-art neural sequence-labeling methods for identifying discourse-level expressions of ideas that align with the NGSS. We discuss challenges for idea detection task in the formative science assessment context.