Search for: All records

We use natural language processing (NLP) to train an automated scoring model to assess students’ reasoning on how to slow climate change. We use the insights from scoring over 1000 explanations to design a knowledge integration intervention and test it in three classrooms. The intervention supported students to distinguish relevant evidence, improving connections between ideas in a revised explanation. We discuss next steps for using the NLP model to support teachers and students in classrooms. 

We used Natural Language Processing (NLP) to design an adaptive computer dialogue that engages students in a conversation to reflect on and revise their written explanations of a science dilemma. We study the accuracy of the NLP idea detection. We analyze how 98 12-13 year-olds interacted with the dialogue as a part of a Diagnostic Inventory. We study students’ initial and revised science explanations along with their logged responses to the dialogue. The dialogue led to a high rate of student revision compared to prior studies of adaptive guidance. The adaptive prompt encouraged students to reflect on prior experiences, to consider new variables, and to raise scientific questions. Students incorporated these new ideas when revising their initial explanations. We discuss how these adaptive dialogues can strengthen science instruction. 

With the widespread adoption of the Next Generation Science Standards (NGSS), science teachers and online learning environments face the challenge of evaluating students’ integration of different dimensions of science learning. Recent advances in representation learning in natural language processing have proven effective across many natural language processing tasks, but a rigorous evaluation of the relative merits of these methods for scoring complex constructed response formative assessments has not previously been carried out. We present a detailed empirical investigation of feature-based, recurrent neural network, and pre-trained transformer models on scoring content in real-world formative assessment data. We demonstrate that recent neural methods can rival or exceed the performance of feature-based methods. We also provide evidence that different classes of neural models take advantage of different learning cues, and pre-trained transformer models may be more robust to spurious, dataset-specific learning cues, better reflecting scoring rubrics. 

We investigated a constructive and an example-based scaffold when learning from dynamic visualizations about climate change. Learners collaboratively or individually generated a diagram that represented energy flow (constructive scaffold) or observed a peer learner generating the diagram (example-based scaffold). We hypothesized that collaborative learners would benefit more from the constructive than the example-based scaffold, but that the opposite would be the case for individual learners. Seventy-one university students were randomly allocated to conditions in the 2X2 between-subjects design. Climate change understanding was measured at pre- and posttest. Preliminary results supported our hypothesis. We conclude that the constructive scaffold elicited questions that led to deep engagement in the collaborative condition, resulting in better understanding. Individual learners possibly failed to recognize crucial concepts in the constructive condition because they had questions but nobody to discuss with. They profited more from the example-based scaffold which emphasized central concepts of climate change. 

The Next Generation Science Standards (NGSS) emphasize integrating three dimensions of science learning: disciplinary core ideas, cross-cutting concepts, and science and engineering practices. In this study, we develop formative assessments that measure student understanding of the integration of these three dimensions along with automated scoring methods that distinguish among them. The formative assessments allow students to express their emerging ideas while also capturing progress in integrating core ideas, cross-cutting concepts, and practices. We describe how item and rubric design can work in concert with an automated scoring system to independently score science explanations from multiple perspectives. We describe item design considerations and provide validity evidence for the automated scores.