skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: How do middle school students think about climate change?
We use natural language processing (NLP) to train an automated scoring model to assess students’ reasoning on how to slow climate change. We use the insights from scoring over 1000 explanations to design a knowledge integration intervention and test it in three classrooms. The intervention supported students to distinguish relevant evidence, improving connections between ideas in a revised explanation. We discuss next steps for using the NLP model to support teachers and students in classrooms.  more » « less
Award ID(s):
1813713
PAR ID:
10342436
Author(s) / Creator(s):
; ; ;
Editor(s):
Chinn, C; Tan, E; Chan, C; Kali, Y.
Date Published:
Journal Name:
Proceedings of the 16th International Conference of the Learning Sciences—ICLS 2022
Page Range / eLocation ID:
2198-2199
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; (, International Society of the Learning Sciences)
    Blikstein, P; Van_Aalst, J; Kizito, R; Brennan, K (Ed.)
    This study takes advantage of advances in Natural Language Processing (NLP) to build an idea detection model that can identify ideas grounded in students’ linguistic experiences. We designed adaptive, interactive dialogs for four explanation items using the NLP idea detection model and investigated whether they similarly support students from distinct language backgrounds. The curriculum, assessments, and scoring rubrics were informed by the Knowledge Integration (KI) pedagogy. We analyzed responses of 1,036 students of different language backgrounds taught by 10 teachers in five schools in the western United States. The adaptive dialog engages students from both monolingual English and multilingual backgrounds in incorporating additional relevant ideas into their explanations, resulting in a significant improvement in student responses from initial to revised explanations. The guidance supports students in both language groups to progress in integrating their scientific ideas. 
    more » « less
  2. ; ; ; ; ; ; ; ; ; (, Proceedings of the 55th ACM Technical Symposium on Computer Science Education (SIGCSE))
    With the increasing prevalence of large language models (LLMs) such as ChatGPT, there is a growing need to integrate natural language processing (NLP) into K-12 education to better prepare young learners for the future AI landscape. NLP, a sub-field of AI that serves as the foundation of LLMs and many advanced AI applications, holds the potential to enrich learning in core subjects in K-12 classrooms. In this experience report, we present our efforts to integrate NLP into science classrooms with 98 middle school students across two US states, aiming to increase students’ experience and engagement with NLP models through textual data analyses and visualizations. We designed learning activities, developed an NLP-based interactive visualization platform, and facilitated classroom learning in close collaboration with middle school science teachers. This experience report aims to contribute to the growing body of work on integrating NLP into K-12 education by providing insights and practical guidelines for practitioners, researchers, and curriculum designers. 
    more » « less
  3. ; (, International Journal of Artificial Intelligence in Education)
    Abstract Argumentation, a key scientific practice presented in theFramework for K-12 Science Education, requires students to construct and critique arguments, but timely evaluation of arguments in large-scale classrooms is challenging. Recent work has shown the potential of automated scoring systems for open response assessments, leveraging machine learning (ML) and artificial intelligence (AI) to aid the scoring of written arguments in complex assessments. Moreover, research has amplified that the features (i.e., complexity, diversity, and structure) of assessment construct are critical to ML scoring accuracy, yet how the assessment construct may be associated with machine scoring accuracy remains unknown. This study investigated how the features associated with the assessment construct of a scientific argumentation assessment item affected machine scoring performance. Specifically, we conceptualized the construct in three dimensions: complexity, diversity, and structure. We employed human experts to code characteristics of the assessment tasks and score middle school student responses to 17 argumentation tasks aligned to three levels of a validated learning progression of scientific argumentation. We randomly selected 361 responses to use as training sets to build machine-learning scoring models for each item. The scoring models yielded a range of agreements with human consensus scores, measured by Cohen’s kappa (mean = 0.60; range 0.38 − 0.89), indicating good to almost perfect performance. We found that higher levels ofComplexityandDiversity of the assessment task were associated with decreased model performance, similarly the relationship between levels ofStructureand model performance showed a somewhat negative linear trend. These findings highlight the importance of considering these construct characteristics when developing ML models for scoring assessments, particularly for higher complexity items and multidimensional assessments. 
    more » « less
  4. ; (, Oxford University PressOxford)
    Natural language processing (NLP) tools can score students’ written explanations, opening new opportunities for science education. Optimally, these scores offer designers opportunities to align guidance with tested pedagogical frameworks and to investigate alternative ways to personalize instruction. We report on research, informed by the knowledge integration (KI) pedagogical framework, using online authorable and customizable environments (ACEs), to promote a deep understanding of complex scientific topics. We study how to personalize guidance to enable students to make productive revisions to written explanations during instruction, where they conduct investigations with models, simulations, hands-on activities, and other materials. We describe how we iteratively refined our assessments and guidance to support students in revising their scientific explanations. We report on recent investigations of hybrid models of personalized guidance that combine NLP scoring with opportunities for teachers to continue the conversation. 
    more » « less
  5. ; ; ; ; ; ; ; ; ; ; et al (, The Journal of Applied Instructional Design)
    Digital learning environments are used frequently in K-12 classrooms. Such use can require skillful orchestration as teachers need to understand the affordances of the learning environment, sequence of activities, and when and how to intervene with students. Using a digital learning environment in a multidisciplinary classroom context makes the design of support materials for teachers and students even more essential. To design for effective teacher orchestration in the classroom, we created a comprehensive set of materials for our multidisciplinary digital learning environment. We employ the design-based intervention research framework to trace the contextual and practical iterations these materials underwent. Additionally, we provide next steps for our work and considerations for the broader community. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email