An official website of the United States government
In this paper, we present a science writing assignment in which students focus on targeting specific audiences when writing about a socioscientific issue as well as participate in a peer review process. This assignment helps students consider inclusive science communication in their writing, focusing on engaging unique audiences about the intersections of science and social justice. Students are introduced to evidence-based tools for formulating communication for unique audiences as well as for assessment of writing quality. This assignment is novel in that it helps students think about inclusion issues in STEM, science writing, and peer review, all of which are key disciplinary skills that are not always included in STEM courses. While this assignment was piloted in chemistry and environmental engineering courses, this assignment could easily be modified for other disciplines.
more »
« less
Automated Support to Scaffold Students’ Written Explanations in Science
In principle, educators can use writing to scaffold students’ understanding of increasingly complex science ideas. In practice, formative assessment of students’ science writing is very labor intensive. We present PyrEval+CR, an automated tool for formative assessment of middle school students’ science essays. It identifies each idea in a student’s science essay, and its importance in the curriculum.
more »
« less
- Award ID(s):
- 2010483
- PAR ID:
- 10329342
- Date Published:
- Journal Name:
- International Conference on Artificial Intelligence in Education
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Martin Fred; Norouzi, Narges; Rosenthal, Stephanie (Ed.)This paper examines the use of LLMs to support the grading and explanation of short-answer formative assessments in K12 science topics. While significant work has been done on programmatically scoring well-structured student assessments in math and computer science, many of these approaches produce a numerical score and stop short of providing teachers and students with explanations for the assigned scores. In this paper, we investigate few-shot, in-context learning with chain-of-thought reasoning and active learning using GPT-4 for automated assessment of students’ answers in a middle school Earth Science curriculum. Our findings from this human-in-the-loop approach demonstrate success in scoring formative assessment responses and in providing meaningful explanations for the assigned score. We then perform a systematic analysis of the advantages and limitations of our approach. This research provides insight into how we can use human-in-the-loop methods for the continual improvement of automated grading for open-ended science assessments.more » « less
-
Teachers in small communities may be geographically isolated and have smaller collegial networks. Consequently, teachers in these settings may have limited exposure to contemporary strategies for engaging learners in science and engineering as suggested in the Next Generation Science Standards (NGSS). Thus, we provided a 5-day online PL experience and a year-long of modest supports (e.g., online professional learning community) to over 150 rural teachers from four states (CA, MT, ND, WY) to bridge the access gap and to enhance their instructional capabilities in teaching NGSS-aligned science and engineering lessons. Considering that the quality of the questions posed in a formative assessment impacts the quality of student thinking and what it reveals, we provided a formative assessment task, “Planning a Park” developed by Stanford NGSS Assessment Project (SNAP) and SCALE Science at WestEd, to participating teachers to implement in their classrooms. Teachers received online professional learning opportunities about the task before and after administering it in their classrooms. To understand their experiences with the task, we collected multiple data sources for triangulation, such as surveys about teachers’ preparedness to implement science lessons, teachers’ self-reported observations while delivering the task, their reflections about students’ performance, examples of student responses to the task, and interview responses from a sub-sample of teachers. As an initial analysis, we employed a descriptive coding process to capture teachers’ diverse experiences with the SCALE task (Saldaña, 2021). In this session, we will report rural teachers’ experiences with the formative assessment task that was provided as part of a year of modest supports. We believe this study will support the science education community, especially individuals preparing teachers to teach science and researchers on assessment, by sharing the benefits of implementing a formative assessment task during inservice teachers’ professional learning.more » « less
-
As use of artificial intelligence (AI) has increased, concerns about AI bias and discrimination have been growing. This paper discusses an application called PyrEval in which natural language processing (NLP) was used to automate assessment and pro- vide feedback on middle school science writing with- out linguistic discrimination. Linguistic discrimination in this study was operationalized as unfair assess- ment of scientific essays based on writing features that are not considered normative such as subject- verb disagreement. Such unfair assessment is espe- cially problematic when the purpose of assessment is not assessing English writing but rather assessing the content of scientific explanations. PyrEval was implemented in middle school science classrooms. Students explained their roller coaster design by stat- ing relationships among such science concepts as potential energy, kinetic energy and law of conser- vation of energy. Initial and revised versions of sci- entific essays written by 307 eighth- grade students were analyzed. Our manual and NLP assessment comparison analysis showed that PyrEval did not pe- nalize student essays that contained non-normative writing features. Repeated measures ANOVAs and GLMM analysis results revealed that essay quality significantly improved from initial to revised essays after receiving the NLP feedback, regardless of non- normative writing features. Findings and implications are discussed.more » « less
-
As use of artificial intelligence (AI) has increased, concerns about AI bias and discrimination have been growing. This paper discusses an application called PyrEval in which natural language processing (NLP) was used to automate assessment and pro- vide feedback on middle school science writing with- out linguistic discrimination. Linguistic discrimination in this study was operationalized as unfair assess- ment of scientific essays based on writing features that are not considered normative such as subject- verb disagreement. Such unfair assessment is espe- cially problematic when the purpose of assessment is not assessing English writing but rather assessing the content of scientific explanations. PyrEval was implemented in middle school science classrooms. Students explained their roller coaster design by stat- ing relationships among such science concepts as potential energy, kinetic energy and law of conser- vation of energy. Initial and revised versions of sci- entific essays written by 307 eighth- grade students were analyzed. Our manual and NLP assessment comparison analysis showed that PyrEval did not pe- nalize student essays that contained non-normative writing features. Repeated measures ANOVAs and GLMM analysis results revealed that essay quality significantly improved from initial to revised essays after receiving the NLP feedback, regardless of non- normative writing features. Findings and implications are discussed.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
