Abstract As use of artificial intelligence (AI) has increased, concerns about AI bias and discrimination have been growing. This paper discusses an application called PyrEval in which natural language processing (NLP) was used to automate assessment and provide feedback on middle school science writing without linguistic discrimination. Linguistic discrimination in this study was operationalized as unfair assessment of scientific essays based on writing features that are not considered normative such as subject‐verb disagreement. Such unfair assessment is especially problematic when the purpose of assessment is not assessing English writing but rather assessing the content of scientific explanations. PyrEval was implemented in middle school science classrooms. Students explained their roller coaster design by stating relationships among such science concepts as potential energy, kinetic energy and law of conservation of energy. Initial and revised versions of scientific essays written by 307 eighth‐grade students were analyzed. Our manual and NLP assessment comparison analysis showed that PyrEval did not penalize student essays that contained non‐normative writing features. Repeated measures ANOVAs and GLMM analysis results revealed that essay quality significantly improved from initial to revised essays after receiving the NLP feedback, regardless of non‐normative writing features. Findings and implications are discussed. Practitioner notesWhat is already known about this topicAdvancement in AI has created a variety of opportunities in education, including automated assessment, but AI is not bias‐free.Automated writing assessment designed to improve students' scientific explanations has been studied.While limited, some studies reported biased performance of automated writing assessment tools, but without looking into actual linguistic features about which the tools may have discriminated.What this paper addsThis study conducted an actual examination of non‐normative linguistic features in essays written by middle school students to uncover how our NLP tool called PyrEval worked to assess them.PyrEval did not penalize essays containing non‐normative linguistic features.Regardless of non‐normative linguistic features, students' essay quality scores significantly improved from initial to revised essays after receiving feedback from PyrEval. Essay quality improvement was observed regardless of students' prior knowledge, school district and teacher variables.Implications for practice and/or policyThis paper inspires practitioners to attend to linguistic discrimination (re)produced by AI.This paper offers possibilities of using PyrEval as a reflection tool, to which human assessors compare their assessment and discover implicit bias against non‐normative linguistic features.PyrEval is available for use ongithub.com/psunlpgroup/PyrEvalv2.
more »
« less
The Plant Science Blogging Project: A curriculum to develop student science communication skills
Societal Impact StatementThe practice of writing science blogs benefits both the scientist and society alike by providing professional development opportunities and delivering information in a format that is accessible to large and diverse audiences. By designing a project that introduced upper‐level undergraduate students to science blog writing with a focus on plant biology, we piqued students' interest in science writing and the content of a popular plant science blog website. If adopted more widely, this work could broaden the scope of science education and promote the development of effective science communication skills for the next generation of scientists. SummarySuccessful scientists must communicate their research to broad audiences, including distilling key scientific concepts for the general public. Students pursuing careers in Science, Technology, Engineering, and Mathematics (STEM) fields benefit from developing public communication skills early in their careers, but opportunities are limited in traditional biology curricula.We created the “Plant Science Blogging Project” for a Plant Biology undergraduate course at the University of Pittsburgh in Fall 2018 and 2019. Students wrote blog posts merging personal connections with plants with plant biology concepts for the popular science blogsPlant Love StoriesandEvoBites. By weaving biology into their narratives, students learned how to share botanical knowledge with the general public.The project had positive impacts on student learning and public engagement. In post‐assignment surveys, the majority of students reported that they enjoyed the assignment, felt it improved their understanding of plant biology, and piqued their interest in reading and writing science blogs in the future. Approximately one‐third of the student‐authored blogs were published, including two that rose to the top 10 most‐read posts on Plant Love Stories. Some dominant themes in student blogs, including medicine and culture, differed from common story themes published on the web, indicating the potential for students to diversify science blog content.Overall, the Plant Science Blogging Project allows undergraduate students to engage with plant biology topics in a new way, sharpen their scientific communication skills in accordance with today's world of mass information sharing, and contribute to the spread of scientific knowledge for public benefit.
more »
« less
- PAR ID:
- 10445340
- Publisher / Repository:
- Wiley Blackwell (John Wiley & Sons)
- Date Published:
- Journal Name:
- PLANTS, PEOPLE, PLANET
- Volume:
- 4
- Issue:
- 5
- ISSN:
- 2572-2611
- Page Range / eLocation ID:
- p. 485-498
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
ABSTRACT Undergraduate research experiences (UREs) cultivate workforce skills, such as critical thinking, project management, and scientific communication. Many UREs in biophysical research have constraints related to limited resources, often resulting in smaller student cohorts, barriers for students entering a research environment, and fewer mentorship opportunities for graduate students. In response to those limitations, we have created a structured URE model that uses an asynchronous training style paired with direct-tiered mentoring delivered by peers, graduate students, and faculty. The adaptive undergraduate research training and experience (AURTE) framework was piloted as part of the Brown Experiential Learning program, a computational biophysics research lab. The program previously demonstrated substantial increases and improvements in the number of students served and skills developed. Here, we discuss the long-term effectiveness of the framework, impacts on graduate and undergraduate students, and efficacy in teaching research skills and computational-based biophysical methods. The longitudinal impact of our structured URE on student outcomes was analyzed by using student exit surveys, interviews, assessments, and 5 years of feedback from alumni. Results indicate high levels of student retention in research compared with university-wide metrics. Also, student feedback emphasizes how tiered mentoring enhanced research skill retention, while allowing graduate mentors to develop mentorship and workforce skills to expedite research. Responses from alumni affirm that workforce-ready skills (communicating science, data management, and scientific writing) acquired in the program persisted and were used in postgraduate careers. The framework reinforces the importance of establishing, iterating, and evaluating a structured URE framework to foster student success in biophysical research, while promoting mentorship skill training for graduate students. Future work will explore the adaptability of the framework in wet lab environments and probe the potential of AURTE in broader educational contexts.more » « less
-
IntroductionIt is critical for STEM students to be able to discuss science with diverse audiences, yet many STEM students do not receive adequate training in these skills. When students have the skills to communicate about science, they may feel a resulting sense of empowerment as a scientist as well as help members of society understand science. MethodsIn this study, we developed, implemented, and evaluated a workshop that gave students understanding of and practice in applying Inclusive Science Communication. We assessed the workshop via a mixed-methods approach. ResultsWe quantified student affective measures that are associated with STEM persistence, such as science self-efficacy and science identity, showing that the workshop increased these measures both for students of marginalized identities and for students who do not hold these identities. We also assessed student open-ended responses for themes related to the Theory of Planned Behavior, Community Cultural Wealth, and White Supremacy Culture, finding that forms of cultural capital empowered students to perform science communication behaviors while power imbalances, fear of conflict, and perfectionism presented barriers to these behaviors. DiscussionThis study highlights the importance of providing explicit training and practice in Inclusive Science Communication for undergraduate STEM students. Our results also suggest that students need the opportunity for reflexivity – that is, the practice of reflecting upon their identities and motivations – in order to develop in their identity and confidence as scientists and science communicators.more » « less
-
Identity and belonging are key components to student longevity in STEM and are particularly important for retaining BIPOC, low-income, and first generation students. Previous work emphasizes the importance of scientific competence, research performance, and recognition by scientific community members as key to building a STEM identity that persists (Carlone & Johnson, 2007), increasing the likelihood of retention into the STEM workforce. The Keck Geology Consortium REU, with funding from NSF, developed the Gateway program for rising sophomores in 2017 to support and build these three components of science identity for underrepresented undergraduates in the geosciences. Here we present stories from the Glacier National Park, Montana environmental geoscience Gateway project, alongside consortium-wide data, and consider ways in which scientific identity is developed. In summers 2018 and 2022 we brought together groups of 5-8 students, mostly from underrepresented groups, all of whom were finishing their first year of college. Students were from a variety of backgrounds and institutions, but primarily from SLACs, and had little/no prior research experience. Projects lasted 5 weeks, with one week of pre-field knowledge and skill development (competence) two weeks in the field in eastern Glacier National Park collecting data, and two weeks at the Continental Scientific Drilling (CSD) laboratory at the University of Minnesota analyzing sediment cores. Students presented their research at subsequent Geological Society of America meetings (performance and recognition). Survey results and anecdotal stories suggest all three key aspects of STEM identity-building were supported by the projects. In addition to identity and values activities at the project start, blogging and reflection activities supported development and integration of a science identity. Performance and recognition opportunities were threaded throughout; students shared their research with the public (Park visitors encountered in the field), with Park Rangers/Interpretive staff (during informal presentations), and with staff, students, and visiting scientists at the CSD (daily interactions in the lab). These less-official forms of performance and recognition greatly contributed to the students’ sense of ownership and expertise.more » « less
-
Abstract Preparing preservice teachers (PSTs) to be able to notice, interpret, respond to and orchestrate student ideas—the core practices of responsive teaching—is a key goal for contemporary science and mathematics teacher education. This mixed‐methods study, employing a virtual reality (VR)‐supported simulation integrated with artificial intelligence (AI)‐powered virtual students, explored the frequent patterns of PSTs' talk moves as they attempted to orchestrate a responsive discussion, as well as the affordances and challenges of leveraging AI‐supported virtual simulation to enhance PSTs' responsive teaching skills. Sequential analysis of the talk moves of both PSTs (n = 24) and virtual students indicated that although PSTs did employ responsive talk moves, they encountered difficulties in transitioning from the authoritative, teacher‐centred teaching approach to a responsive way of teaching. The qualitative analysis with triangulated dialogue transcripts, observational field notes and semi‐structured interviews revealed participants' engagement in (1) orchestrating discussion by leveraging the design features of AI‐supported simulation, (2) iterative rehearsals through naturalistic and contextualized interactions and (3) exploring realism and boundaries in AI‐powered virtual students. The study findings provide insights into the potential of leveraging AI‐supported virtual simulation to improve PSTs' responsive teaching skills. The study also underscores the need for PSTs to engage in well‐designed pedagogical practices with adaptive and in situ support. Practitioner notesWhat is already known about this topicDeveloping the teaching capacity of responsive teaching is an important goal for preservice teacher (PST) education. PSTs need systematic opportunities to build fluency in this approach.Virtual simulations can provide PSTs with the opportunities to practice interactive teaching and have been shown to improve their teaching skills.Artificial intelligence (AI)‐powered virtual students can be integrated into virtual simulations to enable interactive and authentic practice of teaching.What this paper addsAI‐supported simulation has the potential to support PSTs' responsive teaching skills.While PSTs enact responsive teaching talk moves, they struggle to enact those talk moves in challenging teaching scenarios due to limited epistemic and pedagogical resources.AI‐supported simulation affords iterative and contextualized opportunities for PSTs to practice responsive teaching talk moves; it challenges teachers to analyse student discourse and respond in real time.Implications for practice and/or policyPSTs should build a teaching repertoire with both basic and advanced responsive talk moves.The learning module should adapt to PSTs' prior experience and provide PSTs with in situ learning support to navigate challenging teaching scenarios.Integrating interaction features and AI‐based virtual students into the simulation can facilitate PSTs' active participation.more » « less
An official website of the United States government
