Argumentation is fundamental to science education, both as a prominent feature of scientific reasoning and as an effective mode of learning—a perspective reflected in contemporary frameworks and standards. The successful implementation of argumentation in school science, however, requires a paradigm shift in science assessment from the measurement of knowledge and understanding to the measurement of performance and knowledge in use. Performance tasks requiring argumentation must capture the many ways students can construct and evaluate arguments in science, yet such tasks are both expensive and resource‐intensive to score. In this study we explore how machine learning text classification techniques can be applied to develop efficient, valid, and accurate constructed‐response measures of students' competency with written scientific argumentation that are aligned with a validated argumentation learning progression. Data come from 933 middle school students in the San Francisco Bay Area and are based on three sets of argumentation items in three different science contexts. The findings demonstrate that we have been able to develop computer scoring models that can achieve substantial to almost perfect agreement between human‐assigned and computer‐predicted scores. Model performance was slightly weaker for harder items targeting higher levels of the learning progression, largely due to the linguistic complexity of these responses and the sparsity of higher‐level responses in the training data set. Comparing the efficacy of different scoring approaches revealed that breaking down students' arguments into multiple components (e.g., the presence of an accurate claim or providing sufficient evidence), developing computer models for each component, and combining scores from these analytic components into a holistic score produced better results than holistic scoring approaches. However, this analytical approach was found to be differentially biased when scoring responses from English learners (EL) students as compared to responses from non‐EL students on some items. Differences in the severity between human and computer scores for EL between these approaches are explored, and potential sources of bias in automated scoring are discussed.
Making Sense of Uncertainty in the Science Classroom: A Bayesian Approach
Abstract Uncertainty is ubiquitous in science, but scientific knowledge is often represented to the public and in educational contexts as certain and immutable. This contrast can foster distrust when scientific knowledge develops in a way that people perceive as a reversals, as we have observed during the ongoing COVID-19 pandemic. Drawing on research in statistics, child development, and several studies in science education, we argue that a Bayesian approach can support science learners to make sense of uncertainty. We provide a brief primer on Bayes’ theorem and then describe three ways to make Bayesian reasoning practical in K-12 science education contexts. There are a) using principles informed by Bayes’ theorem that relate to the nature of knowing and knowledge, b) interacting with a web-based application (or widget—Confidence Updater) that makes the calculations needed to apply Bayes’ theorem more practical, and c) adopting strategies for supporting even young learners to engage in Bayesian reasoning. We conclude with directions for future research and sum up how viewing science and scientific knowledge from a Bayesian perspective can build trust in science.
more »
« less
- Award ID(s):
- 1937700
- NSF-PAR ID:
- 10377491
- Date Published:
- Journal Name:
- Science & Education
- Volume:
- 31
- Issue:
- 5
- ISSN:
- 0926-7220
- Page Range / eLocation ID:
- 1239 to 1262
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
Abstract -
This article expands upon my presentation to the panel on “The Radical Prescription for Change” at the 2017 ASA (American Statistical Association) symposium on A World Beyond $p<0.05$. It emphasizes that, to greatly enhance the reliability of—and hence public trust in—statistical and data scientific findings, we need to take a holistic approach. We need to lead by example, incentivize study quality, and inoculate future generations with profound appreciations for the world of uncertainty and the uncertainty world. The four “radical” proposals in the title—with all their inherent defects and trade-offs—are designed to provoke reactions and actions. First, research methodologies are trustworthy only if they deliver what they promise, even if this means that they have to be overly protective, a necessary trade-off for practicing quality-guaranteed statistics. This guiding principle may compel us to doubling variance in some situations, a strategy that also coincides with the call to raise the bar from $p<0.05$ to $p<0.005$ [3]. Second, teaching principled practicality or corner-cutting is a promising strategy to enhance the scientific community’s as well as the general public’s ability to spot—and hence to deter—flawed arguments or findings. A remarkable quick-and-dirty Bayes formula for rare events, which simply divides the prevalence by the sum of the prevalence and the false positive rate (or the total error rate), as featured by the popular radio show Car Talk, illustrates the effectiveness of this strategy. Third, it should be a routine mental exercise to put ourselves in the shoes of those who would be affected by our research finding, in order to combat the tendency of rushing to conclusions or overstating confidence in our findings. A pufferfish/selfish test can serve as an effective reminder, and can help to institute the mantra “Thou shalt not sell what thou refuseth to buy” as the most basic professional decency. Considering personal stakes in our statistical endeavors also points to the concept of behavioral statistics, in the spirit of behavioral economics. Fourth, the current mathematical education paradigm that puts “deterministic first, stochastic second” is likely responsible for the general difficulties with reasoning under uncertainty, a situation that can be improved by introducing the concept of histogram, or rather kidstogram, as early as the concept of counting.more » « less
-
Seagroves, Scott ; Barnes, Austin ; Metevier, Anne ; Porter, Jason ; Hunter, Lisa (Ed.)Research suggests that developing an identity as a person in STEM is necessary for learners from marginalized groups to persist in STEM education and careers. These learners may perceive that their race, gender, or other characteristics make it difficult for their peers and supervisors to recognize them as scientists or engineers, thus disrupting their ability to maintain successful degree progress and to pursue their STEM career aspirations. Here we discuss the specific ways we designed inquiry workshops to not only clarify difficult core STEM content, but to also promote learners’ competence, performance, and targeted recognition as scientists. Our workshops were designed for students interested in chemistry, climate science, physics, and toxicology at the University of California, Santa Cruz (UCSC), Workshops for Engineering & Science Transfers (WEST) 2019 program. In designing our workshops, we focused on promoting the scientific identities of our learners by incorporating authentic ways for students to receive recognition from both peers and instructional facilitators, as well as allowing students to tap into their own personal interests and values. Insights from our designed assessments for learners’ understanding of our content demonstrate the success of our initiatives and provide further areas of improvement. Our goals are to create inclusive workshops to support students from all backgrounds, with emphasis on underrepresented backgrounds (community college, first generation, students of color, women, and LGBTQ+ students, etc.) as well as support them in other contexts, such as when mentoring STEM students in academic laboratory settings.more » « less
-
For many decades, educational communities, including computing education, have debated the value of telling students what they need to know (i.e., direct instruction) compared to guiding them to construct knowledge themselves (i.e., constructivism). Comparisons of these two instructional approaches have inconsistent results. Direct instruction can be more efficient for short-term performance but worse for retention and transfer. Constructivism can produce better retention and transfer, but this outcome is unreliable. To contribute to this debate, we propose a new theory to better explain these research results. Our theory, multiple conceptions theory, states that learners develop better conceptual knowledge when they are guided to compare multiple conceptions of a concept during instruction. To examine the validity of this theory, we used this lens to evaluate the literature for eight instructional techniques that guide learners to compare multiple conceptions, four from direct instruction (i.e., test-enhanced learning, erroneous examples, analogical reasoning, and refutation texts) and four from constructivism (i.e., productive failure, ambitious pedagogy, problem-based learning, and inquiry learning). We specifically searched for variations in the techniques that made them more or less successful, the mechanisms responsible, and how those mechanisms promote conceptual knowledge, which is critical for retention and transfer. To make the paper directly applicable to education, we propose instructional design principles based on the mechanisms that we identified. Moreover, we illustrate the theory by examining instructional techniques commonly used in computing education that compare multiple conceptions. Finally, we propose ways in which this theory can advance our instruction in computing and how computing education researchers can advance this general education theory.more » « less
-
Humans frequently make inferences about uncertain future events with limited data. A growing body of work suggests that infants and other primates make surprisingly sophisticated inferences under uncertainty. First, we ask what underlying cognitive mechanisms allow young learners to make such sophisticated inferences under uncertainty. We outline three possibilities, the logic, probabilistic, and heuristics views, and assess the empirical evidence for each. We argue that the weight of the empirical work favors the probabilistic view, in which early reasoning under uncertainty is grounded in inferences about the relationship between samples and populations as opposed to being grounded in simple heuristics. Second, we discuss the apparent contradiction between this early-emerging sensitivity to probabilities with the decades of literature suggesting that adults show limited use of base-rate and sampling principles in their inductive inferences. Third, we ask how these early inductive abilities can be harnessed for improving later mathematics education and inductive inference. We make several suggestions for future empirical work that should go a long way in addressing the many remaining open questions in this growing research area.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1007/s11191-022-00341-3
