Abstract
Expert testimony varies in scientific quality and jurors have a difficult time evaluating evidence quality (McAuliff et al., 2009). In the current study, we apply Fuzzy Trace Theory principles, examining whether visual and gist aids help jurors calibrate to the strength of scientific evidence. Additionally we were interested in the role of jurors’ individual differences in scientific reasoning skills in their understanding of case evidence. Contrary to our preregistered hypotheses, there was no effect of evidence condition or gist aid on evidence understanding. However, individual differences between jurors’ numeracy skills predicted evidence understanding.
Summary
Poor-quality expert evidence is sometimes admitted into court (Smithburn, 2004). Jurors’ calibration to evidence strength varies widely and is not robustly understood. For instance, previous research has established jurors lack understanding of the role of control groups, confounds, and sample sizes in scientific research (McAuliff, Kovera, & Nunez, 2009; Mill, Gray, & Mandel, 1994). Still others have found that jurors can distinguish weak from strong evidence when the evidence is presented alone, yet not when simultaneously presented with case details (Smith, Bull, & Holliday, 2011). This research highlights the need to present evidence to jurors in a way they can understand. Fuzzy Trace Theory purports that people encode information in exact, verbatim representations and through “gist” representations, which represent summary of meaning (Reyna & Brainerd, 1995). It is possible that the presenting complex scientific evidence to people with verbatim content or appealing to the gist, or bottom-line meaning of the information may influence juror understanding of that evidence. Application of Fuzzy Trace Theory in the medical field has shown that gist representations are beneficial for helping laypeople better understand risk and benefits of medical treatment (Brust-Renck, Reyna, Wilhelms, & Lazar, 2016). Yet, little research has applied Fuzzy Trace Theory to information comprehension and application within the context of a jury (c.f. Reyna et. al., 2015). Additionally, it is likely that jurors’ individual characteristics, such as scientific reasoning abilities and cognitive tendencies, influence their ability to understand and apply complex scientific information (Coutinho, 2006).
Methods
The purpose of this study was to examine how jurors calibrate to the strength of scientific information, and whether individual difference variables and gist aids inspired by Fuzzy Trace Theory help jurors better understand complicated science of differing quality. We used a 2 (quality of scientific evidence: high vs. low) x 2 (decision aid to improve calibration - gist information vs. no gist information), between-subjects design. All hypotheses were preregistered on the Open Science Framework.
Jury-eligible community participants (430 jurors across 90 juries; Mage = 37.58, SD = 16.17, 58% female, 56.93% White). Each jury was randomly assigned to one of the four possible conditions. Participants were asked to individually fill out measures related to their scientific reasoning skills prior to watching a mock jury trial. The trial was about an armed bank robbery and consisted of various pieces of testimony and evidence (e.g. an eyewitness testimony, police lineup identification, and a sweatshirt found with the stolen bank money). The key piece of evidence was mitochondrial DNA (mtDNA) evidence collected from hair on a sweatshirt (materials from Hans et al., 2011). Two experts presented opposing opinions about the scientific evidence related to the mtDNA match estimate for the defendant’s identification. The quality and content of this mtDNA evidence differed based on the two conditions. The high quality evidence condition used a larger database than the low quality evidence to compare to the mtDNA sample and could exclude a larger percentage of people. In the decision aid condition, experts in the gist information group presented gist aid inspired visuals and examples to help explain the proportion of people that could not be excluded as a match. Those in the no gist information group were not given any aid to help them understand the mtDNA evidence presented. After viewing the trial, participants filled out a questionnaire on how well they understood the mtDNA evidence and their overall judgments of the case (e.g. verdict, witness credibility, scientific evidence strength). They filled this questionnaire out again after a 45-minute deliberation.
Measures
We measured Attitudes Toward Science (ATS) with indices of scientific promise and scientific reservations (Hans et al., 2011; originally developed by National Science Board, 2004; 2006). We used Drummond and Fischhoff’s (2015) Scientific Reasoning Scale (SRS) to measure scientific reasoning skills. Weller et al.’s (2012) Numeracy Scale (WNS) measured proficiency in reasoning with quantitative information. The NFC-Short Form (Cacioppo et al., 1984) measured need for cognition. We developed a 20-item multiple-choice comprehension test for the mtDNA scientific information in the cases (modeled on Hans et al., 2011, and McAuliff et al., 2009). Participants were shown 20 statements related to DNA evidence and asked whether these statements were True or False. The test was then scored out of 20 points.
Results
For this project, we measured calibration to the scientific evidence in a few different ways. We are building a full model with these various operationalizations to be presented at APLS, but focus only on one of the calibration DVs (i.e., objective understanding of the mtDNA evidence) in the current proposal. We conducted a general linear model with total score on the mtDNA understanding measure as the DV and quality of scientific evidence condition, decision aid condition, and the four individual difference measures (i.e., NFC, ATS, WNS, and SRS) as predictors. Contrary to our main hypotheses, neither evidence quality nor decision aid condition affected juror understanding. However, the individual difference variables did: we found significant main effects for Scientific Reasoning Skills, F(1, 427) = 16.03, p <.001, np2 = .04, Weller Numeracy Scale, F(1, 427) = 15.19, p <.001, np2 = .03, and Need for Cognition, F(1, 427) = 16.80, p <.001, np2 = .04, such that those who scored higher on these measures displayed better understanding of the scientific evidence. In addition there was a significant interaction of evidence quality condition and scores on the Weller’s Numeracy Scale, F(1, 427) = 4.10, p = .04, np2 = .01. Further results will be discussed.
Discussion
These data suggest jurors are not sensitive to differences in the quality of scientific mtDNA evidence, and also that our attempt at helping sensitize them with Fuzzy Trace Theory-inspired aids did not improve calibration. Individual scientific reasoning abilities and general cognition styles were better predictors of understanding this scientific information. These results suggest a need for further exploration of approaches to help jurors differentiate between high and low quality evidence.
Note: The 3rd author was supported by an AP-LS AP Award for her role in this research.
Learning Objective:
Participants will be able to describe how individual differences in scientific reasoning skills help jurors understand complex scientific evidence.
more »
« less
Third Graders' Interpretations of Subtraction Worked Examples: Matching Number Sentences and Visuals [Roundtable Session]
This study investigated 37 third graders’ explanations of subtraction worked examples shown in number sentence or visual form (ten frame or number line) and their justifications for which
visual and numerical worked examples corresponded to the same subtraction strategy. Results showed that third graders gave more detailed explanations in number sentence form than in visual form; whereas, they had higher accuracy in matching number sentences to visuals than vice versa. When matching, they were more likely to reason sufficiently when identifying processes represented in the worked examples as opposed to reasoning about the order of the numbers. When using worked examples, teachers should make use of visuals to help students focus on how the visuals represent the operations.
more »
« less
- Award ID(s):
- 1759254
- NSF-PAR ID:
- 10168731
- Date Published:
- Journal Name:
- Proceedings of the 2020 AERA Annual Meeting [Conference Canceled]
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
In a cross-sectional study, 160 students in Grades 2, 4, 7, and 11 were interviewed about their reasoning when solving integer addition and subtraction open-number sentence problems. We applied our previously developed framework for 5 Ways of Reasoning (WoRs) to our data set to describe patterns within and across participant groups. Our analysis of the WoRs also led to the identification of 3 problem types: change-positive, all-negatives, and counterintuitive. We found that problem type influenced student performance and tended to evoke a different way of reasoning. We showed that those with more experience with negative numbers use WoRs more flexibly than those with less experience and that flexibility is correlated with accuracy. We provide 3 types of resources for educators: (a) WoRs and problem-types frameworks, (b) characterization of flexibility with integer addition and subtraction, and (c) development of a trajectory of learning about integers.more » « less
-
This study aims to investigate the collaboration processes of immigrant families as they search for online information together. Immigrant English-language learning adults of lower socioeconomic status often work collaboratively with their children to search the internet. Family members rely on each other’s language and digital literacy skills in this collaborative process known as online search and brokering (OSB). While previous work has identified ecological factors that impact OSB, research has not yet distilled the specific learning processes behind such collaborations. Design/methodology/approach: For this study, the authors adhere to practices of a case study examination. This study’s participants included parents, grandparents and children aged 10–17 years. Most adults were born in Mexico, did not have a college-degree, worked in service industries and represented a lower-SES population. This study conducted two to three separate in-home family visits per family with interviews and online search tasks. Findings: From a case study analysis of three families, this paper explores the funds of knowledge, resilience, ecological support and challenges that children and parents face, as they engage in collaborative OSB experiences. This study demonstrates how in-home computer-supported collaborative processes are often informal, social, emotional and highly relevant to solving information challenges. Research limitations/implications: An intergenerational OSB process is different from collaborative online information problem-solving that happens between classroom peers or coworkers. This study’s research shows how both parents and children draw on their funds of knowledge, resilience and ecological support systems when they search collaboratively, with and for their family members, to problem solve. This is a case study of three families working in collaboration with each other. This case study informs analytical generalizations and theory-building rather than statistical generalizations about families. Practical implications: Designers need to recognize that children and youth are using the same tools as adults to seek high-level critical information. This study’s model suggests that if parents and children are negotiating information seeking with the same technology tools but different funds of knowledge, experience levels and skills, the presentation of information (e.g. online search results, information visualizations) needs to accommodate different levels of understanding. This study recommends designers work closely with marginalized communities through participatory design methods to better understand how interfaces and visuals can help accommodate youth invisible work. Social implications: The authors have demonstrated in this study that learning and engaging in family online searching is not only vital to the development of individual and digital literacy skills, it is a part of family learning. While community services, libraries and schools have a responsibility to support individual digital and information literacy development, this study’s model highlights the need to recognize funds of knowledge, family resiliency and asset-based learning. Schools and teachers should identify and harness youth invisible work as a form of learning at home. The authors believe educators can do this by highlighting the importance of information problem solving in homes and youth in their families. Libraries and community centers also play a critical role in supporting parents and adults for technical assistance (e.g. WiFi access) and information resources. Originality/value: This study’s work indicates new conditions fostering productive joint media engagement (JME) around OSB. This study contributes a generative understanding that promotes studying and designing for JME, where family responsibility is the focus.more » « less
-
more » « less
Abstract This study explored how different formats of instructional visuals affect the accuracy of students' metacognitive judgments. Undergraduates (
n = 133) studied a series of five biology texts and made judgments of learning. Students were assigned randomly to study the texts only (text only), study the texts with provided visuals (provided visuals group), study the texts and generate their own visuals (learner‐generated visuals group), or study the texts and observe animations of instructor‐generated visuals (instructor‐generated visuals group). After studying the texts and making judgments of learning, all students completed multiple‐choice comprehension tests on each text. The learner‐generated and instructor‐generated visuals groups exhibited significantly higher relative judgment accuracy than the text only and provided visuals groups, though this effect was relatively small. The learner‐generated visuals group also required more study time and was more likely to report the use of visual cues when making their judgments of learning. -
more » « less
Abstract Undergraduates (
n = 132) learned about the human respiratory system and then taught what they learned by explaining aloud on video. Following a 2 × 2 design, students either generated their own words or visuals on paper while explaining aloud, or they viewed instructor‐provided words or visuals while explaining aloud. One week after teaching, students completed explanation, drawing, and transfer tests. Teaching with provided or generated visualizations resulted in significantly higher transfer test performance than teaching with provided or generated words. Furthermore, teaching with provided visuals led to significantly higher drawing test performance than teaching with generated visuals. Finally, the number of elaborations in students' explanations during teaching did not significantly differ across groups but was significantly associated with subsequent explanation and transfer test performance. Overall, the findings partially support the hypothesis that visuals facilitate learning by explaining, yet the benefits appeared stronger for instructor‐provided visuals than learner‐generated drawings.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
