More Like this

1. Predictors of Jurors’ Understanding of Evidence Strength

   McCowan, K. ( March 2020 , Annual Conference of the American Psychology-Law Society.)

   Abstract Expert testimony varies in scientific quality and jurors have a difficult time evaluating evidence quality (McAuliff et al., 2009). In the current study, we apply Fuzzy Trace Theory principles, examining whether visual and gist aids help jurors calibrate to the strength of scientific evidence. Additionally we were interested in the role of jurors’ individual differences in scientific reasoning skills in their understanding of case evidence. Contrary to our preregistered hypotheses, there was no effect of evidence condition or gist aid on evidence understanding. However, individual differences between jurors’ numeracy skills predicted evidence understanding. Summary Poor-quality expert evidence is sometimes admitted into court (Smithburn, 2004). Jurors’ calibration to evidence strength varies widely and is not robustly understood. For instance, previous research has established jurors lack understanding of the role of control groups, confounds, and sample sizes in scientific research (McAuliff, Kovera, & Nunez, 2009; Mill, Gray, & Mandel, 1994). Still others have found that jurors can distinguish weak from strong evidence when the evidence is presented alone, yet not when simultaneously presented with case details (Smith, Bull, & Holliday, 2011). This research highlights the need to present evidence to jurors in a way they can understand. Fuzzy Trace Theory purports that people encode information in exact, verbatim representations and through “gist” representations, which represent summary of meaning (Reyna & Brainerd, 1995). It is possible that the presenting complex scientific evidence to people with verbatim content or appealing to the gist, or bottom-line meaning of the information may influence juror understanding of that evidence. Application of Fuzzy Trace Theory in the medical field has shown that gist representations are beneficial for helping laypeople better understand risk and benefits of medical treatment (Brust-Renck, Reyna, Wilhelms, & Lazar, 2016). Yet, little research has applied Fuzzy Trace Theory to information comprehension and application within the context of a jury (c.f. Reyna et. al., 2015). Additionally, it is likely that jurors’ individual characteristics, such as scientific reasoning abilities and cognitive tendencies, influence their ability to understand and apply complex scientific information (Coutinho, 2006). Methods The purpose of this study was to examine how jurors calibrate to the strength of scientific information, and whether individual difference variables and gist aids inspired by Fuzzy Trace Theory help jurors better understand complicated science of differing quality. We used a 2 (quality of scientific evidence: high vs. low) x 2 (decision aid to improve calibration - gist information vs. no gist information), between-subjects design. All hypotheses were preregistered on the Open Science Framework. Jury-eligible community participants (430 jurors across 90 juries; Mage = 37.58, SD = 16.17, 58% female, 56.93% White). Each jury was randomly assigned to one of the four possible conditions. Participants were asked to individually fill out measures related to their scientific reasoning skills prior to watching a mock jury trial. The trial was about an armed bank robbery and consisted of various pieces of testimony and evidence (e.g. an eyewitness testimony, police lineup identification, and a sweatshirt found with the stolen bank money). The key piece of evidence was mitochondrial DNA (mtDNA) evidence collected from hair on a sweatshirt (materials from Hans et al., 2011). Two experts presented opposing opinions about the scientific evidence related to the mtDNA match estimate for the defendant’s identification. The quality and content of this mtDNA evidence differed based on the two conditions. The high quality evidence condition used a larger database than the low quality evidence to compare to the mtDNA sample and could exclude a larger percentage of people. In the decision aid condition, experts in the gist information group presented gist aid inspired visuals and examples to help explain the proportion of people that could not be excluded as a match. Those in the no gist information group were not given any aid to help them understand the mtDNA evidence presented. After viewing the trial, participants filled out a questionnaire on how well they understood the mtDNA evidence and their overall judgments of the case (e.g. verdict, witness credibility, scientific evidence strength). They filled this questionnaire out again after a 45-minute deliberation. Measures We measured Attitudes Toward Science (ATS) with indices of scientific promise and scientific reservations (Hans et al., 2011; originally developed by National Science Board, 2004; 2006). We used Drummond and Fischhoff’s (2015) Scientific Reasoning Scale (SRS) to measure scientific reasoning skills. Weller et al.’s (2012) Numeracy Scale (WNS) measured proficiency in reasoning with quantitative information. The NFC-Short Form (Cacioppo et al., 1984) measured need for cognition. We developed a 20-item multiple-choice comprehension test for the mtDNA scientific information in the cases (modeled on Hans et al., 2011, and McAuliff et al., 2009). Participants were shown 20 statements related to DNA evidence and asked whether these statements were True or False. The test was then scored out of 20 points. Results For this project, we measured calibration to the scientific evidence in a few different ways. We are building a full model with these various operationalizations to be presented at APLS, but focus only on one of the calibration DVs (i.e., objective understanding of the mtDNA evidence) in the current proposal. We conducted a general linear model with total score on the mtDNA understanding measure as the DV and quality of scientific evidence condition, decision aid condition, and the four individual difference measures (i.e., NFC, ATS, WNS, and SRS) as predictors. Contrary to our main hypotheses, neither evidence quality nor decision aid condition affected juror understanding. However, the individual difference variables did: we found significant main effects for Scientific Reasoning Skills, F(1, 427) = 16.03, p <.001, np2 = .04, Weller Numeracy Scale, F(1, 427) = 15.19, p <.001, np2 = .03, and Need for Cognition, F(1, 427) = 16.80, p <.001, np2 = .04, such that those who scored higher on these measures displayed better understanding of the scientific evidence. In addition there was a significant interaction of evidence quality condition and scores on the Weller’s Numeracy Scale, F(1, 427) = 4.10, p = .04, np2 = .01. Further results will be discussed. Discussion These data suggest jurors are not sensitive to differences in the quality of scientific mtDNA evidence, and also that our attempt at helping sensitize them with Fuzzy Trace Theory-inspired aids did not improve calibration. Individual scientific reasoning abilities and general cognition styles were better predictors of understanding this scientific information. These results suggest a need for further exploration of approaches to help jurors differentiate between high and low quality evidence. Note: The 3rd author was supported by an AP-LS AP Award for her role in this research. Learning Objective: Participants will be able to describe how individual differences in scientific reasoning skills help jurors understand complex scientific evidence. 

   more » « less
2. Computerized Block Games for Automated Cognitive Assessment: Development and Evaluation Study

   https://doi.org/10.2196/40931  

   Cheng, Xiangyi ; Gilmore, Grover C ; Lerner, Alan J ; Lee, Kiju ( January 2023 , JMIR Serious Games)

   Background Cognitive assessment using tangible objects can measure fine motor and hand-eye coordination skills along with other cognitive domains. Administering such tests is often expensive, labor-intensive, and error prone owing to manual recording and potential subjectivity. Automating the administration and scoring processes can address these difficulties while reducing time and cost. e-Cube is a new vision-based, computerized cognitive assessment tool that integrates computational measures of play complexity and item generators to enable automated and adaptive testing. The e-Cube games use a set of cubes, and the system tracks the movements and locations of these cubes as manipulated by the player. Objective The primary objectives of the study were to validate the play complexity measures that form the basis of developing the adaptive assessment system and evaluate the preliminary utility and usability of the e-Cube system as an automated cognitive assessment tool. Methods This study used 6 e-Cube games, namely, Assembly, Shape-Matching, Sequence-Memory, Spatial-Memory, Path-Tracking, and Maze, each targeting different cognitive domains. In total, 2 versions of the games, the fixed version with predetermined sets of items and the adaptive version using the autonomous item generators, were prepared for comparative evaluation. Enrolled participants (N=80; aged 18-60 years) were divided into 2 groups: 48% (38/80) of the participants in the fixed group and 52% (42/80) in the adaptive group. Each was administered the 6 e-Cube games; 3 subtests of the Wechsler Adult Intelligence Scale, Fourth Edition (WAIS-IV; Block Design, Digit Span, and Matrix Reasoning); and the System Usability Scale (SUS). Statistical analyses at the 95% significance level were applied. Results The play complexity values were correlated with the performance indicators (ie, correctness and completion time). The adaptive e-Cube games were correlated with the WAIS-IV subtests (r=0.49, 95% CI 0.21-0.70; P<.001 for Assembly and Block Design; r=0.34, 95% CI 0.03-0.59; P=.03 for Shape-Matching and Matrix Reasoning; r=0.51, 95% CI 0.24-0.72; P<.001 for Spatial-Memory and Digit Span; r=0.45, 95% CI 0.16-0.67; P=.003 for Path-Tracking and Block Design; and r=0.45, 95% CI 0.16-0.67; P=.003 for Path-Tracking and Matrix Reasoning). The fixed version showed weaker correlations with the WAIS-IV subtests. The e-Cube system showed a low false detection rate (6/5990, 0.1%) and was determined to be usable, with an average SUS score of 86.01 (SD 8.75). Conclusions The correlations between the play complexity values and performance indicators supported the validity of the play complexity measures. Correlations between the adaptive e-Cube games and the WAIS-IV subtests demonstrated the potential utility of the e-Cube games for cognitive assessment, but a further validation study is needed to confirm this. The low false detection rate and high SUS scores indicated that e-Cube is technically reliable and usable. 

   more » « less
3. Evaluating the Role of Breathing Guidance on Game-Based Interventions for Relaxation Training

   https://doi.org/10.3389/fdgth.2021.760268  

   Gummidela, Venkata Nitin ; Silva, Dennis R. ; Gutierrez-Osuna, Ricardo ( December 2021 , Frontiers in Digital Health)

   Working in a fast-paced environment can lead to shallow breathing, which can exacerbate stress and anxiety. To address this issue, this study aimed to develop micro-interventions that can promote deep breathing in the presence of stressors. First, we examined two types of breathing guides to help individuals learn deep breathing: providing their breathing rate as a biofeedback signal, and providing a pacing signal to which they can synchronize their breathing. Second, we examined the extent to which these two breathing guides can be integrated into a casual game, to increase enjoyment and skill transfer. We used a 2 × 2 factorial design, with breathing guide (biofeedback vs. pacing) and gaming (game vs. no game) as independent factors. This led to four experimental groups: biofeedback alone, biofeedback integrated into a game, pacing alone, and pacing integrated into a game. In a first experiment, we evaluated the four experimental treatments in a laboratory setting, where 30 healthy participants completed a stressful task before and after performing one of the four treatments (or a control condition) while wearing a chest strap that measured their breathing rate. Two-way ANOVA of breathing rates, with treatment (5 groups) and time (pre-test, post-test) as independent factors shows a significant effect for time [ F (4, 50) = 18.49, p < 0.001, η t i m e 2 = 0 . 27 ] and treatment [ F (4, 50) = 2.54, p = 0.05, η 2 = 0.17], but no interaction effects. Post-hoc t-tests between pre and post-test breathing rates shows statistical significance for the game with biofeedback group [ t (5) = 5.94, p = 0.001, d = 2.68], but not for the other four groups, indicating that only game with biofeedback led to skill transfer at post-test. Further, two-way ANOVA of self-reported enjoyment scores on the four experimental treatments, with breathing guide and game as independent factors, found a main effect for game [ F ( 1 , 20 ) = 24 . 49 , p < 0 . 001 ,   η g a m e 2 = 0 . 55 ], indicating that the game-based interventions were more enjoyable than the non-game interventions. In a second experiment, conducted in an ambulatory setting, 36 healthy participants practiced one of the four experimental treatments as they saw fit over the course of a day. We found that the game-based interventions were practiced more often than the non-game interventions [ t (34) = 1.99, p = 0.027, d = 0.67]. However, we also found that participants in the game-based interventions could only achieve deep breathing 50% of the times, whereas participants in the non-game groups succeeded 85% of the times, which indicated that the former need adequate training time to be effective. Finally, participant feedback indicated that the non-game interventions were better at promoting in-the-moment relaxation, whereas the game-based interventions were more successful at promoting deep breathing during stressful tasks. 

   more » « less
4. Multi-Disciplinary Summer Orientation Sessions for First-Year Students in Engineering, Engineering Technology, Physics, and Computer Science

   https://doi.org/10.18260/p.25760  

   Novoa, Clara ; Ortiz, Araceli ; Talley, Kimberly ( January 2016 , American Society of Engineering Education)

   This work in progress is motivated by a self-study conducted at Texas State University. The study revealed that the average second year science, technology, engineering and math (STEM) student retention rate is 56% vs. 67% for all majors, and that 16% of STEM majors are female while 57% of all undergraduate students are female. Using these statistics, the authors identified the need to offer motivating experiences to freshman in STEM while creating a sense of community among other STEM students. This paper reports on the impact of two interventions designed by the authors and aligned with this need. The interventions are: (1) a one-day multi- disciplinary summer orientation (summer15) to give participants the opportunity to undertake projects that demonstrate the relevance of spatial and computational thinking skills and (2) a subsequent six-week spatial visualization skills training (fall 2015) for students in need to refine these skills. The interventions have spatial skills as a common topic and introduce participants to career applications through laboratory tours and talks. Swail et al.1 mentions that the three elements to address in order to best support students’ persistence and achievement are cognitive, social, and institutional factors. The interventions address all elements to some extent and are part of an NSF IUSE grant (2015-2018) to improve STEM retention. The summer 2015 orientation was attended by 17 freshmen level students in Physics, Engineering, Engineering Technology, and Computer Science. The orientation was in addition to “Bobcat Preview”, a separate mandatory one-week length freshman orientation that includes academic advising and educational and spirit sessions to acclimate students to the campus. The effectiveness of the orientation was assessed through exit surveys administered to participants. Current results are encouraging; 100% of the participants answered that the orientation created a space to learn about science and engineering, facilitated them to make friends and encouraged peer interaction. Eighty percent indicated that the orientation helped them to build confidence in their majors. Exit survey findings were positively linked to a former exit survey from an orientation given to a group of 18 talented and low-income students in 2013. The training on refining spatial visualization skills connects to the summer orientation by its goals. It offers freshman students in need to refine spatial skills a further way to increase motivation to STEM and create community among other students. It is also an effective approach to support students’ persistence and achievement. Bairaktarova et al.2 mention that spatial skills ability is gradually becoming a standard assessment of an individual’s likelihood to succeed as an engineer. Metz et al.3 report that well-developed spatial skills have been shown to lead to success in Engineering and Technology, Computer Science, Chemistry, Computer Aided Design and Mathematics. The effectiveness of the fall 2015 training was assessed through comparison between pre and post tests results and exit surveys administered to participants. All participants improved their pre-training scores and average improvement in students’ scores was 18.334%. 

   more » « less
5. Multi-Disciplinary Orientation Sessions for Fist-Year Students in Engineering, Engineering Technology, Physics, and Computer Science

   Novoa, C. ; Martinez, Ortiz A. ; Talley, K.G. ( January 2016 , ASEE Annual Conference proceedings)

   This work in progress is motivated by a self-study conducted at Texas State University. The study revealed that the average second year science, technology, engineering and math (STEM) student retention rate is 56% vs. 67% for all majors, and that 16% of STEM majors are female while 57% of all undergraduate students are female. Using these statistics, the authors identified the need to offer motivating experiences to freshman in STEM while creating a sense of community among other STEM students. This paper reports on the impact of two interventions designed by the authors and aligned with this need. The interventions are: (1) a one-day multi- disciplinary summer orientation (summer15) to give participants the opportunity to undertake projects that demonstrate the relevance of spatial and computational thinking skills and (2) a subsequent six-week spatial visualization skills training (fall 2015) for students in need to refine these skills. The interventions have spatial skills as a common topic and introduce participants to career applications through laboratory tours and talks. Swail et al.[1] mentions that the three elements to address in order to best support students’ persistence and achievement are cognitive, social, and institutional factors. The interventions address all elements to some extent and are part of an NSF IUSE grant (2015-2018) to improve STEM retention. The summer 2015 orientation was attended by 17 freshmen level students in Physics, Engineering, Engineering Technology, and Computer Science. The orientation was in addition to “Bobcat Preview”, a separate mandatory one-week length freshman orientation that includes academic advising and educational and spirit sessions to acclimate students to the campus. The effectiveness of the orientation was assessed through exit surveys administered to participants. Current results are encouraging; 100% of the participants answered that the orientation created a space to learn about science and engineering, facilitated them to make friends and encouraged peer interaction. Eighty percent indicated that the orientation helped them to build confidence in their majors. Exit survey findings were positively linked to a former exit survey from an orientation given to a group of 18 talented and low-income students in 2013. The training on refining spatial visualization skills connects to the summer orientation by its goals. It offers freshman students in need to refine spatial skills a further way to increase motivation to STEM and create community among other students. It is also an effective approach to support students’ persistence and achievement. Bairaktarova et al.[2] mention that spatial skills ability is gradually becoming a standard assessment of an individual’s likelihood to succeed as an engineer. Metz et al.[3] report that well-developed spatial skills have been shown to lead to success in Engineering and Technology, Computer Science, Chemistry, Computer Aided Design and Mathematics. The effectiveness of the fall 2015 training was assessed through comparison between pre and post tests results and exit surveys administered to participants. All participants improved their pre-training scores and average improvement in students’ scores was 18.334%. 

   more » « less