More Like this

1. Evaluating Beacons, the Role of Variables, Tracing, and Abstract Tracing for Teaching Novices to Understand Program Intent

   https://doi.org/10.1145/3568813.3600140  

   Hassan, Mohammed ; Cunningham, Kathryn ; Zilles, Craig ( August 2023 , Proceedings of the 2023 ACM Conference on International Computing Education Research)

   Background and context. “Explain in Plain English” (EiPE) questions ask students to explain the high-level purpose of code, requiring them to understand the macrostructure of the program’s intent. A lot is known about techniques that experts use to comprehend code, but less is known about how we should teach novices to develop this capability. Objective. Identify techniques that can be taught to students to assist them in developing their ability to comprehend code and contribute to the body of knowledge of how novices develop their code comprehension skills. Method. We developed interventions that could be taught to novices motivated by previous research about how experts comprehend code: prompting students to identify beacons, identify the role of variables, tracing, and abstract tracing. We conducted think-aloud interviews of introductory programming students solving EiPE questions, varying which interventions each student was taught. Some participants were interviewed multiple times throughout the semester to observe any changes in behavior over time. Findings. Identifying beacons and the name of variable roles were rarely helpful, as they did not encourage students to integrate their understanding of that piece in relation to other lines of code. However, prompting students to explain each variable’s purpose helped them focus on useful subsets of the code, which helped manage cognitive load. Tracing was helpful when students incorrectly recognized common programming patterns or made mistakes comprehending syntax (text-surface). Prompting students to pick inputs that potentially contradicted their current understanding of the code was found to be a simple approach to them effectively selecting inputs to trace. Abstract tracing helped students see high-level, functional relationships between variables. In addition, we observed student spontaneously sketching algorithmic visualizations that similarly helped them see relationships between variables. Implications. Because students can get stuck at many points in the process of code comprehension, there seems to be no silver bullet technique that helps in every circumstance. Instead, effective instruction for code comprehension will likely involve teaching a collection of techniques. In addition to these techniques, meta-knowledge about when to apply each technique will need to be learned, but that is left for future research. At present, we recommend teaching a bottom-up, concrete-to-abstract approach. 

   more » « less
2. When Am I (N)ever Going to Use This? How Engineers Use Algebra

   Istas, Brooke ; Walkington, Candace ; Leyva, Elizabeth ( July 2021 , 2021 ASEE Virtual Annual Conference)

   Mathematics is an important tool in engineering practice, as mathematical rules govern many designed systems (e.g., Nathan et al., 2013; Nathan et al., 2017). Investigations of structural engineers suggest that mathematical modelling is ubiquitous in their work, but the nature of the tasks they confront is not well-represented in the K-12 classroom (e.g., Gainsburg, 2006). This follows a larger literature base suggesting that school mathematics is often inauthentic and does represent how mathematics is used in practice. At the same time, algebra is a persistent gatekeeper to careers in engineering (e.g., Harackiewicz et al., 2012; Olson & Riordan, 2012). In the present study, we interviewed 12 engineers, asking them a series of questions about how they use specific kinds of algebraic function (e.g., linear, exponential, quadratic) in their work. The purpose of these interviews was to use the responses to create mathematical scenarios for College Algebra activities that would be personalized to community college students’ career interests. This curriculum would represent how algebra is used in practice by STEM professionals. However, our results were not what we expected. In this paper, we discuss three major themes that arose from qualitative analyses of the interviews. First, we found that engineers resoundingly endorsed the importance of College Algebra concepts for their day-to-day work, and uniformly stated that math was vital to engineering. However, the second theme was that the engineers struggled to describe how they used functions more complex than linear (i.e., y=mx+b) in their work. Students typically learn about linear functions prior to College Algebra, and in College Algebra explore more complex functions like polynomial, logarithmic, and exponential. Third, we found that engineers rarely use the explicit algebraic form of an algebraic function (e.g., y=3x+5), and instead rely on tables, graphs, informal arithmetic, and computerized computation systems where the equation is invisible. This was surprising, given that the bulk of the College Algebra course involves learning how to use and manipulate these formal expressions, learning skills like factoring, simplifying, solving, and interpreting parameters. We also found that these trends for engineers followed trends we saw in our larger sample where we interviewed professionals from across STEM fields. This study calls into question the gatekeeping role of formal algebraic courses like College Algebra for STEM careers. If engineers don’t actually use 75% of the content in these courses, why are they required? One reason might be that the courses are simply outdated, or arguments might be made that learning mathematics builds more general modelling and problem-solving skills. However, research from educational psychology on the difficulty of transfer would strongly refute this point – people tend to learn things that are very specific. Another reason to consider is that formal mathematics courses like advanced algebra have emerged as a very convenient mechanism to filter people by race, gender, and socioeconomic background, and to promote the maintenance of the “status quo” inequality in STEM fields. This is a critical issue to investigate for the future of the field of engineering as a whole. 

   more » « less
3. Predictors of Jurors’ Understanding of Evidence Strength

   McCowan, K. ( March 2020 , Annual Conference of the American Psychology-Law Society.)

   Abstract Expert testimony varies in scientific quality and jurors have a difficult time evaluating evidence quality (McAuliff et al., 2009). In the current study, we apply Fuzzy Trace Theory principles, examining whether visual and gist aids help jurors calibrate to the strength of scientific evidence. Additionally we were interested in the role of jurors’ individual differences in scientific reasoning skills in their understanding of case evidence. Contrary to our preregistered hypotheses, there was no effect of evidence condition or gist aid on evidence understanding. However, individual differences between jurors’ numeracy skills predicted evidence understanding. Summary Poor-quality expert evidence is sometimes admitted into court (Smithburn, 2004). Jurors’ calibration to evidence strength varies widely and is not robustly understood. For instance, previous research has established jurors lack understanding of the role of control groups, confounds, and sample sizes in scientific research (McAuliff, Kovera, & Nunez, 2009; Mill, Gray, & Mandel, 1994). Still others have found that jurors can distinguish weak from strong evidence when the evidence is presented alone, yet not when simultaneously presented with case details (Smith, Bull, & Holliday, 2011). This research highlights the need to present evidence to jurors in a way they can understand. Fuzzy Trace Theory purports that people encode information in exact, verbatim representations and through “gist” representations, which represent summary of meaning (Reyna & Brainerd, 1995). It is possible that the presenting complex scientific evidence to people with verbatim content or appealing to the gist, or bottom-line meaning of the information may influence juror understanding of that evidence. Application of Fuzzy Trace Theory in the medical field has shown that gist representations are beneficial for helping laypeople better understand risk and benefits of medical treatment (Brust-Renck, Reyna, Wilhelms, & Lazar, 2016). Yet, little research has applied Fuzzy Trace Theory to information comprehension and application within the context of a jury (c.f. Reyna et. al., 2015). Additionally, it is likely that jurors’ individual characteristics, such as scientific reasoning abilities and cognitive tendencies, influence their ability to understand and apply complex scientific information (Coutinho, 2006). Methods The purpose of this study was to examine how jurors calibrate to the strength of scientific information, and whether individual difference variables and gist aids inspired by Fuzzy Trace Theory help jurors better understand complicated science of differing quality. We used a 2 (quality of scientific evidence: high vs. low) x 2 (decision aid to improve calibration - gist information vs. no gist information), between-subjects design. All hypotheses were preregistered on the Open Science Framework. Jury-eligible community participants (430 jurors across 90 juries; Mage = 37.58, SD = 16.17, 58% female, 56.93% White). Each jury was randomly assigned to one of the four possible conditions. Participants were asked to individually fill out measures related to their scientific reasoning skills prior to watching a mock jury trial. The trial was about an armed bank robbery and consisted of various pieces of testimony and evidence (e.g. an eyewitness testimony, police lineup identification, and a sweatshirt found with the stolen bank money). The key piece of evidence was mitochondrial DNA (mtDNA) evidence collected from hair on a sweatshirt (materials from Hans et al., 2011). Two experts presented opposing opinions about the scientific evidence related to the mtDNA match estimate for the defendant’s identification. The quality and content of this mtDNA evidence differed based on the two conditions. The high quality evidence condition used a larger database than the low quality evidence to compare to the mtDNA sample and could exclude a larger percentage of people. In the decision aid condition, experts in the gist information group presented gist aid inspired visuals and examples to help explain the proportion of people that could not be excluded as a match. Those in the no gist information group were not given any aid to help them understand the mtDNA evidence presented. After viewing the trial, participants filled out a questionnaire on how well they understood the mtDNA evidence and their overall judgments of the case (e.g. verdict, witness credibility, scientific evidence strength). They filled this questionnaire out again after a 45-minute deliberation. Measures We measured Attitudes Toward Science (ATS) with indices of scientific promise and scientific reservations (Hans et al., 2011; originally developed by National Science Board, 2004; 2006). We used Drummond and Fischhoff’s (2015) Scientific Reasoning Scale (SRS) to measure scientific reasoning skills. Weller et al.’s (2012) Numeracy Scale (WNS) measured proficiency in reasoning with quantitative information. The NFC-Short Form (Cacioppo et al., 1984) measured need for cognition. We developed a 20-item multiple-choice comprehension test for the mtDNA scientific information in the cases (modeled on Hans et al., 2011, and McAuliff et al., 2009). Participants were shown 20 statements related to DNA evidence and asked whether these statements were True or False. The test was then scored out of 20 points. Results For this project, we measured calibration to the scientific evidence in a few different ways. We are building a full model with these various operationalizations to be presented at APLS, but focus only on one of the calibration DVs (i.e., objective understanding of the mtDNA evidence) in the current proposal. We conducted a general linear model with total score on the mtDNA understanding measure as the DV and quality of scientific evidence condition, decision aid condition, and the four individual difference measures (i.e., NFC, ATS, WNS, and SRS) as predictors. Contrary to our main hypotheses, neither evidence quality nor decision aid condition affected juror understanding. However, the individual difference variables did: we found significant main effects for Scientific Reasoning Skills, F(1, 427) = 16.03, p <.001, np2 = .04, Weller Numeracy Scale, F(1, 427) = 15.19, p <.001, np2 = .03, and Need for Cognition, F(1, 427) = 16.80, p <.001, np2 = .04, such that those who scored higher on these measures displayed better understanding of the scientific evidence. In addition there was a significant interaction of evidence quality condition and scores on the Weller’s Numeracy Scale, F(1, 427) = 4.10, p = .04, np2 = .01. Further results will be discussed. Discussion These data suggest jurors are not sensitive to differences in the quality of scientific mtDNA evidence, and also that our attempt at helping sensitize them with Fuzzy Trace Theory-inspired aids did not improve calibration. Individual scientific reasoning abilities and general cognition styles were better predictors of understanding this scientific information. These results suggest a need for further exploration of approaches to help jurors differentiate between high and low quality evidence. Note: The 3rd author was supported by an AP-LS AP Award for her role in this research. Learning Objective: Participants will be able to describe how individual differences in scientific reasoning skills help jurors understand complex scientific evidence. 

   more » « less
4. Student Understanding of the Sign of Negative Definite Integrals in Mathematics and Physics

   https://doi.org/10.1007/s40753-022-00202-y  

   Bajracharya, Rabindra R. ; Sealey, Vicki L. ; Thompson, John R. ( January 2023 , International Journal of Research in Undergraduate Mathematics Education)

   In a study of student understanding of negative definite integrals at two institutions, we administered a written survey and follow-up clinical interviews at one institution and found that “backward integrals”, where the integral was taken from right to left on thex-axis, were the most difficult for students to interpret. We then conducted additional interviews focused on backward integrals at a second institution. Our analysis uses the concept image framework and a recent categorical framework for mathematical sense making. We found that students were most successful using the Fundamental Theorem of Calculus to determine the sign of an integral when a symbolic expression was provided. When considering a definite integral in a graphical context, students often had difficulty if they viewed the integral as a spatial area, stating that area must always be positive. Some students were able to conceptualize$$\Delta x$$$\Delta x$or$$dx$$$\mathrm{dx}$as a difference or change and thus a signed quantity; when these students were able to view the area as a sum of smaller pieces, they were more successful in justifying a negative backward integral. Students who used amounts or similar images often had difficulty making sense of the negative sign. Even more progress was made when students either invoked or were asked explicitly about a specific physical context to be represented by the backward integral, other than spatial area. The context provided a meaning to the difference represented by$$\Delta x$$$\Delta x$or$$dx$$$\mathrm{dx}$and thus to the sign of that difference and the definite integral.

    

   more » « less
5. . Expert and novice conceptions of the biotic impacts of climate change

   Holt, EA ; Sexton, J ; Hinerman, K ; Romano, A ; Heim, A. ( July 2019 , Society for the Advancement of Biology Education Researchers Conference)

   Research Problem: Climate change is one of the most important environmental, social, and economic issues of our time. The documented impacts of climate change are extensive. Climate change education can help students link this global issue to students’ everyday lives, foster a climate-literate public, and serve as motivation for action. Yet prior to instructional interventions, the first step in promoting conceptual change is to describe expert and novice conceptions or mental models of the topic (Treagust and Duit 2009). Published studies about students’ climate change knowledge primarily stem from the earth and atmospheric sciences, and focus on students’ knowledge of the mechanisms causing global warming and of the abiotic systems important to climate change. Limited research has documented undergraduate students’ knowledge about the biotic impacts of climate change. Our goal was to describe student/novice and instructor/expert conceptual knowledge of the biotic impacts of climate change. Research Design: We conducted interviews with 30 undergraduates and 10 instructors who are students or teaching in Introductory Biology or Ecology classes. Our semi-structured interview protocol probed participants’ conceptions of the mechanisms, outcomes and levels of impact that climate change has on the biological world. Participants were taken from varying institutions across the US (Baccalaureate, Master’s, and Doctoral). Analyses: Following transcription of all interviews, we used thematic coding analysis to describe novice and expert conceptions of the biotic impacts to climate change. We also compared across interview populations to describe how novice and expert conceptions compare. Contribution: Our findings contribute understanding of biology student and expert knowledge of the biotic impacts of climate change and contribute more broadly to the field of climate science where research on understanding of the biotic impacts of climate change is minimal. Our work will represent a novel perspective because most climate education research at the university-level has focused on earth and atmospheric science students. Further, this work is the first step in a larger project that aims to develop valid and reliable concept inventory related to biotic impacts of climate change – an instrument sorely needed to properly address improvements to climate change education. 

   more » « less