More Like this

1. Effectiveness of Undergraduate-Generated Animations: Increasing Comprehension and Engagement for Neuroscience Majors and Non-Majors

   https://doi.org/10.1177/00986283211023061  

   Reed, Catherine L. ; Hagen, Emilia ; Bukach, Cindy M. ; Couperus, Jane W. ( July 2021 , Teaching of Psychology)

   Background: Animations of scientific concepts may improve comprehension by explaining and visualizing the steps of complex processes, but unless they engage student interest in meaningful ways, their effectiveness as teaching tools is limited. We achieve this through a novel approach to animation design that includes the target audience (undergraduates) so that the resultant animations align with their learner characteristics. Objective: This case study investigated whether undergraduate-generated animations were more effective educational tools than informationally equivalent text-and-illustration presentations and whether learners’ background influenced the relative benefits of animations. Method: Incorporating feedback from faculty and undergraduates, we created animations and text-plus-illustration content to explain how neural signals are generated and measured by scalp electrodes. Neuroscience majors and non-majors were presented with either animations or static presentations followed by comprehension and engagement assessments. Results: Both groups showed comprehension and engagement benefits for animations. Although majors showed better overall comprehension, animations improved comprehension for non-majors over static presentations. Conclusion: When educational content is directed for a target audience, animations can be more effective teaching tools for a broader student audience. Teaching Implications: The relevance of online tools for remote instruction makes animations, developed for and by undergraduates, important tools for effectively introducing difficult content. 

   more » « less
2. Generalizing Predictive Models of Reading Ability in Adaptive Mathematics Software

   Almoubayyed, Husni ; Fancsali, Stephen ; Ritter, Steve ( January 2023 , Proceedings of the 16th International Conference on Educational Data Mining)

   Feng, Mingyu ; Käser, Tanja ; Talukdar, Partha (Ed.)

   Recent research seeks to develop more comprehensive learner models for adaptive learning software. For example, models of reading comprehension built using data from students’ use of adaptive instructional software for mathematics have recently been developed. These models aim to deliver experiences that consider factors related to learning beyond performance in the target domain for instruction. We investigate the extent to which generalization is possible for a recently developed predictive model that seeks to infer students’ reading comprehension ability (as measured by end-of-year standardized test scores) using an introductory learning experience in Carnegie Learning’s MATHia intelligent tutoring system for mathematics. Building on a model learned on data from middle school students in a single school district in a mid-western U.S. state, using that state’s end-of-year English Language Arts (ELA) standardized test score as an outcome, we consider data from a school district in a south-eastern U.S. state as well as that state’s end-of-year ELA standardized test outcome. Generalization is explored by considering prediction performance when training and testing models on data from each of the individual school districts (and for their respective state’s test outcomes) as well as pooling data from both districts together. We conclude with discussion of investigations of some algorithmic fairness characteristics of the learned models. The results suggest that a model trained on data from the smaller of the two school districts considered may achieve greater fairness in its predictions over models trained on data from the other district or both districts, despite broad, overall similarities in some demographic characteristics of the two school districts. This raises interesting questions for future research on generalizing these kinds of models as well as on ensuring algorithmic fairness of resulting models for use in real-world adaptive systems for learning. 

   more » « less
3. Cognitive agency and computer-based tasks

   Riling, M. ( January 2018 , Proceedings of the 40th annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education)

   As computer-focused policies and trends become more popular in schools, more students access math curriculum online. While computer-based programs may be responsive to some student input, their algorithmic basis can make it more difficult for them to be prepared for divergent student thinking, especially in comparison to a teacher. Consider programs that assess student work by judging how well it matches pre-set answers. Unless designed and enacted in classrooms with care, computer-based curriculum materials might encourage students to think about mathematics in pre-determined ways. How do students approach the process of mathematics while using online materials, especially in terms of engaging in original thought? Drawing on Pickering’s (1995) dance of agency and Sinclair’s (2001) conception of students as path-finders or track-takers, I define two modes of mathematical behavior: trail-taking and bushwhacking. While trail-taking, students follow an established approach, often relying on Pickering’s (1995) disciplinary agency, wherein the mathematics “leads [them] through a series of manipulations” (p. 115). The series of manipulations can be seen as a trail that a student may choose to follow. Bushwhacking, on the other hand, refers to actions a student takes of their own invention. It is possible that, unknown to the student, these actions have been taken before by others. In bushwhacking, the student possesses agency, which Pickering (1995) describes as active (rather than passive) and as hallmarked by “choice and discretion” (p. 117). In this study, students worked in several dynamic geometric environments (DGEs) during a geometry lesson about the midline theorem. The lesson was originally recorded as part of a larger study designing mathematically captivating lessons. Students accessed both problems and online addresses for corresponding DGEs via a printed packet. Students interacted with the DGEs on individual laptops, but were seated in groups of three or four. Passages of group conversations in which students transitioned between trail-taking and bushwhacking were selected for closer analysis, which involved identifying evidence of each mode and highlighting the curricular or social forces that may have contributed to shifts between modes. Of particular interest were episodes in which students asked one another to share results, which led to students reconsidering previously set approaches, and episodes in which students interacted with DGEs containing a relatively high proportion of drag-able components, which corresponded to some students working in bushwhacking mode, spontaneously suggesting and revising approaches for manipulating the DGE (e.g., “unless you make this parallel to the bottom, but I don’t think you... yes you can.”). Both types of episodes were found in multiple groups’ conversations. Further analysis of student interactions with tasks, especially with varying levels of student control and sharing, could serve to inform future computer-based task design aimed to encourage students to productively engage in bushwhacking while problem-solving. 

   more » « less
4. What We Can Learn from Jury Note Taking: A Content Analysis

   Denne, E. ( March 2020 , Annual Conference of the American Psychology-Law Society.)

   Abstract: Jury notetaking can be controversial despite evidence suggesting benefits for recall and understanding. Research on note taking has historically focused on the deliberation process. Yet, little research explores the notes themselves. We developed a 10-item coding guide to explore what jurors take notes on (e.g., simple vs. complex evidence) and how they take notes (e.g., gist vs. specific representation). In general, jurors made gist representations of simple and complex information in their notes. This finding is consistent with Fuzzy Trace Theory (Reyna & Brainerd, 1995) and suggests notes may serve as a general memory aid, rather than verbatim representation. Summary: The practice of jury notetaking in the courtroom is often contested. Some states allow it (e.g., Nebraska: State v. Kipf, 1990), while others forbid it (e.g., Louisiana: La. Code of Crim. Proc., Art. 793). Some argue notes may serve as a memory aid, increase juror confidence during deliberation, and help jurors engage in the trial (Hannaford & Munsterman, 2001; Heuer & Penrod, 1988, 1994). Others argue notetaking may distract jurors from listening to evidence, that juror notes may be given undue weight, and that those who took notes may dictate the deliberation process (Dann, Hans, & Kaye, 2005). While research has evaluated the efficacy of juror notes on evidence comprehension, little work has explored the specific content of juror notes. In a similar project on which we build, Dann, Hans, and Kaye (2005) found jurors took on average 270 words of notes each with 85% including references to jury instructions in their notes. In the present study we use a content analysis approach to examine how jurors take notes about simple and complex evidence. We were particularly interested in how jurors captured gist and specific (verbatim) information in their notes as they have different implications for information recall during deliberation. According to Fuzzy Trace Theory (Reyna & Brainerd, 1995), people extract “gist” or qualitative meaning from information, and also exact, verbatim representations. Although both are important for helping people make well-informed judgments, gist-based understandings are purported to be even more important than verbatim understanding (Reyna, 2008; Reyna & Brainer, 2007). As such, it could be useful to examine how laypeople represent information in their notes during deliberation of evidence. Methods Prior to watching a 45-minute mock bank robbery trial, jurors were given a pen and notepad and instructed they were permitted to take notes. The evidence included testimony from the defendant, witnesses, and expert witnesses from prosecution and defense. Expert testimony described complex mitochondrial DNA (mtDNA) evidence. The present analysis consists of pilot data representing 2,733 lines of notes from 52 randomly-selected jurors across 41 mock juries. Our final sample for presentation at AP-LS will consist of all 391 juror notes in our dataset. Based on previous research exploring jury note taking as well as our specific interest in gist vs. specific encoding of information, we developed a coding guide to quantify juror note-taking behaviors. Four researchers independently coded a subset of notes. Coders achieved acceptable interrater reliability [(Cronbach’s Alpha = .80-.92) on all variables across 20% of cases]. Prior to AP-LS, we will link juror notes with how they discuss scientific and non-scientific evidence during jury deliberation. Coding Note length. Before coding for content, coders counted lines of text. Each notepad line with at minimum one complete word was coded as a line of text. Gist information vs. Specific information. Any line referencing evidence was coded as gist or specific. We coded gist information as information that did not contain any specific details but summarized the meaning of the evidence (e.g., “bad, not many people excluded”). Specific information was coded as such if it contained a verbatim descriptive (e.g.,“<1 of people could be excluded”). We further coded whether this information was related to non-scientific evidence or related to the scientific DNA evidence. Mentions of DNA Evidence vs. Other Evidence. We were specifically interested in whether jurors mentioned the DNA evidence and how they captured complex evidence. When DNA evidence was mention we coded the content of the DNA reference. Mentions of the characteristics of mtDNA vs nDNA, the DNA match process or who could be excluded, heteroplasmy, references to database size, and other references were coded. Reliability. When referencing DNA evidence, we were interested in whether jurors mentioned the evidence reliability. Any specific mention of reliability of DNA evidence was noted (e.g., “MT DNA is not as powerful, more prone to error”). Expert Qualification. Finally, we were interested in whether jurors noted an expert’s qualifications. All references were coded (e.g., “Forensic analyst”). Results On average, jurors took 53 lines of notes (range: 3-137 lines). Most (83%) mentioned jury instructions before moving on to case specific information. The majority of references to evidence were gist references (54%) focusing on non-scientific evidence and scientific expert testimony equally (50%). When jurors encoded information using specific references (46%), they referenced non-scientific evidence and expert testimony equally as well (50%). Thirty-three percent of lines were devoted to expert testimony with every juror including at least one line. References to the DNA evidence were usually focused on who could be excluded from the FBIs database (43%), followed by references to differences between mtDNA vs nDNA (30%), and mentions of the size of the database (11%). Less frequently, references to DNA evidence focused on heteroplasmy (5%). Of those references that did not fit into a coding category (11%), most focused on the DNA extraction process, general information about DNA, and the uniqueness of DNA. We further coded references to DNA reliability (15%) as well as references to specific statistical information (14%). Finally, 40% of jurors made reference to an expert’s qualifications. Conclusion Jury note content analysis can reveal important information about how jurors capture trial information (e.g., gist vs verbatim), what evidence they consider important, and what they consider relevant and irrelevant. In our case, it appeared jurors largely created gist representations of information that focused equally on non-scientific evidence and scientific expert testimony. This finding suggests note taking may serve not only to represent information verbatim, but also and perhaps mostly as a general memory aid summarizing the meaning of evidence. Further, jurors’ references to evidence tended to be equally focused on the non-scientific evidence and the scientifically complex DNA evidence. This observation suggests jurors may attend just as much to non-scientific evidence as they to do complex scientific evidence in cases involving complicated evidence – an observation that might inform future work on understanding how jurors interpret evidence in cases with complex information. Learning objective: Participants will be able to describe emerging evidence about how jurors take notes during trial. 

   more » « less
5. Where’s My Whiteboard? The Challenge of Moving Active-learning Mathematics Classes Online

   Nelson, Jill K. ; Rosenberg, Jessica ; Fernandez, Kathryn ; Shank, Julie ( July 2021 , 2021 ASEE Virtual Annual Conference)

   null (Ed.)

   This research paper studies the challenges that mathematics faculty and graduate teaching assistants (GTAs) faced when moving active and collaborative calculus courses from in-person to virtual instruction. As part of a larger pedagogical change project (described below), the math department at a public Research-1 university began transitioning pre-calculus and calculus courses to an active and collaborative learning (ACL) format in Fall 2019. The change began with the introduction of collaborative worksheets in recitations which were led by GTAs and supported by undergraduate learning assistants (LAs). Students recitation periods collaboratively solving the worksheet problems on whiteboards. When COVID-19 forced the rapid transition to online teaching, these ACL efforts faced an array of challenges. Faculty and GTA reflections on the changes to teaching and learning provide insight into how instructional staff can be supported in implementing ACL across various modes of instruction. The calculus teaching change efforts discussed in this paper are part of an NSF-supported project that aims to make ACL the default method of instruction in highly enrolled gateway STEM courses across the institution. The theoretical framework for the project builds on existing work on grassroots change in higher education (Kezar and Lester, 2011) to study the effect of communities of practice on changing teaching culture. The project uses course-based communities of practice (Wenger, 1999) that include instructors, GTAs, and LAs working together to design and enact teaching change in the targeted courses alongside ongoing professional development for GTAs and LAs. Six faculty and five GTAs involved in the teaching change effort in mathematics were interviewed after the Spring 2020 semester ended. Interview questions focused on faculty and GTA experiences implementing active learning after the rapid transition to online teaching. A grounded coding scheme was used to identify common themes in the challenges faced by instructors and GTAs as they moved online and in the impacts of technology, LA support, and the department community of practice on the move to online teaching. Technology, including both access and capabilities, emerged as a common barrier to student engagement. A particular barrier was students’ reluctance to share video or participate orally in sessions that were being recorded, making group work more difficult than it had been in a physical classroom. In addition, most students lacked access to a tablet for freehand writing, presenting a significant hurdle for sharing mathematical notation when physical whiteboards were no longer an option. These challenges point to the importance of incorporating flexibility in active learning implementation and in the professional development that supports teaching changes toward active learning, since what is conceived for a collaborative physical classroom may be implemented in a much different environment. The full paper will present a detailed analysis of the data to better understand how faculty and GTA experiences in the transition to online delivery can inform planning and professional development as the larger institutional change effort moves forward both in mathematics and in other STEM fields. 

   more » « less