More Like this

1. Learning Analytics for Assessing Hands-on Laboratory Skills in Science Classrooms Using Bayesian Network Analysis

   https://doi.org/10.1007/s11165-022-10061-x  

   Jiang, Shiyan ; Huang, Xudong ; Sung, Shannon H. ; Xie, Charles ( July 2022 , Research in Science Education)

   Learning analytics, referring to the measurement, collection, analysis, and reporting of data about learners and their contexts in order to optimize learning and the environments in which it occurs, is proving to be a powerful approach for understanding and improving science learning. However, few studies focused on leveraging learning analytics to assess hands-on laboratory skills in K-12 science classrooms. This study demonstrated the feasibility of gauging laboratory skills based on students’ process data logged by a mobile augmented reality (AR) application for conducting science experiments. Students can use the mobile AR technology to investigate a variety of science phenomena thatmore »involve concepts central to physics understanding. Seventy-two students from a suburban middle school in the Northeastern United States participated in this study. They conducted experiments in pairs. Mining process data using Bayesian networks showed that most students who participated in this study demonstrated some degree of proficiency in laboratory skills. Also, findings indicated a positive correlation between laboratory skills and conceptual learning. The results suggested that learning analytics provides a possible solution to measure hands-on laboratory learning in real-time and at scale.« less
2. Virtual Reality Laboratory Experiences for Electricity and Magnetism Courses

   Ilie, R. ( July 2021 , 2021 ASEE Virtual Annual Conference Content Access)

   A solid understanding of electromagnetic (E&M) theory is key to the education of electrical engineering students. However, these concepts are notoriously challenging for students to learn, due to the difficulty in grasping abstract concepts such as the electric force as an invisible force that is acting at a distance, or how electromagnetic radiation is permeating and propagating in space. Building physical intuition to manipulate these abstractions requires means to visualize them in a three-dimensional space. This project involves the development of 3D visualizations of abstract E&M concepts in Virtual Reality (VR), in an immersive, exploratory, and engaging environment. VR providesmore »the means of exploration, to construct visuals and manipulable objects to represent knowledge. This leads to a constructivist way of learning, in the sense that students are allowed to build their own knowledge from meaningful experiences. In addition, the VR labs replace the cost of hands-on labs, by recreating the experiments and experiences on Virtual Reality platforms. The development of the VR labs for E&M courses involves four distinct phases: (I) Lab Design, (II) Experience Design, (III) Software Development, and (IV) User Testing. During phase I, the learning goals and possible outcomes are clearly defined, to provide context for the VR laboratory experience, and to identify possible technical constraints pertaining to the specific laboratory exercise. During stage II, the environment (the world) the player (user) will experience is designed, along with the foundational elements, such as ways of navigation, key actions, and immersion elements. During stage III, the software is generated as part of the course projects for the Virtual Reality course taught in the Computer Science Department at the same university, or as part of independent research projects involving engineering students. This reflects the strong educational impact of this project, as it allows students to contribute to the educational experiences of their peers. During phase IV, the VR experiences are played by different types of audiences that fit the player type. The team collects feedback and if needed, implements changes. The pilot VR Lab, introduced as an additional instructional tool for the E&M course during the Fall 2019, engaged over 100 students in the program, where in addition to the regular lectures, students attended one hour per week in the E&M VR lab. Student competencies around conceptual understanding of electromagnetism topics are measured via formative and summative assessments. To evaluate the effectiveness of VR learning, each lab is followed by a 10-minute multiple-choice test, designed to measure conceptual understanding of the various topics, rather than the ability to simply manipulate equations. This paper discusses the implementation and the pedagogy of the Virtual Reality laboratory experiences to visualize concepts in E&M, with examples for specific labs, as well as challenges, and student feedback with the new approach. We will also discuss the integration of the 3D visualizations into lab exercises, and the design of the student assessment tools used to assess the knowledge gain when the VR technology is employed.« less
3. How Does Augmented Observation Facilitate Multimodal Representational Thinking? Applying Deep Learning to Decode Complex Student Construct

   https://doi.org/10.1007/s10956-020-09856-2  

   Sung, Shannon H. ; Li, Chenglu ; Chen, Guanhua ; Huang, Xudong ; Xie, Charles ; Massicotte, Joyce ; Shen, Ji ( October 2020 , Journal of Science Education and Technology)

   In this paper, we demonstrate how machine learning could be used to quickly assess a student’s multimodal representational thinking. Multimodal representational thinking is the complex construct that encodes how students form conceptual, perceptual, graphical, or mathematical symbols in their mind. The augmented reality (AR) technology is adopted to diversify student’s representations. The AR technology utilized a low-cost, high-resolution thermal camera attached to a smartphone which allows students to explore the unseen world of thermodynamics. Ninth-grade students (N= 314) engaged in a prediction–observation–explanation (POE) inquiry cycle scaffolded to leverage the augmented observation provided by the aforementioned device. The objective is tomore »investigate how machine learning could expedite the automated assessment of multimodal representational thinking of heat energy. Two automated text classification methods were adopted to decode different mental representations students used to explain their haptic perception, thermal imaging, and graph data collected in the lab. Since current automated assessment in science education rarely considers multilabel classification, we resorted to the help of the state-of-the-art deep learning technique—bidirectional encoder representations from transformers (BERT). The BERT model classified open-ended responses into appropriate categories with higher precision than the traditional machine learning method. The satisfactory accuracy of deep learning in assigning multiple labels is revolutionary in processing qualitative data. The complex student construct, such as multimodal representational thinking, is rarely mutually exclusive. The study avails a convenient technique to analyze qualitative data that does not satisfy the mutual-exclusiveness assumption. Implications and future studies are discussed.« less
4. Developing a Model of Disciplinary Literacy Instruction for K-12 Engineering Education: Comparing the Literacy Practices of Electrical and Mechanical Engineers (Fundamental)

   Green, T. ; Wilson-Lopez, A. ; Minichiello, A. ( July 2019 , ASEE Annual Conference proceedings)

   Despite efforts to diversify the engineering workforce, the field remains dominated by White, male engineers. Research shows that underrepresented groups, including women and minorities, are less likely to identify and engage with scientific texts and literacy practices. Often, children of minority groups and/or working-class families do not receive the same kinds of exposure to science, technology, engineering, and mathematics (STEM) knowledge and practices as those from majority groups. Consequently, these children are less likely to engage in school subjects that provide pathways to engineering careers. Therefore, to mitigate the lack of diversity in engineering, new approaches able to broadly supportmore »engineering literacy are needed. One promising approach is disciplinary literacy instruction (DLI). DLI is a method for teaching students how advanced practitioners in a given field generate, interpret, and evaluate discipline-specific texts. DLI helps teachers provide access to to high quality, discipline-specific content to all students, regardless of race, ethnicity, gender, or socio-economic status, Therefore, DLI has potential to reduce literacy-based barriers that discourage underrepresented students from pursuing engineering careers. While models of DLI have been developed and implemented in history, science, and mathematics, little is known about DLI in engineering. The purpose of this research is to identify the authentic texts, practices, and evaluative frameworks employed by professional engineers to inform a model of DLI in engineering. While critiques of this approach may suggest that a DLI model will reflect the literacy practices of majority engineering groups, (i.e., White male engineers), we argue that a DLI model can directly empower diverse K-16 students to become engineers by instructing them in the normed knowledge and practices of engineering. This paper presents a comparative case study conducted to investigate the literacy practices of electrical and mechanical engineers. We scaffolded our research using situated learning theory and rhetorical genre studies and considered the engineering profession as a community of practice. We generated multiple types of data with four participants (i.e., two electrical and two mechanical engineers). Specifically, we generated qualitative data, including written field notes of engineer observations, interview transcripts, think-aloud protocols, and engineer logs of literacy practices. We used constant comparative analysis (CCA) coding techniques to examine how electrical and mechanical engineers read, wrote, and evaluated texts to identify the frameworks that guide their literacy practices. We then conducted within-group and cross-group constant comparative analyses (CCA) to compare and contrast the literacy practices specific to each sub-discipline Findings suggest that there are two types of engineering literacy practices: those that resonate across both mechanical and electrical engineering disciplines and those that are specific to each discipline. For example, both electrical and mechanical engineers used test procedures to review and assess steps taken to evaluate electrical or mechanical system performance. In contrast, engineers from the two sub-disciplines used different forms of representation when depicting components and arrangements of engineering systems. While practices that are common across sub-disciplines will inform a model of DLI in engineering for K-12 settings, discipline-specific practices can be used to develop and/or improve undergraduate engineering curricula.« less
5. Predictors of Jurors’ Understanding of Evidence Strength

   McCowan, K. ( March 2020 , Annual Conference of the American Psychology-Law Society.)

   Abstract Expert testimony varies in scientific quality and jurors have a difficult time evaluating evidence quality (McAuliff et al., 2009). In the current study, we apply Fuzzy Trace Theory principles, examining whether visual and gist aids help jurors calibrate to the strength of scientific evidence. Additionally we were interested in the role of jurors’ individual differences in scientific reasoning skills in their understanding of case evidence. Contrary to our preregistered hypotheses, there was no effect of evidence condition or gist aid on evidence understanding. However, individual differences between jurors’ numeracy skills predicted evidence understanding. Summary Poor-quality expert evidence is sometimesmore »admitted into court (Smithburn, 2004). Jurors’ calibration to evidence strength varies widely and is not robustly understood. For instance, previous research has established jurors lack understanding of the role of control groups, confounds, and sample sizes in scientific research (McAuliff, Kovera, & Nunez, 2009; Mill, Gray, & Mandel, 1994). Still others have found that jurors can distinguish weak from strong evidence when the evidence is presented alone, yet not when simultaneously presented with case details (Smith, Bull, & Holliday, 2011). This research highlights the need to present evidence to jurors in a way they can understand. Fuzzy Trace Theory purports that people encode information in exact, verbatim representations and through “gist” representations, which represent summary of meaning (Reyna & Brainerd, 1995). It is possible that the presenting complex scientific evidence to people with verbatim content or appealing to the gist, or bottom-line meaning of the information may influence juror understanding of that evidence. Application of Fuzzy Trace Theory in the medical field has shown that gist representations are beneficial for helping laypeople better understand risk and benefits of medical treatment (Brust-Renck, Reyna, Wilhelms, & Lazar, 2016). Yet, little research has applied Fuzzy Trace Theory to information comprehension and application within the context of a jury (c.f. Reyna et. al., 2015). Additionally, it is likely that jurors’ individual characteristics, such as scientific reasoning abilities and cognitive tendencies, influence their ability to understand and apply complex scientific information (Coutinho, 2006). Methods The purpose of this study was to examine how jurors calibrate to the strength of scientific information, and whether individual difference variables and gist aids inspired by Fuzzy Trace Theory help jurors better understand complicated science of differing quality. We used a 2 (quality of scientific evidence: high vs. low) x 2 (decision aid to improve calibration - gist information vs. no gist information), between-subjects design. All hypotheses were preregistered on the Open Science Framework. Jury-eligible community participants (430 jurors across 90 juries; Mage = 37.58, SD = 16.17, 58% female, 56.93% White). Each jury was randomly assigned to one of the four possible conditions. Participants were asked to individually fill out measures related to their scientific reasoning skills prior to watching a mock jury trial. The trial was about an armed bank robbery and consisted of various pieces of testimony and evidence (e.g. an eyewitness testimony, police lineup identification, and a sweatshirt found with the stolen bank money). The key piece of evidence was mitochondrial DNA (mtDNA) evidence collected from hair on a sweatshirt (materials from Hans et al., 2011). Two experts presented opposing opinions about the scientific evidence related to the mtDNA match estimate for the defendant’s identification. The quality and content of this mtDNA evidence differed based on the two conditions. The high quality evidence condition used a larger database than the low quality evidence to compare to the mtDNA sample and could exclude a larger percentage of people. In the decision aid condition, experts in the gist information group presented gist aid inspired visuals and examples to help explain the proportion of people that could not be excluded as a match. Those in the no gist information group were not given any aid to help them understand the mtDNA evidence presented. After viewing the trial, participants filled out a questionnaire on how well they understood the mtDNA evidence and their overall judgments of the case (e.g. verdict, witness credibility, scientific evidence strength). They filled this questionnaire out again after a 45-minute deliberation. Measures We measured Attitudes Toward Science (ATS) with indices of scientific promise and scientific reservations (Hans et al., 2011; originally developed by National Science Board, 2004; 2006). We used Drummond and Fischhoff’s (2015) Scientific Reasoning Scale (SRS) to measure scientific reasoning skills. Weller et al.’s (2012) Numeracy Scale (WNS) measured proficiency in reasoning with quantitative information. The NFC-Short Form (Cacioppo et al., 1984) measured need for cognition. We developed a 20-item multiple-choice comprehension test for the mtDNA scientific information in the cases (modeled on Hans et al., 2011, and McAuliff et al., 2009). Participants were shown 20 statements related to DNA evidence and asked whether these statements were True or False. The test was then scored out of 20 points. Results For this project, we measured calibration to the scientific evidence in a few different ways. We are building a full model with these various operationalizations to be presented at APLS, but focus only on one of the calibration DVs (i.e., objective understanding of the mtDNA evidence) in the current proposal. We conducted a general linear model with total score on the mtDNA understanding measure as the DV and quality of scientific evidence condition, decision aid condition, and the four individual difference measures (i.e., NFC, ATS, WNS, and SRS) as predictors. Contrary to our main hypotheses, neither evidence quality nor decision aid condition affected juror understanding. However, the individual difference variables did: we found significant main effects for Scientific Reasoning Skills, F(1, 427) = 16.03, p <.001, np2 = .04, Weller Numeracy Scale, F(1, 427) = 15.19, p <.001, np2 = .03, and Need for Cognition, F(1, 427) = 16.80, p <.001, np2 = .04, such that those who scored higher on these measures displayed better understanding of the scientific evidence. In addition there was a significant interaction of evidence quality condition and scores on the Weller’s Numeracy Scale, F(1, 427) = 4.10, p = .04, np2 = .01. Further results will be discussed. Discussion These data suggest jurors are not sensitive to differences in the quality of scientific mtDNA evidence, and also that our attempt at helping sensitize them with Fuzzy Trace Theory-inspired aids did not improve calibration. Individual scientific reasoning abilities and general cognition styles were better predictors of understanding this scientific information. These results suggest a need for further exploration of approaches to help jurors differentiate between high and low quality evidence. Note: The 3rd author was supported by an AP-LS AP Award for her role in this research. Learning Objective: Participants will be able to describe how individual differences in scientific reasoning skills help jurors understand complex scientific evidence.« less