skip to main content


Title: Learning Analytics for Assessing Hands-on Laboratory Skills in Science Classrooms Using Bayesian Network Analysis
Learning analytics, referring to the measurement, collection, analysis, and reporting of data about learners and their contexts in order to optimize learning and the environments in which it occurs, is proving to be a powerful approach for understanding and improving science learning. However, few studies focused on leveraging learning analytics to assess hands-on laboratory skills in K-12 science classrooms. This study demonstrated the feasibility of gauging laboratory skills based on students’ process data logged by a mobile augmented reality (AR) application for conducting science experiments. Students can use the mobile AR technology to investigate a variety of science phenomena that involve concepts central to physics understanding. Seventy-two students from a suburban middle school in the Northeastern United States participated in this study. They conducted experiments in pairs. Mining process data using Bayesian networks showed that most students who participated in this study demonstrated some degree of proficiency in laboratory skills. Also, findings indicated a positive correlation between laboratory skills and conceptual learning. The results suggested that learning analytics provides a possible solution to measure hands-on laboratory learning in real-time and at scale.  more » « less
Award ID(s):
2054079
NSF-PAR ID:
10339588
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
Research in Science Education
ISSN:
0157-244X
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    Augmented reality (AR) has the potential to fundamentally transform science education by making learning of abstract science ideas tangible and engaging. However, little is known about how students interacted with AR technologies and how these interactions may affect learning performance in science laboratories. This study examined high school students’ navigation patterns and science learning with a mobile AR technology, developed by the research team, in laboratory settings. The AR technology allows students to conduct hands-on laboratory experiments and interactively explore various science phenomena covering biology, chemistry, and physics concepts. In this study, seventy ninth-grade students carried out science laboratory experiments in pairs to learn thermodynamics. Our cluster analysis identified two groups of students, which differed significantly in navigation length and breadth. The two groups demonstrated unique navigation patterns that revealed students’ various ways of observing, describing, exploring, and evaluating science phenomena. These navigation patterns were associated with learning performance as measured by scores on lab reports. The results suggested the need for providing access to multiple representations and different types of interactions with these representations to support effective science learning as well as designing representations and connections between representations to cultivate scientific reasoning skills and nuanced understanding of scientific processes. 
    more » « less
  2. This Work-in-Progress paper investigates how students participating in a chemical engineering (ChE) Research Experience for Undergraduates (REU) program conceptualize and make plans for research projects. The National Science Foundation has invested substantial financial resources in REU programs, which allow undergraduate students the opportunity to work with faculty in their labs and to conduct hands-on experiments. Prior research has shown that REU programs have an impact on students’ perceptions of their research skills, often measured through the Undergraduate Research Student Self-Assessment (URSSA) survey. However, few evaluation and research studies have gone beyond perception data to include direct measures of students’ gains from program participation. This work-in-progress describes efforts to evaluate the impact of an REU on students’ conceptualization and planning of research studies using a pre-post semi-structured interview process. The construct being investigated for this study is planning, which has been espoused as a critical step in the self-regulated learning (SRL) process (Winne & Perry, 2000; Zimmerman, 2008). Students who effectively self-regulate demonstrate higher levels of achievement and comprehension (Dignath & Büttner, 2008), and (arguably) work efficiency. Planning is also a critical step in large projects, such as research (Dvir & Lechler, 2004). Those who effectively plan their projects make consistent progress and are more likely to achieve project success (Dvir, Raz, & Shenhar, 2003). Prior REU research has been important in demonstrating some positive impacts of REU programs, but it is time to dig deeper into the potential benefits to REU participation. Many REU students are included in weekly lab meetings, and thus potentially take part in the planning process for research projects. Thus, the research question explored here is: How do REU participants conceptualize and make plans for research projects? The study was conducted in the ChE REU program at a large, mid-Atlantic research-oriented university during the summer of 2018. Sixteen students in the program participated in the study, which entailed them completing a planning task followed by a semi-structured interview at the start and the end of the REU program. During each session, participants read a case statement that asked them to outline a plan in writing for a research project from beginning to end. Using semi-structured interview procedures, their written outlines were then verbally described. The verbalizations were recorded and transcribed. Two members of the research team are currently analyzing the responses using an open coding process to gain familiarity with the transcripts. The data will be recoded based on the initial open coding and in line with a self-regulatory and project-management framework. Results: Coding is underway, preliminary results will be ready by the draft submission deadline. The methods employed in this study might prove fruitful in understanding the direct impact on students’ knowledge, rather than relying on their perceptions of gains. Future research could investigate differences in students’ research plans based on prior research experience, research intensity of students’ home institutions, and how their plans may be impacted by training. 
    more » « less
  3. null (Ed.)
    Spatial reasoning skills contribute to performance in many STEM fields. For example, drawing sectional views of three-dimensional objects is an essential skill for engineering students. There is considerable variation in the spatial reasoning skills of prospective engineering students, putting some at risk for compromised performance in their classes. This study takes place in a first-year engineering Spatial Visualization course to integrate recent practices in engineering design education with cognitive psychology research on the nature of spatial learning. We employed three main pedagogical strategies in the course - 1) in class instruction on sketching; 2) spatial visualization training; and 3) manipulation of physical objects (CAD/3D print creations). This course endeavors to use current technology, online accessibility, and implementation of the three pedagogical strategies to bring about student growth in spatial reasoning. This study is designed to determine the effect of adding two different spatial reasoning training apps to this environment. Over 230 students (three sections) participated in our study. In two of the three sections, students received interactive spatial visualization training using either a spatial visualization mobile touchscreen app in one section or an Augmented Reality (AR) app in the other section. Research suggests that there are benefits to using the Spatial Vis Classroom mobile app for college students.The app has been shown to increase student persistence resulting in large learning gains as measured by the Purdue assessment of spatial visualization (PSVT-R), especially for students starting with poor spatial visualization skills. The Spatial Vis Classroom app can be used in the classroom or assigned as homework. The AR app is designed to help users develop their mental rotation abilities. It is designed to support a holistic understanding of 3-dimensional objects, and research has shown that, in combination with a traditional curriculum, it increases students’ abilities also measured by the PSVT-R. Of particular interest, the data suggest that the app overcomes the advantage found by males over females in a traditional class alone focused on spatial reasoning. Both of the course sections were required to use the apps for approximately the same time in class and outside of class. Students in the control section were required to do hand sketching activities in class and outside of class, with roughly the same completion time as for the sections with the apps. Students grades were not affected by using the three different approaches as grading was based on completion only. Based on current literature, we hypothesize that overall benefits (PSVT-R gains) will be comparable across the 3 treatments but there will be different effects on attitude and engagement (confidence,enjoyment, and self-efficacy). Lastly, we hypothesize that the treatments will have different effects on male/female and ethnic categories of the study participants. The final paper will include an analysis of results and a report of the findings. 
    more » « less
  4. This paper examines the learning experiences of undergraduate students who conducted research as part of a multidisciplinary team. The research project involved five undergraduate students with different backgrounds in engineering as well as in arts and sciences, supervised by four architecture and civil engineering faculty and their three PhD students. The research investigates the behavior of new Tessellated Structural-Architectural (TeSA) systems made of repetitive patterns of tiles (tessellations) that are both aesthetically appealing and load bearing. The undergraduate students worked on three tasks: (1) studying the behavior of TeSA shear walls using small scale earthquake simulator tests, (2) studying the shear capacity of reinforced concrete TeSA tiles, and (3) studying the effect of different shapes and interlocking patterns on the performance of small scale TeSA beams. The undergraduate students used hands-on experiments and laboratory testing to study the performance of 3D printed or prefabricated interlocking tessellations. This paper discusses the technical skills, fundamental concepts, and power skills (communicating, writing, presenting, etc.) that the students obtained, as well as the challenges that they encountered. The students found the process of developing and executing hands-on experiments and analyzing experimental results effective for learning new technologies and fundamental concepts. These concepts included 3D printing methods, natural frequency of a structure, and structural response subjected to a shear force. Peer learning, collaboration between students with different backgrounds, and group discussions with all the team members facilitated a deeper understanding and broader perspective on design, performance, and construction of TeSA systems. The project took place during the COVID-19 pandemic, and the students found working and meeting remotely challenging at times. Proper guidance and timely feedback by the project investigators and their PhD students helped with resolving the challenges. 
    more » « less
  5. Metacognition is the understanding of your own knowledge including what knowledge you do not have and what knowledge you do have. This includes knowledge of strategies and regulation of one’s own cognition. Studying metacognition is important because higher-order thinking is commonly used, and problem-solving skills are positively correlated with metacognition. A positive previous disposition to metacognition can improve problem-solving skills. Metacognition is a key skill in design and manufacturing, as teams of engineers must solve complex problems. Moreover, metacognition increases individual and team performance and can lead to more original ideas. This study discusses the assessment of metacognitive skills in engineering students by having the students participate in hands-on and virtual reality activities related to design and manufacturing. The study is guided by two research questions: (1) do the proposed activities affect students’ metacognition in terms of monitoring, awareness, planning, self-checking, or strategy selection, and (2) are there other components of metacognition that are affected by the design and manufacturing activities? The hypothesis is that the participation in the proposed activities will improve problem-solving skills and metacognitive awareness of the engineering students. A total of 34 undergraduate students participated in the study. Of these, 32 were male and 2 were female students. All students stated that they were interested in pursuing a career in engineering. The students were divided into two groups with the first group being the initial pilot run of the data. In this first group there were 24 students, in the second group there were 10 students. The groups’ demographics were nearly identical to each other. Analysis of the collected data indicated that problem-solving skills contribute to metacognitive skills and may develop first in students before larger metacognitive constructs of awareness, monitoring, planning, self-checking, and strategy selection. Based on this, we recommend that the problem-solving skills and expertise in solving engineering problems should be developed in students before other skills emerge or can be measured. While we are sure that the students who participated in our study have awareness as well as the other metacognitive skills in reading, writing, science, and math, they are still developing in relation to engineering problems. 
    more » « less