An official website of the United States government
With the increasing use of virtual simulated environments and immersive technologies in STEM education and workforce training, it is becoming increasingly important to study and understand how learners’ interactions and navigation in virtual environments affect their learning and skill development. In this paper, we quantify and assess the effect of learners’ navigation in an immersive simulated environment on learning outcomes, where navigation is characterized by the total time spent in the simulation and time allocations to different areas within the virtual environment. We implement a set of immersive simulation-based learning (ISBL) modules in an undergraduate computer science course with eighteen students and record their screen as they navigate in the simulation environment to perform the tasks needed to complete the ISBL assignments. We use a video analytics tool to process and analyze the videos and collect statistics related to a set of navigation-related measures for each student. We also use surveys to collect data on students’ demographics, prior knowledge and experience, personality, experiential learning, and self-assessment of learning. We then perform a set of multivariable regression analyses to characterize and explain the relationship between navigation measures and constructs assessed via survey instruments to determine how/if users’ navigation in the simulated environment can be a predictor of their learning outcomes. The results indicate that the total time spent in the simulation and the distribution of time allocations among different areas within the simulated environment are predictors of experiential learning and students’ self-assessment of learning.
more »
« less
An Open Source Video Analytics Tool for Analyzing Learner Navigation in Immersive Simulated Environments
In educational research, user-simulation interaction is gaining importance as it provides key insights into the effectiveness of simulation-based learning and immersive technologies. A common approach to study user-simulation interaction involves manually analyzing participant interaction in real-time or via video recordings, which is a tedious process. Surveys/questionnaires are also commonly used but are open to subjectivity and only provide qualitative data. The tool proposed in this paper, which we call Environmental Detection for User-Simulation Interaction Measurement (EDUSIM), is a publicly available video analytics tool that receives screen-recorded video input from participants interacting with a simulated environment and outputs statistical data related to time spent in pre-defined areas of interest within the simulation model. The proposed tool utilizes machine learning, namely multi-classification Convolutional Neural Networks, to provide an efficient, automated process for extracting such navigation data. EDUSIM also implements a binary classification model to flag imperfect input video data such as video frames that are outside the specified simulation environment. To assess the efficacy of the tool, we implement a set of immersive simulation-based learning (ISBL) modules in an undergraduate database course, where learners record their screens as they interact with a simulation to complete their ISBL assignments. We then use the EDUSIM tool to analyze the videos collected and compare the tool’s outputs with the expected results obtained by manually analyzing the videos.
more »
« less
- Award ID(s):
- 2000599
- PAR ID:
- 10415731
- Date Published:
- Journal Name:
- 2023 Annual Modeling and Simulation Conference
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
We propose and assess the effectiveness of novel immersive simulation-based learning (ISBL) modules for teaching and learning engineering economy concepts. The proposed intervention involves technology-enhanced problem-based learning where the problem context is represented via a three-dimensional (3D), animated discrete-event simulation model that resembles a real-world system or situation that students may encounter in future professional settings. Students can navigate the simulated environment in both low- and high-immersion modes (i.e., on a typical personal computer or via a virtual reality headset). The simulation helps contextualize and visualize the problem setting, allowing students to observe and understand the underlying dynamics, collect relevant data/information, evaluate the effect of changes on the system, and learn by doing. The proposed ISBL approach is supported by multiple pedagogical and psychological theories, namely the information processing approach to learning theory, constructivism theory, self-determination theory, and adult learning theory. We design and implement a set of ISBL modules in an introductory undergraduate engineering economy class. The research experiments involve two groups of students: a control group and an intervention group. Students in the control group complete a set of traditional assignments, while the intervention group uses ISBL modules. We use well-established survey instruments to collect data on demographics, prior preparation, motivation, experiential learning, engineering identity, and self-assessment of learning objectives based on Bloom’s taxonomy. Statistical analysis of the results suggests that ISBL enhances certain dimensions related to motivation and experiential learning, namely relevance, confidence, and utility. We also provide a qualitative assessment of the proposed intervention based on detailed, one-on-one user testing and evaluation interviews.more » « less
-
Badminton is a fast-paced sport that requires a strategic combination of spatial, temporal, and technical tactics. To gain a competitive edge at high-level competitions, badminton professionals frequently analyze match videos to gain insights and develop game strategies. However, the current process for analyzing matches is time-consuming and relies heavily on manual note-taking, due to the lack of automatic data collection and appropriate visualization tools. As a result, there is a gap in effectively analyzing matches and communicating insights among badminton coaches and players. This work proposes an end-to-end immersive match analysis pipeline designed in close collaboration with badminton professionals, including Olympic and national coaches and players. We present VIRD, a VR Bird (i.e., shuttle) immersive analysis tool, that supports interactive badminton game analysis in an immersive environment based on 3D reconstructed game views of the match video. We propose a top-down analytic workflow that allows users to seamlessly move from a high-level match overview to a detailed game view of individual rallies and shots, using situated 3D visualizations and video. We collect 3D spatial and dynamic shot data and player poses with computer vision models and visualize them in VR. Through immersive visualizations, coaches can interactively analyze situated spatial data (player positions, poses, and shot trajectories) with flexible viewpoints while navigating between shots and rallies effectively with embodied interaction. We evaluated the usefulness of VIRD with Olympic and national-level coaches and players in real matches. Results show that immersive analytics supports effective badminton match analysis with reduced context-switching costs and enhances spatial understanding with a high sense of presence.more » « less
-
null (Ed.)Situational awareness provides the decision making capability to identify, process, and comprehend big data. In our approach, situational awareness is achieved by integrating and analyzing multiple aspects of data using stacked bar graphs and geographic representations of the data. We provide a data visualization tool to represent COVID pandemic data on top of the geographical information. The combination of geospatial and temporal data provides the information needed to conduct situational analysis for the COVID-19 pandemic. By providing interactivity, geographical maps can be viewed from different perspectives and offer insight into the dynamical aspects of the COVID-19 pandemic for the fifty states in the USA. We have overlaid dynamic information on top of a geographical representation in an intuitive way for decision making. We describe how modeling and simulation of data increase situational awareness, especially when coupled with immersive virtual reality interaction. This paper presents an immersive virtual reality (VR) environment and mobile environment for data visualization using Oculus Rift head-mounted display and smartphones. This work combines neural network predictions with human-centric situational awareness and data analytics to provide accurate, timely, and scientific strategies in combatting and mitigating the spread of the coronavirus pandemic. Testing and evaluation of the data visualization tool have been done with real-time feed of COVID pandemic data set for immersive environment, non-immersive environment, and mobile environment.more » « less
-
This paper presents a study on the impact of class delivery mode (remote vs. in-person) on students’ learning experience when Immersive Simulation-Based Learning (ISBL) modules are used as course assignments. ISBL involves problem-based learning via a 3-dimensional (3D) simulated environment that mimics real-life applications such as manufacturing and healthcare systems, airports, and other service systems. Within the simulated environment, students can observe the corresponding system, collect data, understand relationships between the system components, make changes to the model and observe the impact of those changes, and learn by doing. ISBL is advantageous when access to real-world facilities is difficult or impossible due to geographical barriers or safety concerns as well as in remote and online learning due to geographically dispersed students. This study compares two groups of students. Both groups are taught by the same instructor and use the same course material, including the ISBL modules. The only difference between the two groups is the course delivery mode, where one group is taught remotely through synchronous online sessions, and the other is taught in person in a traditional classroom setting. We collect data on demographics, prior preparation, motivation, experiential learning, usability scale, and self-assessment of learning objectives based on Bloom’s taxonomy. We then perform statistical comparisons to investigate the impact of delivery mode when ISBL modules are used. We use the comparison results to test the hypothesis that ISBL modules will help maintain remote students’ motivation and learning outcomes compared to in-person students. The results show no statistically significant difference between the two groups on any measure, suggesting that ISBL is equally effective in the two delivery modes.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
