Search for: All records

Not AvailableExtended Reality (XR) learning environments generate rich behavioral data through embodied interaction—head movements, gaze patterns, and spatial navigation—that could enable passive assessment without interrupting the learning experience. We investigate spatial exploration behavior in ACHIEVE, an XR environment for neural network visualization learning. In a between-subjects study (N = 56), XR participants exhibited significantly different spatial behavior than desktop users: over 14 times greater pointer movement (M = 132.5 m vs. M = 9.5 m), extensive head rotation (M = 9,818◦ ), and M = 22.5 m of head translation during the learning session. We visualize individual exploration patterns through head and pointer trajectory traces, revealing substantial variation in how learners navigate the 3D content. These spatial metrics, automatically captured during learning, represent a promising avenue for passive assessment of embodied engagement—enabling educators to identify struggling learners, provide personalized feedback, and adapt content delivery without intrusive testing. Full learning outcomes and user experience metrics are reported in companion publications; here we focus on spatial behavior as a novel contribution toward spatialized learning analytics in education 

The application of extended reality (XR) technology in education has been growing for the last two decades. XR offers immersive and interactive visualization experiences that can enhance learning by making it engaging. Recent technological advances have led to the availability of high-quality and affordable XR headsets. These advancements have spurred a wave of research focused on designing, implementing, and validating XR educational interventions. Limited literature focuses on the recent trends of XR within science, technology, engineering, and mathematics (STEM) education. Thus, this paper presents an umbrella review that explores the exploding field of XR and its transformative potential in STEM education. Using six online databases, the review zoomed in on 17 out of 1972 papers on XR for STEM education, published between 2020 and 2023, following the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines. The results highlighted the types of XR technology applied (i.e., virtual reality and augmented reality), the specific STEM disciplines involved, the focus of each study reviewed, and the major findings from recent reviews. Overall, the educational benefits of using XR technology in STEM education are apparent: XR boosts student motivation, facilitates learning engagement, and improves skills, for example. However, using XR in education still has challenges that must be addressed, such as the physical discomfort of the learner wearing the XR headset and technical glitches. Besides revealing trends of using XR in STEM education, this umbrella review encourages reflection on current practices and suggests ways to apply XR to STEM education effectively. 

We introduce, implement, and test VR BioTalk, a hands-free, immersive, voice-controlled visual analytics system for phenotypic data. Our system does not require any programming knowledge. Yet, it enables users to receive an interactive solution to complex tasks involving large datasets through simple verbal commands, such as “Show me all leaves smaller than the average and calculate their leaf area index.” We claim three main contributions: 1) preprocessing and feature extraction of point cloud data for interactive visual analytics, 2) development of a novel interface that converts user speech into commands, and 3) an immersive VR visualization that executes the commands and displays the results in VR. The speech recognition system’s precision has been validated on 416 spoken commands across 13 English accents, with an accuracy of around 99.7% for transcription and 94% for command recognition. The visualization averages 63 FPS, and the system’s response time is approximately 1.25 seconds. We tested VR BioTalk on several tasks that would otherwise require extensive programming knowledge. We tested our system with 9 participants, and the results show that VR BioTalk is highly usable, engaging, and easy to use, enabling experts with no programming background to explore large phenotyping datasets and generate hypotheses in natural language. 

Abstract We consider semantic image segmentation. Our method is inspired by Bayesian deep learning which improves image segmentation accuracy by modeling the uncertainty of the network output. In contrast to uncertainty, our method directly learns to predict the erroneous pixels of a segmentation network, which is modeled as a binary classification problem. It can speed up training comparing to the Monte Carlo integration often used in Bayesian deep learning. It also allows us to train a branch to correct the labels of erroneous pixels. Our method consists of three stages: (i) predict pixel-wise error probability of the initial result, (ii) redetermine new labels for pixels with high error probability, and (iii) fuse the initial result and the redetermined result with respect to the error probability. We formulate the error-pixel prediction problem as a classification task and employ an error-prediction branch in the network to predict pixel-wise error probabilities. We also introduce a detail branch to focus the training process on the erroneous pixels. We have experimentally validated our method on the Cityscapes and ADE20K datasets. Our model can be easily added to various advanced segmentation networks to improve their performance. Taking DeepLabv3+ as an example, our network can achieve 82.88% of mIoU on Cityscapes testing dataset and 45.73% on ADE20K validation dataset, improving corresponding DeepLabv3+ results by 0.74% and 0.13% respectively.