skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Enabling Intelligent Immersive Learning using Deep Learning-based Learner Confidence Estimation
In today’s world, augmented reality and virtual reality (AR/VR) technologies have become more accessible to the public than ever. This brings the possibility of immersive learning to the forefront of education for future generations. However, there is still much to discover and improve in using these technologies to analyze and understand learning. This paper explores the utilization of data captured through AR/VR headsets during an immersive training program for industrial robotics. This includes data on time spent, eye gaze, and hand movement during a range of activities to track a learner’s understanding of the content and intelligently estimate learner confidence within these environments using deep learning. Leveraging a dataset that comprises responses and confidence levels from 10 individuals across 35 questions, we aim to improve the uses and applicability of confidence estimation. We explore the possibility of training a model using learners’ data to dynamically fine-tune lessons and activities for each individual, thereby improving performance. We demonstrate that a pre-trained compact LSTM classification model can be fine-tuned with relatively small data, for enhanced performance on an individual basis for better personalized learning.  more » « less
Award ID(s):
2202610
PAR ID:
10599076
Author(s) / Creator(s):
; ; ; ;
Corporate Creator(s):
Publisher / Repository:
Institute of Electrical and Electronics Engineers (IEEE)
Date Published:
Page Range / eLocation ID:
55-60
Subject(s) / Keyword(s):
International Conference on Information Reuse and Integration for Data Science (IRI)
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; (, IEEE 22nd International Conference on Information Reuse and Integration for Data Science (IRI))
    Immersive Learning Environments (ILEs) developed in Virtual and Augmented Reality (VR/AR) are a novel pro- fessional training platform. An ILE can facilitate an Adaptive Learning System (ALS), which has proven beneficial to the learning process. However, there is no existing AI-ready ILE that facilitates collecting multimedia multimodal data from the environment and users for training AI models, nor allows for the learning contents and complex learning process to be dynamically adapted by an ALS. This paper proposes a novel multimedia system in VR/AR to dynamically build ILEs for a wide range of use-cases, based on a description language for the generalizable ILE structure. It will detail users’ paths and conditions for completing learning activities, and a content adaptation algorithm to update the ILE at runtime. Human and AI systems can customize the environment based on user learning metrics. Results show that this framework is efficient and low- overhead, suggesting a path to simplifying and democratizing the ILE development without introducing bloat. Index Terms—virtual reality, augmented reality, content generation, immersive learning, 3D environments 
    more » « less
  2. (, University of Arkansas)
    Personalized learning, which customizes content and instructional sequences to account for differences in ability, experience, and sociocultural backgrounds, holds great promise for transforming education. This transformation is increasingly driven by significant advancements in Artificial Intelligence (AI). AI enables detailed analysis and reporting of learners' performance data, paving the way for the development of intelligent adaptive learning systems that offer personalized feedback aligned with each learner’s unique needs and progress. In parallel, immersive technologies are playing a pivotal role in enhancing educational experiences. Technologies such as Virtual Reality (VR) and Augmented Reality (AR) create engaging, interactive environments that deepen learners' understanding and retention of complex concepts. Dr. Vassigh's presentation explores the integration of AI and VR in education, illustrated through a case study from an ongoing project. The talk will highlight the refinement of learning processes through these technologies and demonstrate how they can impact learner engagement and performance. 
    more » « less
  3. ; ; ; ; ; ; (, World Conference on Engineering Education)
    null (Ed.)
    Advancements in Artificial Intelligence (AI), Information Technology, Augmented Reality (AR) and Virtual Reality (VR), and Robotic Automation is transforming jobs in the Architecture, Engineering and Construction (AEC) industries. However, it is also expected that these technologies will lead to job displacement, alter skill profiles for existing jobs, and change how people work. Therefore, preparing the workforce for an economy defined by these technologies is imperative. This ongoing research focuses on developing an immersive learning training curriculum to prepare the future workforce of the building industry. In this paper we are demonstrating a prototype of a mobile AR application to deliver lessons for training in robotic automation for construction industry workers. The application allows a user to interact with a virtual robot manipulator to learn its basic operations. The goal is to evaluate the effectiveness of the AR application by gauging participants' performance using pre and post surveys. 
    more » « less
  4. ; ; (, AMPS Architecture, media, politics, society proceedings series)
    This project addresses the urgent need for inclusive and scalable robotics training in architecture, engineering, and construction (AEC) through the integration of artificial intelligence (AI) and extended reality (XR) technologies. In collaboration with three Minority Serving Institutions (Florida International University, Arizona State University, and University of Hawai‘i at Mānoa), we developed and tested immersive, adaptive learning environments that personalize robotics education for diverse student populations. These efforts include a VR-based curriculum for industrial robotics, an AR curriculum for environmental sensing technologies, and an overarching Robotics Academy framework that promotes open knowledge exchange and workforce connectivity. By combining real-time performance analytics, natural language processing, and biometric inputs, our systems support individualized learning paths and help mitigate algorithmic bias. This research advances equitable access to robotics education and provides a replicable model for technology-driven workforce development in the AEC sector. Ongoing evaluation demonstrates improved learner engagement, accessibility, and cross-platform skill transferability. 
    more » « less
  5. ; ; (, Springer)
    Yan, C; Chai, H; Sun, T; Yuan, PF (Ed.)
    Abstract. The building industry is facing environmental, technological, and economic challenges, placing significant pressure on preparing the workforce for Industry 4.0 needs. The fields of Architecture, Engineering, and Construction (AEC) are being reshaped by robotics technologies which demand new skills and creating disruptive change to job markets. Addressing the learning needs of AEC students, professionals, and industry workers is critical to ensuring the competitiveness of the future workforce. In recent years advancements in Information Technology, Augmented Reality (AR), Virtual Reality (VR), and Artificial Intelligence (AI) have led to new research and theories on virtual learning environments. In the AEC fields researchers are beginning to rethink current robotics training to counteract costly and resource-intensive in-person learning. However, much of this work has been focused on simulation physics and has yet to adequately address how to engage AEC learners with different learning abilities, styles, and diverse backgrounds.This paper presents the advantages and difficulties associated with using new technologies to develop virtual reality (VR) learning games for robotics. It describes an ongoing project for creating performance driven curriculum. Drawing on the Constructivist Learning Theory, the affordances of Adaptive Learning Systems, and data collection methods from the VR game environment, the project provides a customized and performance-oriented approach to carrying out practical robotics tasks in real-world scenarios. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email