An official website of the United States government
Digitalization shapes the ways of learning, working, and entertainment. The Internet, which enables us to connect and socialize is evolving to become the metaverse, a post-reality universe, enabling virtual life parallel to reality. In addition to gaming and entertainment, industry and academia have noticed the metaverse’s benefits and possibilities. For industry, the metaverse is the enabler of the future digital workplace, and for academia, digital learning spaces enable realistic virtual training environments. A connection bridging the virtual world with physical production systems is required to enable digital workplaces and digital learning spaces. In this publication, extended reality–digital twin to real use cases are presented. The presented use cases utilize extended reality as high-level user interfaces and digital twins to create a bridge between virtual environments and robotic systems in industry, academia, and underwater exploration.
more »
« less
Digital Twins Utilizing XR-Technology as Robotic Training Tools
Digital technology has evolved towards a new way of processing information: web searches, social platforms, internet forums, and video games have substituted reading books and writing essays. Trainers and educators currently face the challenge of providing natural training and learning environments for digital-natives. In addition to physical spaces, effective training and education require virtual spaces. Digital twins enable trainees to interact with real hardware in virtual training environments. Interactive real-world elements are essential in the training of robot operators. A natural environment for the trainee supports an interesting learning experience while including enough professional substances. We present examples of how virtual environments utilizing digital twins and extended reality can be applied to enable natural and effective robotic training scenarios. Scenarios are validated using cross-platform client devices for extended reality implementations and safety training applications.
more »
« less
- PAR ID:
- 10471227
- Publisher / Repository:
- Machines
- Date Published:
- Journal Name:
- Machines
- Volume:
- 11
- Issue:
- 1
- ISSN:
- 2075-1702
- Page Range / eLocation ID:
- 13
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Digital Twin (DT) technology offers real-time monitoring, simulation, optimization, and precise forecasting. However, its theoretical framework and practical implementation, particularly in digital random controlled trials (RCT), remain underdeveloped, limiting its full potential. Therefore, we seek to develop an application of digital twins with the virtual reality using data from four longitudinal RCT in Massachusetts. As case studies, we create digital twins of two individuals from these RCT. This application of digital twins provides an avenue for a more personalized healthcare experience for patients, the enhancement of medical simulations, and a visualization for predictive analytics. In the future, we will move from virtual reality to extended reality with AI-generated digital twin models.more » « less
-
Abstract PurposeSpecialized robotic and surgical tools are increasing the complexity of operating rooms (ORs), requiring elaborate preparation especially when techniques or devices are to be used for the first time. Spatial planning can improve efficiency and identify procedural obstacles ahead of time, but real ORs offer little availability to optimize space utilization. Methods for creating reconstructions of physical setups, i.e., digital twins, are needed to enable immersive spatial planning of such complex environments in virtual reality. MethodsWe present a neural rendering-based method to create immersive digital twins of complex medical environments and devices from casual video capture that enables spatial planning of surgical scenarios. To evaluate our approach we recreate two operating rooms and ten objects through neural reconstruction, then conduct a user study with 21 graduate students carrying out planning tasks in the resulting virtual environment. We analyze task load, presence, perceived utility, plus exploration and interaction behavior compared to low visual complexity versions of the same environments. ResultsResults show significantly increased perceived utility and presence using the neural reconstruction-based environments, combined with higher perceived workload and exploratory behavior. There’s no significant difference in interactivity. ConclusionWe explore the feasibility of using modern reconstruction techniques to create digital twins of complex medical environments and objects. Without requiring expert knowledge or specialized hardware, users can create, explore and interact with objects in virtual environments. Results indicate benefits like high perceived utility while being technically approachable, which may indicate promise of this approach for spatial planning and beyond.more » « less
-
The rapid evolution of modern manufacturing systems is driven by the integration of emerging metaverse technologies such as artificial intelligence (AI), digital twin (DT), and different forms of extended reality (XR) like virtual reality (VR), augmented reality (AR), and mixed reality (MR). These advances confront manufacturing workers with complex and evolving environments that demand digital literacy for problem solving in the future workplace. However, manufacturing industry faces a critical shortage of skilled workforce with digital literacy in the world. Further, global pandemic has significantly changed how people work and collaborate digitally and remotely. There is an urgent need to rethink digital platformization and leverage emerging technologies to propel industrial evolution toward human-centered manufacturing metaverse (MfgVerse). This paper presents a forward-looking perspective on the development of MfgVerse, highlighting current efforts in learning factory, cognitive digital twinning, and the new sharing economy of manufacturing-as-a-service (MaaS). MfgVerse is converging into multiplex networks, including a social network of human stakeholders, an interconnected network of manufacturing things or agents (e.g., machines, robots, facilities, material handling systems), a network of digital twins of physical things, as well as auxiliary networks of sales, supply chain, logistics, and remanufacturing systems. We also showcase the design and development of a virtual learning factory for workforce training. Finally, future directions, challenges, and opportunities are discussed for human-centered manufacturing metaverse. We hope this work helps stimulate more comprehensive studies and in-depth research efforts to advance MfgVerse technologies.more » « less
-
Abstract Fluid power systems can be expensive and difficult to access, making it challenging to provide hands-on training. This work discusses the incorporation of Mixed Reality (MR) technology in Fluid Power applications for providing a virtual training environment that simulates the behavior of fluid power systems, allowing users to receive immediate feedback on the system’s performance. Mixed reality is a digitized-based technology that integrates a virtual environment with our real world by utilizing real-world sensor data and computer models. This technology allows running simulations that examine the complexity of highly-coupled systems, producing new digital environments where physical and digital elements can interact in real-time. With all these features, MR technology can be a practical training tool for running virtual simulations that mimic real-life industry settings. It can extend the user with a virtual training environment, thus preparing the next generation of fluid power engineers and specialists. Throughout this work, we present the development and capabilities of a digitized virtual copy of a hydraulic excavator’s arm in an MR environment as a proof of concept. The MR arm module is developed and deployed using Microsoft’s Mixed Reality Tool Kit (MRTK) for Unity through HoloLens 2 MR headset. The MR development involves generating virtual copies of the mechanical and hydraulic subsystems, conducting the virtual assembly, and creating a user interface in the MR environment to visualize and interact with the model. The developed MR module enables visualizing the excavator’s internal structure, conducting the virtual assembly, and running virtual simulations, all of which assist in training future fluid power operators. It is an effective training tool that helps train junior engineers/technicians, cutting down on cost and time.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.3390/machines11010013
