skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Multi-3D-Models Registration-Based Augmented Reality Instructions for Assembly
BRICKxAR (Multi 3D Models/M3D) prototype offers markerless, in-situ, and step-by-step, highly accurate Augmented Reality (AR) assembly instructions for large or small part assembly. The prototype employs multiple assembly phases of deep learning-trained 3D model-based AR registration coupled with a step count. This ensures object recognition and tracking persist while the model updates at each step, even if a part's location is not visible to the AR camera. The use of phases simplifies the complex assembly instructions. The testing and heuristic evaluation findings indicate that BRICKxAR (M3D) provides robust instructions for assembly, promising potential applicability at different scales and scenarios.  more » « less
Award ID(s):
2242432 2119549
PAR ID:
10492324
Author(s) / Creator(s):
;
Publisher / Repository:
IEEE Xplore Digital Library
Date Published:
Journal Name:
IEEE Conference on Virtual Reality + 3D User Interfaces Abstracts and Workshops (VRW)
Subject(s) / Keyword(s):
In-situ AR AR-Assisted Assembly Augmented Reality Assembly Human-Computer Interaction
Format(s):
Medium: X
Location:
Orlando, FL
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; ; (, IEEE International Symposium on Mixed and Augmented Reality (ISMAR))
    Augmented/Virtual reality and video-based media play a vital role in the digital learning revolution to train novices in spatial tasks. However, creating content for these different media requires expertise in several fields. We present EditAR, a unified authoring, and editing environment to create content for AR, VR, and video based on a single demonstration. EditAR captures the user’s interaction within an environment and creates a digital twin, enabling users without programming backgrounds to develop content. We conducted formative interviews with both subject and media experts to design the system. The prototype was developed and reviewed by experts. We also performed a user study comparing traditional video creation with 2D video creation from 3D recordings, via a 3D editor, which uses freehand interaction for in-headset editing. Users took 5 times less time to record instructions and preferred EditAR, along with giving significantly higher usability scores. 
    more » « less
  2. ; (, Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI'19))
    We propose a method that helps an unskilled user to carve a physical replica of a 3D CAD model while only using manual cutting tools. The method starts by analyzing the input CAD model and generates a set of carving instructions. Then using a projector, we project the instructions sequentially one at a time to a block of material to guide the user in performing each of them. After each cutting step, we use the projector-camera setup to 3D scan the object after cutting. And automatically align the scanned point cloud to the CAD model, to prepare the position for the next instruction. We demonstrate a complete system to support this operation and show several examples manually carved while using the system. 
    more » « less
  3. ; ; ; ; (, ACM)
    We present ASM Visualizer, a tool that is designed to help students learn assembly programming, aiding in their understanding of how assembly instructions are executed and the relationship between assembly and equivalent high-level language code. Our tool allows a user to step both forward and backward through the execution of an assembly program, one instruction at a time, seeing how instruc- tions use and modify values in stack memory and CPU registers. ASM Visualizer presents three user-interface modes, supporting different stages of learning assembly programming. Beginners can step through basic arithmetic instructions, whereas more advanced learners can trace through function call/return sequences, stack frame manipulation, or entire assembly programs. We present our experiences using ASM Visualizer in introductory-level courses at our two institutions, and we discuss other ways in which our tool could be used by educators in both introductory and advanced CS courses. Results from a preliminary assessment of students using our tool show that students gain confidence in their understanding of different aspects of assembly programming. We feel that the visual interface to assembly code execution that ASM Visualizer provides is key to helping students understand assembly. 
    more » « less
  4. ; ; ; ; ; (, Frameless Symposium)
    Hyperion is a 3D visualization platform for optical design. It provides a fully immersive, intuitive, and interactive 3D user experience by leveraging existing AR/VR technologies. It enables the visualization of models of folded freeform optical systems in a dynamic 3D environment. The frontend user experience is supported by the computational ray-tracing engine of Eikonal+, an optical design research software currently being developed. We have built a cross-platform light-weight version of Eikonal+ that can communicate with any user interface or other scientific software. We have also demonstrated a prototype of the Hyperion 3D user experience using a Hololens AR display. 
    more » « less
  5. ; ; ; (, CIRP Annals)
    Autonomous robots that understand human instructions can significantly enhance the efficiency in human-robot assembly operations where robotic support is needed to handle unknown objects and/or provide on-demand assistance. This paper introduces a vision AI-based method for human-robot collaborative (HRC) assembly, enabled by a large language model (LLM). Upon 3D object reconstruction and pose establishment through neural object field modelling, a visual servoing-based mobile robotic system performs object manipulation and navigation guidance to a mobile robot. The LLM model provides text-based logic reasoning and high-level control command generation for natural human-robot interactions. The effectiveness of the presented method is experimentally demonstrated. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email