An official website of the United States government
Decaying infrastructure maintenance cost allocation depends heavily on accurate and safe inspection in the field. New tools to conduct inspections can assist in prioritizing investments in maintenance and repairs. The industrial revolution termed as “Industry 4.0” is based on the intelligence of machines working with humans in a collaborative workspace. Contrarily, infrastructure management has relied on the human for making day-to-day decisions. New emerging technologies can assist during infrastructure inspections, to quantify structural condition with more objective data. However, today’s owners agree in trusting the inspector’s decision in the field over data collected with sensors. If data collected in the field is accessible during the inspections, the inspector decisions can be improved with sensors. New research opportunities in the human–infrastructure interface would allow researchers to improve the human awareness of their surrounding environment during inspections. This article studies the role of Augmented Reality (AR) technology as a tool to increase human awareness of infrastructure in their inspection work. The domains of interest of this research include both infrastructure inspections (emphasis on the collection of data of structures to inform management decisions) and emergency management (focus on the data collection of the environment to inform human actions). This article describes the use of a head-mounted device to access real-time data and information during their field inspection. The authors leverage the use of low-cost smart sensors and QR code scanners integrated with Augmented Reality applications for augmented human interface with the physical environment. This article presents a novel interface architecture for developing Augmented Reality–enabled inspection to assist the inspector’s workflow in conducting infrastructure inspection works with two new applications and summarizes the results from various experiments. The main contributions of this work to computer-aided community are enabling inspectors to visualize data files from database and real-time data access using an Augmented Reality environment.
more »
« less
This content will become publicly available on May 28, 2026
A surface electromyography–based deep learning model for guiding semi‐autonomous drones in road infrastructure inspection
While semi‐autonomous drones are increasingly used for road infrastructure inspection, their insufficient ability to independently handle complex scenarios beyond initial job planning hinders their full potential. To address this, the paper proposes a human–drone collaborative inspection approach leveraging flexible surface electromyography (sEMG) for conveying inspectors' speech guidance to intelligent drones. Specifically, this paper contributes a new data set,sEMGCommands forPilotingDrones (sCPD), and ansEMG‐basedCross‐subjectClassificationNetwork (sXCNet), for both command keyword recognition and inspector identification. sXCNet acquires the desired functions and performance through a synergetic effort of sEMG signal processing, spatial‐temporal‐frequency deep feature extraction, and multitasking‐enabled cross‐subject representation learning. The cross‐subject design permits deploying one unified model across all authorized inspectors, eliminating the need for subject‐dependent models tailored to individual users. sXCNet achieves notable classification accuracies of 98.1% on the sCPD data set and 86.1% on the public Ninapro db1 data set, demonstrating strong potential for advancing sEMG‐enabled human–drone collaboration in road infrastructure inspection.
more »
« less
- Award ID(s):
- 2335863
- PAR ID:
- 10612531
- Publisher / Repository:
- Wiley
- Date Published:
- Journal Name:
- Computer-Aided Civil and Infrastructure Engineering
- ISSN:
- 1093-9687
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Drones are increasingly used during routine inspections of bridges to improve data consistency, work efficiency, inspector safety, and cost effectiveness. Most drones, however, are operated manually within a visual line of sight and thus unable to inspect long-span bridges that are not completely visible to operators. In this paper, aerial nondestructive evaluation (aNDE) will be envisioned for elevated structures such as bridges, buildings, dams, nuclear power plants, and tunnels. To enable aerial nondestructive testing (aNDT), a human-robot system will be created to integrate haptic sensing and dexterous manipulation into a drone or a structural crawler in augmented/virtual reality (AR/VR) for beyond-visual-line-of-sight (BVLOS) inspection of bridges. Some of the technical challenges and potential solutions associated with aNDT&E will be presented. Example applications of the advanced technologies will be demonstrated in simulated bridge decks with stipulated conditions. The developed human-robot system can transform current on-site inspection to future tele-inspection, minimizing impact to traffic passing over the bridges. The automated tele-inspection can save as much as 75% in time and 95% in cost.more » « less
-
null (Ed.)The use of augmented reality (AR) with drones in infrastructure inspection can increase human capabilities by helping workers access hard-to-reach areas and supplementing their field of view with useful information. Still unknown though is how these aids impact performance when they are imperfect. A total of 28 participants flew as an autonomous drone while completing a target detection task around a simulated bridge. Results indicated significant differences between cued and un-cued trials but not between the four cue types: none, bounding box, corner-bound box, and outline. Differences in trust amongst the four cues indicate that participants may trust some cue styles more than others.more » « less
-
The use of augmented reality (AR) with drones in infrastructure inspection can increase human capabilities by helping workers access hard-to-reach areas and supplementing their field of view with useful information. Still unknown though is how these aids impact performance when they are imperfect. A total of 28 participants flew as an autonomous drone while completing a target detection task around a simulated bridge. Results indicated significant differences between cued and un-cued trials but not between the four cue types: none, bounding box, corner-bound box, and outline. Differences in trust amongst the four cues indicate that participants may trust some cue styles more than others.more » « less
-
null (Ed.)Bridge inspection is an important step in preserving and rehabilitating transportation infrastructure for extending their service lives. The advancement of mobile robotic technology allows the rapid collection of a large amount of inspection video data. However, the data are mainly the images of complex scenes, wherein a bridge of various structural elements mix with a cluttered background. Assisting bridge inspectors in extracting structural elements of bridges from the big complex video data, and sorting them out by classes, will prepare inspectors for the element-wise inspection to determine the condition of bridges. This article is motivated to develop an assistive intelligence model for segmenting multiclass bridge elements from the inspection videos captured by an aerial inspection platform. With a small initial training dataset labeled by inspectors, a Mask Region-based Convolutional Neural Network pre-trained on a large public dataset was transferred to the new task of multiclass bridge element segmentation. Besides, the temporal coherence analysis attempts to recover false negatives and identify the weakness that the neural network can learn to improve. Furthermore, a semi-supervised self-training method was developed to engage experienced inspectors in refining the network iteratively. Quantitative and qualitative results from evaluating the developed deep neural network demonstrate that the proposed method can utilize a small amount of time and guidance from experienced inspectors (3.58 h for labeling 66 images) to build the network of excellent performance (91.8% precision, 93.6% recall, and 92.7% f1-score). Importantly, the article illustrates an approach to leveraging the domain knowledge and experiences of bridge professionals into computational intelligence models to efficiently adapt the models to varied bridges in the National Bridge Inventory.more » « less
