An official website of the United States government
Reconnaissance teams collect perishable data after each disaster to learn about building performance. However, often these large image sets are not adequately curated, nor do they have sufficient metadata (e.g., GPS), hindering any chance to identify images from the same building when collected by different reconnaissance teams. In this study, Siamese convolutional neural networks (S‐CNN) are implemented and repurposed to establish a building search capability suitable for post‐disaster imagery. This method can automatically rank and retrieve corresponding building images in response to a single query using an image. In the demonstration, we utilize real‐world images collected from 174 reinforced‐concrete buildings affected by the 2016 Southern Taiwan and the 2017 Pohang (South Korea) earthquake events. A quantitative performance evaluation is conducted by examining two metrics introduced for this application: Similarity Score (SS) and Similarity Rank (SR).
more »
« less
Learning from Earthquakes Using the Automatic Reconnaissance Image Organizer (ARIO)
In the aftermath of earthquake events, reconnaissance teams are deployed to gather vast amounts of images, moving quickly to capture perishable data to document the performance of infrastructure before they are destroyed. Learning from such data enables engineers to gain new knowledge about the real-world performance of structures. This new knowledge, extracted from such visual data, is critical to mitigate the risks (e.g., damage and loss of life) associated with our built environment in future events. Currently, this learning process is entirely manual, requiring considerable time and expense. Thus, unfortunately, only a tiny portion of these images are shared, curated, and actually utilized. The power of computers and artificial intelligence enables a new approach to organize and catalog such visual data with minimal manual effort. Here we discuss the development and deployment of an organizational system to automate the analysis of large volumes of post-disaster visual data, images. Our application, named the Automated Reconnaissance Image Organizer (ARIO), allows a field engineer to rapidly and automatically categorize their reconnaissance images. ARIO exploits deep convolutional neural networks and trained classifiers, and yields a structured report combined with useful metadata. Classifiers are trained using our ground-truth visual database that includes over 140,000 images from past earthquake reconnaissance missions to study post-disaster buildings in the field. Here we discuss the novel deployment of the ARIO application within a cloud-based system that we named VISER (Visual Structural Expertise Replicator), a comprehensive cloud-based visual data analytics system with a novel Netflix-inspired technical search capability. Field engineers can exploit this research and our application to search an image repository for visual content. We anticipate that these tools will empower engineers to more rapidly learn new lessons from earthquakes using reconnaissance data.
more »
« less
- Award ID(s):
- 1835473
- PAR ID:
- 10308795
- Date Published:
- Journal Name:
- Proceeding of the 17th World Conference on Earthquake Engineering
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Remote reconnaissance missions are promising solutions for the assessment of earthquake-induced structural damage and cascading geological hazards. Space-borne remote sensing can complement in-field missions when safety and accessibility concerns limit post-earthquake operations on the ground. However, the implementation of remote sensing techniques in post-disaster missions is limited by the lack of methods that combine different techniques and integrate them with field survey data. This paper presents a new approach for rapid post-earthquake building damage assessment and landslide mapping, based on Synthetic Aperture Radar (SAR) data. The proposed texture-based building damage classification approach exploits very high resolution post-earthquake SAR data integrated with building survey data. For landslide mapping, a backscatter intensity-based landslide detection approach, which also includes the separation between landslides and flooded areas, is combined with optical-based manual inventories. The approach was implemented during the joint Structural Extreme Event Reconnaissance, GeoHazards International and Earthquake Engineering Field Investigation Team mission that followed the 2021 Haiti Earthquake and Tropical Cyclone Grace.more » « less
-
The purpose of a routine bridge inspection is to assess the physical and functional condition of a bridge according to a regularly scheduled interval. The Federal Highway Administration (FHWA) requires these inspections to be conducted at least every 2 years. Inspectors use simple tools and visual inspection techniques to determine the conditions of both the elements of the bridge structure and the bridge overall. While in the field, the data is collected in the form of images and notes; after the field work is complete, inspectors need to generate a report based on these data to document their findings. The report generation process includes several tasks: (1) evaluating the condition rating of each bridge element according to FHWA Recording and Coding Guide for Structure Inventory and Appraisal of the Nation’s Bridges; and (2) updating and organizing the bridge inspection images for the report. Both of tasks are time-consuming. This study focuses on assisting with the latter task by developing an artificial intelligence (AI)-based method to rapidly organize bridge inspection images and generate a report. In this paper, an image organization schema based on the FHWA Recording and Coding Guide for the Structure Inventory and Appraisal of the Nation’s Bridges and the Manual for Bridge Element Inspection is described, and several convolutional neural network-based classifiers are trained with real inspection images collected in the field. Additionally, exchangeable image file (EXIF) information is automatically extracted to organize inspection images according to their time stamp. Finally, the Automated Bridge Image Reporting Tool (ABIRT) is described as a browser-based system built on the trained classifiers. Inspectors can directly upload images to this tool and rapidly obtain organized images and associated inspection report with the support of a computer which has an internet connection. The authors provide recommendations to inspectors for gathering future images to make the best use of this tool.more » « less
-
Post-disaster field observations of the built environment are critical for advancing fundamental research that links hazard data to structural performance, cascading community impacts, and the development of effective mitigation strategies. Yet, data collection remain fragmented across hazard types and infrastructure systems due to varying objectives, methodologies, protocols, and standards among investigators and organizations. To address this, a Unified Assessment Framework has been developed for standardized post-disaster hazard and structural assessment data and metadata collection across multiple natural hazards (earthquake, windstorm, coastal events) and infrastructure typologies. The framework encompasses a tiered performance assessment structure with increasing rigor and fidelity levels: Basic Assessment (BA), Load Path Assessment (LPA), and Detailed Component Assessment (DCA). The framework has been implemented as an open-access mobile application, the Structural Extreme Events Reconnaissance (StEER) Network’s StEER Unified App, hosted on Fulcrum data collection platform . Along with unification of data fields, preliminary mapping rules were developed to map out existing hazard-specific damage rating scales (e.g., wind, surge/flooding, rainwater ingress) to the European Macroseismic Scale (EMS-98) compatible unified damage scale, enabling consolidation of global damage ratings into a common data field, facilitating the unification of multiple hazards within a single app. In the mapping of damage ratings, overarching level definitions were retained (e.g., slight, moderate, severe damage) while customizing the specific descriptors to reflect hazard-specific damage mechanisms. Two use cases are presented to demonstrate the application of this framework through the StEER Unified App: a supervised pilot after the 2022 Hurricane Ian, Florida and an unsupervised deployment for the 2023 Turkey earthquake sequence. These deployments illustrate the framework’s flexibility and scalability, validate the feasibility of standardized assessments, and offer insights into how data quality is influenced by assessor pre-deployment training and assessment tier—particularly for complex tasks such as load path evaluation. This work advances the field by providing a scalable, standardized, and hazard-agnostic approach to structural field reconnaissance. The open-access framework and app support real-time deployments and enable integration of legacy datasets into a unified platform—laying the foundation for longitudinal analyses, cross-hazard comparisons, and expanded data reuse in the Natural Hazards Engineering community.more » « less
-
A first foundational assessment is provided for disaster debris reconnaissance that includes identifying tools and techniques for reconnaissance activities, identifying challenges in field reconnaissance, and identifying and developing preliminary guidelines and standards based on advancements from a workshop held in 2022. In this workshop, reconnaissance activities were analyzed in twofold: in relation to post-disaster debris and waste materials and in relation to waste management infrastructure. A four-phase timeline was included to capture the full lifecycle of management activities ranging from collection to temporary storage to final management route: pre-disaster or pre-reconnaissance, post-disaster response (days/weeks), short-term recovery (weeks/months), and long-term recovery (months/years). For successful reconnaissance, objectives of field activities and data collection needs; data types and metrics; and measurement and determination methods need to be identified. A reconnaissance framework, represented using a 3x2x2x4 matrix, is proposed to incorporate data attributes (tools, challenges, guides), reconnaissance attributes (debris, infrastructure; factors, actions), and time attributes (pre-event, response, short-term, long-term). This framework supports field reconnaissance missions and protocols that are longitudinally based and focused on post-disaster waste material and infrastructure metrics that advance sustainable materials management practices. To properly frame and develop effective reconnaissance activities, actions for all data attributes (tools, challenges, guides) are proposed to integrate sustainability and resilience considerations. While existing metrics, tools, methods, standards, and protocols can be adapted for sustainable post-disaster materials management reconnaissance, development of new approaches are needed for addressing unique aspects of disaster debris management.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
