Reconnaissance teams collect perishable data after each disaster to learn about building performance. However, often these large image sets are not adequately curated, nor do they have sufficient metadata (e.g., GPS), hindering any chance to identify images from the same building when collected by different reconnaissance teams. In this study, Siamese convolutional neural networks (S‐CNN) are implemented and repurposed to establish a building search capability suitable for post‐disaster imagery. This method can automatically rank and retrieve corresponding building images in response to a single query using an image. In the demonstration, we utilize real‐world images collected from 174 reinforced‐concrete buildings affected by the 2016 Southern Taiwan and the 2017 Pohang (South Korea) earthquake events. A quantitative performance evaluation is conducted by examining two metrics introduced for this application: Similarity Score (SS) and Similarity Rank (SR).
more »
« less
Learning from Earthquakes Using the Automatic Reconnaissance Image Organizer (ARIO)
In the aftermath of earthquake events, reconnaissance teams are deployed to gather vast amounts of images, moving quickly to capture perishable data to document the performance of infrastructure before they are destroyed. Learning from such data enables engineers to gain new knowledge about the real-world performance of structures. This new knowledge, extracted from such visual data, is critical to mitigate the risks (e.g., damage and loss of life) associated with our built environment in future events. Currently, this learning process is entirely manual, requiring considerable time and expense. Thus, unfortunately, only a tiny portion of these images are shared, curated, and actually utilized. The power of computers and artificial intelligence enables a new approach to organize and catalog such visual data with minimal manual effort. Here we discuss the development and deployment of an organizational system to automate the analysis of large volumes of post-disaster visual data, images. Our application, named the Automated Reconnaissance Image Organizer (ARIO), allows a field engineer to rapidly and automatically categorize their reconnaissance images. ARIO exploits deep convolutional neural networks and trained classifiers, and yields a structured report combined with useful metadata. Classifiers are trained using our ground-truth visual database that includes over 140,000 images from past earthquake reconnaissance missions to study post-disaster buildings in the field. Here we discuss the novel deployment of the ARIO application within a cloud-based system that we named VISER (Visual Structural Expertise Replicator), a comprehensive cloud-based visual data analytics system with a novel Netflix-inspired technical search capability. Field engineers can exploit this research and our application to search an image repository for visual content. We anticipate that these tools will empower engineers to more rapidly learn new lessons from earthquakes using reconnaissance data.
more »
« less
- Award ID(s):
- 1835473
- NSF-PAR ID:
- 10308795
- Date Published:
- Journal Name:
- Proceeding of the 17th World Conference on Earthquake Engineering
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The purpose of a routine bridge inspection is to assess the physical and functional condition of a bridge according to a regularly scheduled interval. The Federal Highway Administration (FHWA) requires these inspections to be conducted at least every 2 years. Inspectors use simple tools and visual inspection techniques to determine the conditions of both the elements of the bridge structure and the bridge overall. While in the field, the data is collected in the form of images and notes; after the field work is complete, inspectors need to generate a report based on these data to document their findings. The report generation process includes several tasks: (1) evaluating the condition rating of each bridge element according to FHWA Recording and Coding Guide for Structure Inventory and Appraisal of the Nation’s Bridges; and (2) updating and organizing the bridge inspection images for the report. Both of tasks are time-consuming. This study focuses on assisting with the latter task by developing an artificial intelligence (AI)-based method to rapidly organize bridge inspection images and generate a report. In this paper, an image organization schema based on the FHWA Recording and Coding Guide for the Structure Inventory and Appraisal of the Nation’s Bridges and the Manual for Bridge Element Inspection is described, and several convolutional neural network-based classifiers are trained with real inspection images collected in the field. Additionally, exchangeable image file (EXIF) information is automatically extracted to organize inspection images according to their time stamp. Finally, the Automated Bridge Image Reporting Tool (ABIRT) is described as a browser-based system built on the trained classifiers. Inspectors can directly upload images to this tool and rapidly obtain organized images and associated inspection report with the support of a computer which has an internet connection. The authors provide recommendations to inspectors for gathering future images to make the best use of this tool.more » « less
-
Puerto Rico is exposed to multiple hazards including hurricanes, earthquakes, and floods. The Resilient Infrastructure and Sustainability Education Undergraduate Program (RISE-UP) at the University of Puerto Rico aims to introduce students to interdisciplinary problem-solving related to real challenges, especially those associated with the occurrence of natural disasters. The objective of this work is to share our experience with experiential learning related to structural engineering. The lessons learned from this experience, from the student ́s perspective, could encourage faculty members to develop similar undertakings in their programs and students to participate when opportunities arise. During the 2019 fall semester, we enrolled in a course which covered the relationship between design and natural disasters, with an emphasis on rapid response to recover during the aftermath. The course combined lectures and in-class exercises on basic structural analysis, classifications of structures and the use of the FEMA Rapid Visual Screening (P-154) form. This was complemented with field visits of structures affected by Hurricane Maria where we developed several case studies. From December of 2019 to February 2020, Puerto Rico suffered an earthquake swarm reaching magnitudes as high as 6.4, which caused structural damages throughout the South West of the island. Following these events, we were able to use the training acquired during our course in a real-life, post-disaster situation. At the University, we participated in visual inspection brigades, where we aided professional engineers and faculty members in data collection and categorizing building damages. Our involvement helped streamline efforts as we provided additional support in report writing and organization of the data collected using GIS and other tools. The results of the visual inspections indicated that in many cases pre- existing conditions were aggravated by the earthquakes. Furthermore, we also witnessed firsthand the complexities of assessing infrastructure damage during and following high seismic activity. This experience enhanced our awareness of the significance of our profession in ensuring the safety of others both immediately after an earthquake and in the face of future disasters.more » « less
-
Surface meltwater generated on ice shelves fringing the Antarctic Ice Sheet can drive ice-shelf collapse, leading to ice sheet mass loss and contributing to global sea level rise. A quantitative assessment of supraglacial lake evolution is required to understand the influence of Antarctic surface meltwater on ice-sheet and ice-shelf stability. Cloud computing platforms have made the required remote sensing analysis computationally trivial, yet a careful evaluation of image processing techniques for pan-Antarctic lake mapping has yet to be performed. This work paves the way for automating lake identification at a continental scale throughout the satellite observational record via a thorough methodological analysis. We deploy a suite of different trained supervised classifiers to map and quantify supraglacial lake areas from multispectral Landsat-8 scenes, using training data generated via manual interpretation of the results from k-means clustering. Best results are obtained using training datasets that comprise spectrally diverse unsupervised clusters from multiple regions and that include rock and cloud shadow classes. We successfully apply our trained supervised classifiers across two ice shelves with different supraglacial lake characteristics above a threshold sun elevation of 20°, achieving classification accuracies of over 90% when compared to manually generated validation datasets. The application of our trained classifiers produces a seasonal pattern of lake evolution. Cloud shadowed areas hinder large-scale application of our classifiers, as in previous work. Our results show that caution is required before deploying ‘off the shelf’ algorithms for lake mapping in Antarctica, and suggest that careful scrutiny of training data and desired output classes is essential for accurate results. Our supervised classification technique provides an alternative and independent method of lake identification to inform the development of a continent-wide supraglacial lake mapping product.more » « less
-
Earthquake geotechnical engineering is an experience-driven discipline. Field observations are particularly important, because it is difficult to replicate in the laboratory the characteristics and response of in situ soil deposits. Much of the data generated by a major earthquake is perishable, so it is critical that it is collected soon after an event occurs. Detailed mapping and surveying of damaged and undamaged areas provides the data for the well-documented case histories that drive the development of many of the earthquake geotechnical engineering design procedures. New technologies are being employed to capture earthquake-induced ground deformation, including Light Detection and Ranging, Structure-from-Motion, and Unmanned Aerial Vehicles. Post-earthquake reconnaissance has moved beyond taking photographs and field notes to taking advantage of technologies that can capture ground and structure deformations more completely and accurately. Moreover, electronic data enables effective sharing and archiving of the measurements. Unanticipated observations from major events often catalyze new research directions. Important advancements are possible through post-event research if their effects are captured and shared effectively. An overview of some of recent integrated technology deployments and their role in advancing knowledge are presented.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
