An official website of the United States government
Vision-based structural health monitoring (SHM) has become an important approach to recognize and evaluate structural damage after natural disasters. Deep convolutional neural networks (CNNs) have recently attained a breakthrough in computer vision field, in particular for image classification task. In this article, we adopted deep residual neural network (ResNet) whose residual representations and shortcut connections mechanism has gained significant performance in various computer vision tasks. In addition, we applied transfer learning due to a relatively small number of training images. To test our approach, we used the dataset from the 2018 PEER Hub ImageNet Challenge distributed by Pacific Earthquake Engineering Research Center. This challenge proposed eight structural damage detection tasks: scene classification, damage check, spalling condition, material type, collapse check, component type, damage level and damage type which can be categorized as binary and multi-class (3 or 4 classes) classification problems. Our experiments with eight different tasks showed that reliable classification can be obtained for some tasks. Corresponding above eight tasks, classification accuracy varied from 63.1% to 99.4%. Our approach has attained third place for overall tasks in this challenge. Through the individual observation of training dataset, it is found that there are a large number of confusing images. Therefore, it is believed that the accuracy will be improved after making a precise training data.
more »
« less
Engineering deep learning methods on automatic detection of damage in infrastructure due to extreme events
This paper presents a few comprehensive experimental studies for automated Structural Damage Detection (SDD) in extreme events using deep learning methods for processing 2D images. In the first study, a 152-layer Residual network (ResNet) is utilized to classify multiple classes in eight SDD tasks, which include identification of scene levels, damage levels, and material types. The proposed ResNet achieved high accuracy for each task while the positions of the damage are not identifiable. In the second study, the existing ResNet and a segmentation network (U-Net) are combined into a new pipeline, cascaded networks, for categorizing and locating structural damage. The results show that the accuracy of damage detection is significantly improved compared to only using a segmentation network. In the third and fourth studies, end-to-end networks are developed and tested as a new solution to directly detect cracks and spalling in the image collections of recent large earthquakes. One of the proposed networks can achieve an accuracy above 67 .6% for all tested images at various scales and resolutions, and shows its robustness for these human-free detection tasks. As a preliminary field study, we applied the proposed method to detect damage in a concrete structure that was tested to study its progressive collapse performance. The experiments indicate that these solutions for automatic detection of structural damage using deep learning methods are feasible and promising. The training datasets and codes will be made available for the public upon the publication of this paper.
more »
« less
- Award ID(s):
- 2036193
- PAR ID:
- 10388272
- Publisher / Repository:
- SAGE Publications
- Date Published:
- Journal Name:
- Structural Health Monitoring
- Volume:
- 22
- Issue:
- 1
- ISSN:
- 1475-9217
- Format(s):
- Medium: X Size: p. 338-352
- Size(s):
- p. 338-352
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract. In this paper, two different convolutional neural networks (CNNs) are applied on images for automated structural damage detection (SDD) in earthquake damaged structures and cracking localization (e.g., detection of cracks, their widths and distributions) at various scales, such as pixel level, object level, and structural level. The proposed method has two main steps: 1) diagnosis, and 2) localization of cracking or other damage. At first a residual CNN with transfer learning is employed to classify the damage in the structures and structural components. This step performs damage detection using two public datasets. The second step uses another CNN with U-Net structure to locate the cracking on low resolution images. The implementations using public and self-collected datasets show promising performance for a problem that had remained a challenge in the structure engineering field for a long time and indicate that the proposed approach can perform detection and localization of structural damage with an acceptable accuracy.more » « less
-
Detecting Cracks and Spalling Automatically in Extreme Events by End-to-end Deep Learning FrameworksIn this paper, we develop and implement end-to-end deep learning approaches to automatically detect two important types of structural failures, cracks and spalling, of buildings and bridges in extreme events such as major earthquakes. A total of 2,229 images were annotated, and are used to train and validate three newly developed Mask Regional Convolutional Neural Networks (Mask R-CNNs). In addition, three sets of public images for different disasters were used to test the accuracy of these models. For detecting and marking these two types of structural failures, one of proposed methods can achieve an accuracy of 67.6% and 81.1%, respectively, on low- and high-resolution images collected from field investigations. The results demonstrate that it is feasible to use the proposed end-to-end method for automatically locating and segmenting the damage using 2D images which can help human experts in cases of disasters.more » « less
-
As one of the popular deep learning methods, deep convolutional neural networks (DCNNs) have been widely adopted in segmentation tasks and have received positive feedback. However, in segmentation tasks, DCNN-based frameworks are known for their incompetence in dealing with global relations within imaging features. Although several techniques have been proposed to enhance the global reasoning of DCNN, these models are either not able to gain satisfying performances compared with traditional fully-convolutional structures or not capable of utilizing the basic advantages of CNN-based networks (namely the ability of local reasoning). In this study, compared with current attempts to combine FCNs and global reasoning methods, we fully extracted the ability of self-attention by designing a novel attention mechanism for 3D computation and proposed a new segmentation framework (named 3DTU) for three-dimensional medical image segmentation tasks. This new framework processes images in an end-to-end manner and executes 3D computation on both the encoder side (which contains a 3D transformer) and the decoder side (which is based on a 3D DCNN). We tested our framework on two independent datasets that consist of 3D MRI and CT images. Experimental results clearly demonstrate that our method outperforms several state-of-the-art segmentation methods in various metrics.more » « less
-
Digital pathology has played a key role in replacing glass slides with digital images, enhancing various pathology workflows. Whole slide images are digitized pathological images improving the capabilities of digital pathology and contributing to the overall turnaround time for diagnoses. The digitized images have been successfully integrated with artificial intelligence algorithms assisting pathologists in many tasks, but there are still demands to develop a new algorithm for a better diagnosis process. In this paper, we propose a new deep convolutional neural network model integrating a feature pyramid network with a self-attention mechanism in three pathways: encoder, decoder, and self-attention nested for providing accurate tumor region segmentation on whole slide images. The encoder pathway adopts ResNet50 architecture for the bottom-up network. The decoder pathway adopts the feature pyramid network for the top-down network. The self-attention nested pathway forms the attention map represented by the distribution of attention scores focusing on localizing tumor regions and avoiding irrelevant information. The results of our experiment show that the proposed model outperforms the state-of-the-art deep convolutional neural network models in terms of tumor and stromal region segmentation. Moreover, various encoder networks were equipped with the proposed model and compared with each other. The results indicate that the ResNet series using the proposed model outperforms other encoder networks.more » « less
