An official website of the United States government
In the wake of Hurricane Ian's catastrophic landfall in September 2022 as a Category 4 storm, the devastation spread from western Cuba to central Florida, generating an overwhelming volume of debris and underscoring the urgent need for robust debris management strategies. In response, our SUMMEER team, through the RAPID project, embarked on collecting comprehensive data on the hurricane's impacts, illegal dumping, and the ensuing debris management efforts in severely affected areas, including North Port, Venice, and Arcadia. Utilizing advanced technologies such as UAVs, 3-D LiDAR scanners, and multispectral and thermal cameras, the RAPID project primarily focused on quantifying uncollected and illegally dumped debris along publicly owned roadways. Conducted over three distinct periods—November 16-18, December 5-7, and January 9-11—the field assessment aimed to provide a comprehensive understanding of the evolving conditions in the aftermath of Hurricane Ian. These specific timeframes enabled us to document not only the immediate aftermath but also the changes over time, thereby facilitating a thorough analysis of the disaster's impact and the effectiveness of the debris management efforts undertaken. This project's comprehensive approach has enabled a detailed analysis of the volume and trajectory of debris, highlighting the dynamics of debris collection and disposal processes. By creating a detailed empirical database, including aerial maps, 3D LiDAR maps, and multispectral and thermal images, this project provides essential data that supports effective cleanup and recovery operations, thereby contributing to more efficient future disaster preparedness and response strategies.
more »
« less
This content will become publicly available on October 1, 2026
Debris segmentation using post‐hurricane aerial imagery
Timely and accurate detection of hurricane debris is critical for effective disaster response and community resilience. While post‐disaster aerial imagery is readily available, robust debris segmentation solutions applicable across multiple disaster regions remain limited. Since environmental and imaging conditions vary by region and alter debris's visual signatures, developing a debris segmentation model suitable for different hazard scenarios is challenging, especially given the scarcity of training data. This study addresses these challenges by relying on vast pretrained knowledge from the foundation vision model Contrastive Language‐Image Pre‐Training (CLIP) and fine‐tuning a lightweight decoder on labeled debris imagery, thereby achieving robust performance in segmenting non‐vegetative debris within complex urban backgrounds with a relatively small, high‐quality dataset. To enable fine‐tuning, this work introduces an open‐source dataset comprising approximately 1200 manually annotated aerial RGB images of structural debris from Hurricanes Ian, Ida, and Ike. To mitigate human biases and enhance data quality, labels from multiple annotators are strategically aggregated into consensus annotations. The resulting fine‐tuned model, named CLIPSeg‐debris, achieves a Dice score of 0.86 on data from Hurricane Ida—a disaster event entirely excluded during training—outperforming baseline models. This approach thereby enables broader adoption across multiple hazard scenarios and effectively adapts to various backgrounds, sensor resolutions, and debris conditions.
more »
« less
- Award ID(s):
- 2107313
- PAR ID:
- 10668825
- Publisher / Repository:
- Wiley Online Library
- Date Published:
- Journal Name:
- Computer-Aided Civil and Infrastructure Engineering
- Volume:
- 40
- Issue:
- 25
- ISSN:
- 1093-9687
- Page Range / eLocation ID:
- 4116 to 4131
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Reliable assessment of building damage is essential for effective disaster management. Synthetic Aperture Radar (SAR) has become a valuable tool for damage detection, as it operates independently of the daylight and weather conditions. However, the limited availability of high-resolution pre-disaster SAR data remains a major obstacle to accurate damage evaluation, constraining the applicability of traditional change-detection approaches. This study proposes a comprehensive framework that leverages generated SAR data alongside optical imagery for building damage detection and further examines the influence of elevation data quality on SAR synthesis and model performance. The method integrates SAR image synthesis from a Digital Surface Model (DSM) and land cover inputs with a multimodal deep learning architecture capable of jointly localizing buildings and classifying damage levels. Two data modality scenarios are evaluated: a change-detection setting using pre-disaster authentic SAR and another using GAN-generated SAR, both combined with post-disaster SAR imagery for building damage assessment. Experimental results demonstrate that GAN-generated SAR can effectively substitute for authentic SAR in multimodal damage mapping. Models using generated pre-disaster SAR achieved comparable or superior performance to those using authentic SAR, with F1 scores of 0.730, 0.442, and 0.790 for the survived, moderate, and destroyed classes, respectively. Ablation studies further reveal that the model relies more heavily on land cover segmentation than on fine elevation details, suggesting that coarse-resolution DSMs (30 m) are sufficient as auxiliary input. Incorporating additional training regions further improved generalization and inter-class balance, confirming that high-quality generated SAR can serve as a viable alternative especially in the absence of authentic SAR, for scalable, post-disaster building damage assessment.more » « less
-
The Segment Anything Model (SAM) is a large-scale foundation model that has revolutionized segmentation methodology. Despite its impressive generalization ability, the segmentation accuracy of SAM on images with intricate structures is often unsatisfactory. Recent works have proposed lightweight fine-tuning using high-quality annotated data to improve accuracy on such images. However, here we provide extensive empirical evidence that this strategy leads to forgetting how to "segment anything": these models lose the original generalization abilities of SAM, in the sense that they perform worse for segmentation tasks not represented in the annotated fine-tuning set. To improve performance without forgetting, we introduce a novel framework that combines high-quality annotated data with a large unlabeled dataset. The framework relies on two methodological innovations. First, we quantify the uncertainty in the SAM pseudo labels associated with the unlabeled data and leverage it to perform uncertainty-aware fine-tuning. Second, we encode the type of segmentation task associated with each training example using a task prompt to reduce ambiguity. We evaluated the proposed Segmentation with Uncertainty Model (SUM) on a diverse test set consisting of 14 public benchmarks, where it achieves state-of-the-art results. Notably, our method consistently surpasses SAM by 3-6 points in mean IoU and 4-7 in mean boundary IoU across point-prompt interactive segmentation rounds.more » « less
-
As solar photovoltaic (PV) has emerged as a dominant player in the energy market, there has been an exponential surge in solar deployment and investment within this sector. With the rapid growth of solar energy adoption, accurate and efficient detection of PV panels has become crucial for effective solar energy mapping and planning. This paper presents the application of the Mask2Former model for segmenting PV panels from a diverse, multi-resolution dataset of satellite and aerial imagery. Our primary objective is to harness Mask2Former’s deep learning capabilities to achieve precise segmentation of PV panels in real-world scenarios. We fine-tune the pre-existing Mask2Former model on a carefully curated multi-resolution dataset and a crowdsourced dataset of satellite and aerial images, showcasing its superiority over other deep learning models like U-Net and DeepLabv3+. Most notably, Mask2Former establishes a new state-of-the-art in semantic segmentation by achieving over 95% IoU scores. Our research contributes significantly to the advancement solar energy mapping and sets a benchmark for future studies in this field.more » « less
-
Multimodal fusion networks play a pivotal role in leveraging diverse sources of information for enhanced machine learning applications in aerial imagery. However, current approaches often suffer from a bias towards certain modalities, diminishing the potential benefits of multimodal data. This paper addresses this issue by proposing a novel modality utilization-based training method for multimodal fusion networks. The method aims to guide the network’s utilization on its input modalities, ensuring a balanced integration of complementary information streams, effectively mitigating the overutilization of dominant modalities. The method is validated on multimodal aerial imagery classification and image segmentation tasks, effectively maintaining modality utilization within ±10% of the user-defined target utilization and demonstrating the versatility and efficacy of the proposed method across various applications. Furthermore, the study explores the robustness of the fusion networks against noise in input modalities, a crucial aspect in real-world scenarios. The method showcases better noise robustness by maintaining performance amidst environmental changes affecting different aerial imagery sensing modalities. The network trained with 75.0% EO utilization achieves significantly better accuracy (81.4%) in noisy conditions (noise variance = 0.12) compared to traditional training methods with 99.59% EO utilization (73.7%). Additionally, it maintains an average accuracy of 85.0% across different noise levels, outperforming the traditional method’s average accuracy of 81.9%. Overall, the proposed approach presents a significant step towards harnessing the full potential of multimodal data fusion in diverse machine learning applications such as robotics, healthcare, satellite imagery, and defense applications.more » « less
