An official website of the United States government
Remote sensing datasets usually have a wide range of spatial and spectral resolutions. They provide unique advantages in surveillance systems, and many government organizations use remote sensing multispectral imagery to monitor security-critical infrastructures or targets. Artificial Intelligence (AI) has advanced rapidly in recent years and has been widely applied to remote image analysis, achieving state-of-the-art (SOTA) performance. However, AI models are vulnerable and can be easily deceived or poisoned. A malicious user may poison an AI model by creating a stealthy backdoor. A backdoored AI model performs well on clean data but behaves abnormally when a planted trigger appears in the data. Backdoor attacks have been extensively studied in machine learning-based computer vision applications with natural images. However, much less research has been conducted on remote sensing imagery, which typically consists of many more bands in addition to the red, green, and blue bands found in natural images. In this paper, we first extensively studied a popular backdoor attack, BadNets, applied to a remote sensing dataset, where the trigger was planted in all of the bands in the data. Our results showed that SOTA defense mechanisms, including Neural Cleanse, TABOR, Activation Clustering, Fine-Pruning, GangSweep, Strip, DeepInspect, and Pixel Backdoor, had difficulties detecting and mitigating the backdoor attack. We then proposed an explainable AI-guided backdoor attack specifically for remote sensing imagery by placing triggers in the image sub-bands. Our proposed attack model even poses stronger challenges to these SOTA defense mechanisms, and no method was able to defend it. These results send an alarming message about the catastrophic effects the backdoor attacks may have on satellite imagery.
more »
« less
This content will become publicly available on July 15, 2026
MO-SAM: Testing the reliability and limits of mine feature delineation using Segment Anything Model to democratize mine observation and research
The purpose of this paper is to leverage the growth of AI-enabled tools to support the democratization of mine observation (MO) research. Mining is essential to meet projected demand for renewable energy technologies crucial to global climate mitigation objectives, but all mining activities pose local and regional challenges to environmental sustainability. Such challenges can be mitigated by good governance, but unequal access among stakeholders to accurately interpreted satellite imagery can weaken good governance. Using readily available software—QGIS, and Segment Anything Model (SAM)—this paper develops and tests the reliability of MO-SAM, a new method to identify and delineate features within the spatially-explicit mine extent at a high level of detail. It focuses on dry tailings, waste dumps, and stockpiles in above-ground mining areas. While we intend for MO-SAM to be used generally, this study tested it on mining areas for energy-critical materials: lithium (Li), cobalt (Co), rare earth elements (REE), and platinum group elements (PGE), selected for their importance to the global transition to renewable energy. MO-SAM demonstrates generalizability through prompt engineering, but performance limitations were observed in imagery with complex mining landscape scenarios, including spatial variations in image morphology and boundary sharpness. Our analysis provides data-driven insights to support advances in the use of MO-SAM for analyzing and monitoring large-scale mining activities with greater speed than methods that rely on manual delineation, and with greater precision than practices that focus primarily on changes in the spatially-explicit mine extent. It also provides insights into the importance of multidisciplinary human expertise in designing processes for and assessing the accuracy of AI-assisted remote sensing image segmentation as well as in evaluating the significance of the land use and land cover changes identified. This has widespread potential to advance the multidisciplinary application of AI for scientific and public interest, particularly in research on global scale human-environment interactions such as industrial mining activities. This is methodologically significant because the potential and limitations of using large pre-trained image segmentation models such as SAM for analyzing remote sensing data is an emergent and underexplored issue. The results can help advance the utilization of large pre-trained segmentation models for remote sensing imagery analysis to support sustainability research and policy.
more »
« less
- Award ID(s):
- 2039857
- PAR ID:
- 10621691
- Editor(s):
- Bale, Catherine
- Publisher / Repository:
- PLOS
- Date Published:
- Journal Name:
- PLOS Sustainability and Transformation
- Volume:
- 4
- Issue:
- 7
- ISSN:
- 2767-3197
- Page Range / eLocation ID:
- e0000182
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The Segment Anything Model (SAM) is a recently proposed prompt-based segmentation model in a generic zero-shot segmentation approach. With the zero-shot segmentation capacity, SAM achieved impressive flexibility and precision on various segmentation tasks. However, the current pipeline requires manual prompts during the inference stage, which is still resource intensive for biomedical image segmentation. In this paper, instead of using prompts during the inference stage, we introduce a pipeline that utilizes the SAM, called all-in-SAM, through the entire AI development workflow (from annotation generation to model finetuning) without requiring manual prompts during the inference stage. Specifically, SAM is first employed to generate pixel-level annotations from weak prompts (e.g., points, bounding box). Then, the pixel-level annotations are used to finetune the SAM segmentation model rather than training from scratch. Our experimental results reveal two key findings: 1) the proposed pipeline surpasses the state-of-the-art (SOTA) methods in a nuclei segmentation task on the public Monuseg dataset, and 2) the utilization of weak and few annotations for SAM finetuning achieves competitive performance compared to using strong pixel-wise annotated data.more » « less
-
Geotechnical characterization of rock masses in underground mines often involves physical measurements in supported excavations. However, unsupported stopes and drifts prevent safe access for mapping by geotechnical personnel. The advent of inexpensive, open platform unmanned aerial vehicles (UAVs) allows geotechnical personnel to characterize hazardous rock masses by utilizing traditional photogrammetric and FLIR (forward looking infrared) imagery techniques. The photogrammetric imagery can be used to capture geological structural data from the rock mass for kinematic and numerical analyses, as well as for generating geological models. In particular, the FLIR imagery has the potential to assist in identifying areas of loose rock, which typically goes unnoticed until it becomes a hazard. This paper summarizes the results of a study involving UAV flights underground at the Barrick Golden Sunlight Mine, the generation of 3D models from UAV-captured imagery, and the identification of geological data from photogrammetry models. Results confirm that the combination of off-the-shelf technologies used in this study can be successfully employed as a geotechnical tool in the underground mining environment.more » « less
-
Weathering and transport of potentially acid generating material (PAGM) at abandoned mines can degrade downstream environments and contaminate water resources. Monitoring the thousands of abandoned mine lands (AMLs) for exposed PAGM using field surveys is time intensive. Here, we explore the use of Remotely Piloted Aerial Systems (RPASs) as a complementary remote sensing platform to map the spatial and temporal changes of PAGM across a mine waste rock pile on an AML. We focus on testing the ability of established supervised and unsupervised classification algorithms to map PAGM on imagery with very high spatial resolution, but low spectral sampling. At the Perry Canyon, NV, USA AML, we carried out six flights over a 29-month period, using a RPAS equipped with a 5-band multispectral sensor measuring in the visible to near infrared (400–1000 nm). We built six different 3 cm resolution orthorectified reflectance maps, and our tests using supervised and unsupervised classifications revealed benefits to each approach. Supervised classification schemes allowed accurate mapping of classes that lacked published spectral libraries, such as acid mine drainage (AMD) and efflorescent mineral salts (EMS). The unsupervised method produced similar maps of PAGM, as compared to supervised schemes, but with little user input. Our classified multi-temporal maps, validated with multiple field and lab-based methods, revealed persistent and slowly growing ‘hotspots’ of jarosite on the mine waste rock pile, whereas EMS exhibit more rapid fluctuations in extent. The mapping methods we detail for a RPAS carrying a broadband multispectral sensor can be applied extensively to AMLs. Our methods show promise to increase the spatial and temporal coverage of accurate maps critical for environmental monitoring and reclamation efforts over AMLs.more » « less
-
VishnuRadhan, Renjith (Ed.)Satellite-based remote sensing and uncrewed aerial imagery play increasingly important roles in the mapping of wildlife populations and wildlife habitat, but the availability of imagery has been limited in remote areas. At the same time, ecotourism is a rapidly growing industry and can yield a vast catalog of photographs that could be harnessed for monitoring purposes, but the inherently ad-hoc and unstructured nature of these images make them difficult to use. To help address this, a subfield of computer vision known as phototourism has been developed to leverage a diverse collection of unstructured photographs to reconstruct a georeferenced three-dimensional scene capturing the environment at that location. Here we demonstrate the use of phototourism in an application involving Antarctic penguins, sentinel species whose dynamics are closely tracked as a measure of ecosystem functioning, and introduce a semi-automated pipeline for aligning and registering ground photographs using a digital elevation model (DEM) and satellite imagery. We employ the Segment Anything Model (SAM) for the interactive identification and segmentation of penguin colonies in these photographs. By creating a textured 3D mesh from the DEM and satellite imagery, we estimate camera poses to align ground photographs with the mesh and register the segmented penguin colony area to the mesh, achieving a detailed representation of the colony. Our approach has demonstrated promising performance, though challenges persist due to variations in image quality and the dynamic nature of natural landscapes. Nevertheless, our method offers a straightforward and effective tool for the georegistration of ad-hoc photographs in natural landscapes, with additional applications such as monitoring glacial retreat.more » « less
