More Like this

1. Study of AI Object Detection: Patterns on Animals with YOLO and Adversarial Patches

   Hopson, Aniya; Boonthum-Denecke, Chutima; Mkpong-Ruffin, Idongesit (March 2025, The 2025 ADMI Symposium.)

   In this paper, we document our findings from previous research and literature related to adversarial examples and object detection. Artificial Intelligence (AI) is an increasingly powerful tool in various fields, particularly in image classification and object detection. As AI becomes more advanced, new methods to deceive machine learning models, such as adversarial patches, have emerged. These subtle modifications to images can cause AI models to misclassify objects, posing a significant challenge to their reliability. This research builds upon our earlier work by investigating how small patches affect object detection on YOLOv8. Last year, we explored patterns within images and their impact on model accuracy. This study extends that work by testing how adversarial patches, particularly those targeting animal patterns, affect YOLOv8's ability to accurately detect objects. We also explore how untrained patterns influence the model’s performance, aiming to identify weaknesses and improve the robustness of object detection systems. 

   more » « less
2. Application of Image Processing and Big Data Science for Flood Label Detection

   Pally, R; Samadi, S. (April 2021, European Geosciences Union)

   null (Ed.)

   Due to the importance of object detection in video analysis and image annotation, it is widely utilized in a number of computer vision tasks such as face recognition, autonomous vehicles, activity recognition, tracking objects and identity verification. Object detection does not only involve classification and identification of objects within images, but also involves localizing and tracing the objects by creating bounding boxes around the objects and labelling them with their respective prediction scores. Here, we leverage and discuss how connected vision systems can be used to embed cameras, image processing, Edge Artificial Intelligence (AI), and data connectivity capabilities for flood label detection. We favored the engineering definition of label detection that a label is a sequence of discrete measurable observations obtained using a capturing device such as web cameras, smart phone, etc. We built a Big Data service of around 1000 images (image annotation service) including the image geolocation information from various flooding events in the Carolinas (USA) with a total of eight different object categories. Our developed platform has several smart AI tools and task configurations that can detect objects’ edges or contours which can be manually adjusted with a threshold setting so as to best segment the image. The tool has the ability to train the dataset and predict the labels for large scale datasets which can be used as an object detector to drastically reduce the amount of time spent per object particularly for real-time image-based flood forecasting. This research is funded by the US National Science Foundation (NSF). 

   more » « less
3. Segment and Recover: Defending Object Detectors Against Adversarial Patch Attacks

   https://doi.org/10.3390/jimaging11090316  

   Gu, Haotian; Jafarnejadsani, Hamidreza (September 2025, Journal of Imaging)

   Object detection is used to automatically identify and locate specific objects within images or videos for applications like autonomous driving, security surveillance, and medical imaging. Protecting object detection models against adversarial attacks, particularly malicious patches, is crucial to ensure reliable and safe performance in safety-critical applications, where misdetections can lead to severe consequences. Existing defenses against patch attacks are primarily designed for stationary scenes and struggle against adversarial image patches that vary in scale, position, and orientation in dynamic environments.In this paper, we introduce SAR, a patch-agnostic defense scheme based on image preprocessing that does not require additional model training. By integration of the patch-agnostic detection frontend with an additional broken pixel restoration backend, Segment and Recover (SAR) is developed for the large-mask-covered object-hiding attack. Our approach breaks the limitation of the patch scale, shape, and location, accurately localizes the adversarial patch on the frontend, and restores the broken pixel on the backend. Our evaluations of the clean performance demonstrate that SAR is compatible with a variety of pretrained object detectors. Moreover, SAR exhibits notable resilience improvements over state-of-the-art methods evaluated in this paper. Our comprehensive evaluation studies involve diverse patch types, such as localized-noise, printable, visible, and adaptive adversarial patches. 

   more » « less
4. Physical Adversarial Examples for Object Detectors

   Eykholt, Kevin; Evtimov, Ivan; Fernandes, Earlence; Li, Bo; Rahmati, Amir; Tramer, Florian; Prakash, Atul; Kohno, Tadayoshi; Song, Dawn (August 2018, 12th USENIX Workshop on Offensive Technologies (WOOT 18))

   Deep neural networks (DNNs) are vulnerable to adversarial examples—maliciously crafted inputs that cause DNNs to make incorrect predictions. Recent work has shown that these attacks generalize to the physical domain, to create perturbations on physical objects that fool image classifiers under a variety of real-world conditions. Such attacks pose a risk to deep learning models used in safety-critical cyber-physical systems. In this work, we extend physical attacks to more challenging object detection models, a broader class of deep learning algorithms widely used to detect and label multiple objects within a scene. Improving upon a previous physical attack on image classifiers, we create perturbed physical objects that are either ignored or mislabeled by object detection models. We implement a Disappearance Attack, in which we cause a Stop sign to “disappear” according to the detector—either by covering the sign with an adversarial Stop sign poster, or by adding adversarial stickers onto the sign. In a video recorded in a controlled lab environment, the state-of-the-art YOLO v2 detector failed to recognize these adversarial Stop signs in over 85% of the video frames. In an outdoor experiment, YOLO was fooled by the poster and sticker attacks in 72.5% and 63.5% of the video frames respectively. We also use Faster R-CNN, a different object detection model, to demonstrate the transferability of our adversarial perturbations. The created poster perturbation is able to fool Faster R-CNN in 85.9% of the video frames in a controlled lab environment, and 40.2% of the video frames in an outdoor environment. Finally, we present preliminary results with a new Creation Attack, wherein innocuous physical stickers fool a model into detecting nonexistent objects. 

   more » « less
5. Reducing The Effects Of Adversarial Patches Using Ai-Based Inpainting

   Hill, A; Boonthum-Denecke, C; Mkpong-Ruffin, I (March 2025, The 2025 ADMI Symposium)

   Adversarial patches represent a critical vulnerability in computer vision systems, as they are specifically created in order to deceive object detection algorithms, which can compromise their reliability in real-world applications. This research investigates the impact of adversarial patches on object detection models and proposes a novel mitigation strategy to address this challenge. The study's primary objective was to design a comprehensive framework that integrates adversarial patch detection with image restoration. To achieve this, a YOLOv8-based detection framework was employed, trained on a specialized dataset of adversarial patches to ensure high detection accuracy. Upon identification of patches, advanced inpainting techniques utilizing AI models were applied to mask and fill the affected areas, restoring the image with expected content. The methodology combines the precision of object detection with the generative capabilities of modern inpainting algorithms, ensuring minimal disruption to the visual integrity of the image. This work contributes to the field of adversarial robustness by providing a comprehensive approach that integrates detection, masking, and content restoration. The results highlight the potential of AI-driven solutions to enhance the resilience of object detection systems against adversarial attacks, paving the way for safer deployment of vision-based technologies in critical domains such as autonomous vehicles, surveillance, and medical imaging. 

   more » « less