More Like this

1. UAVs Control Using 3D Hand Keypoint Gestures

   Van Khoi Nguyen and R. Alba-Flores (February 2022, SoutheastCon 2022)

   IEEE (Ed.)

   Over past few years, unmanned aircraft vehicles (UAVs) have been becoming more and more popular for various purposes such as surveillance, automated industry, robotics, vehicle guidance, traffic monitoring and control system. It is very important to have multiple methods of UAVs controlling to fit in UAVs usages. The goal of this work was to develop a new technique to control an UAV by using different hand gestures. To achieve this, a hand keypoint detection algorithm was used to detect 21 keypoints in the hand. Then this keypoints were used as the input to an intelligent system based on Convolutional Neural Networks (CNN) that was able to classify the hand gestures. To capture the hand gestures, the video camera of the UAV was used. A database containing 2400 hand images was created and used to train the CNN. The database contained 8 different hand gestures that were selected to send specific motion commands to the UAV. The accuracy of the CNN to classify the hand gestures was 93%. To test the capabilities of our intelligent control system, a small UAV, the DJI Ryze Tello drone, was used. The experimental results demonstrated that the DJI Tello drone was able to be successfully controlled by hand gestures in real time. 

   more » « less
2. Long-Range Thermal Target Detection in Data-Limited Settings Using Restricted Receptive Fields

   https://doi.org/10.3390/s23187806  

   Poster, Domenick; Hu, Shuowen; Nasrabadi, Nasser M. (September 2023, Sensors)

   Long-range target detection in thermal infrared imagery is a challenging research problem due to the low resolution and limited detail captured by thermal sensors. The limited size and variability in thermal image datasets for small target detection is also a major constraint for the development of accurate and robust detection algorithms. To address both the sensor and data constraints, we propose a novel convolutional neural network (CNN) feature extraction architecture designed for small object detection in data-limited settings. More specifically, we focus on long-range ground-based thermal vehicle detection, but also show the effectiveness of the proposed algorithm on drone and satellite aerial imagery. The design of the proposed architecture is inspired by an analysis of popular object detectors as well as custom-designed networks. We find that restricted receptive fields (rather than more globalized features, as is the trend), along with less down sampling of feature maps and attenuated processing of fine-grained features, lead to greatly improved detection rates while mitigating the model’s capacity to overfit on small or poorly varied datasets. Our approach achieves state-of-the-art results on the Defense Systems Information Analysis Center (DSIAC) automated target recognition (ATR) and the Tiny Object Detection in Aerial Images (AI-TOD) datasets. 

   more » « less
3. Application Framework and Optimal Features for UAV-Based Earthquake-Induced Structural Displacement Monitoring

   https://doi.org/10.3390/a18020066  

   Ji, Ruipu; Sorosh, Shokrullah; Lo, Eric; Norton, Tanner J; Driscoll, John W; Kuester, Falko; Barbosa, Andre R; Simpson, Barbara G; Hutchinson, Tara C (February 2025, Algorithms)

   Unmanned aerial vehicle (UAV) vision-based sensing has become an emerging technology for structural health monitoring (SHM) and post-disaster damage assessment of civil infrastructure. This article proposes a framework for monitoring structural displacement under earthquakes by reprojecting image points obtained courtesy of UAV-captured videos to the 3-D world space based on the world-to-image point correspondences. To identify optimal features in the UAV imagery, geo-reference targets with various patterns were installed on a test building specimen, which was then subjected to earthquake shaking. A feature point tracking-based algorithm for square checkerboard patterns and a Hough Transform-based algorithm for concentric circular patterns are developed to ensure reliable detection and tracking of image features. Photogrammetry techniques are applied to reconstruct the 3-D world points and extract structural displacements. The proposed methodology is validated by monitoring the displacements of a full-scale 6-story mass timber building during a series of shake table tests. Reasonable accuracy is achieved in that the overall root-mean-square errors of the tracking results are at the millimeter level compared to ground truth measurements from analog sensors. Insights on optimal features for monitoring structural dynamic response are discussed based on statistical analysis of the error characteristics for the various reference target patterns used to track the structural displacements. 

   more » « less
4. Transferability of the Deep Learning Mask R-CNN Model for Automated Mapping of Ice-Wedge Polygons in High-Resolution Satellite and UAV Images

   https://doi.org/10.3390/rs12071085  

   Zhang, Weixing; Liljedahl, Anna K.; Kanevskiy, Mikhail; Epstein, Howard E.; Jones, Benjamin M.; Jorgenson, M. Torre; Kent, Kelcy (April 2020, Remote Sensing)

   State-of-the-art deep learning technology has been successfully applied to relatively small selected areas of very high spatial resolution (0.15 and 0.25 m) optical aerial imagery acquired by a fixed-wing aircraft to automatically characterize ice-wedge polygons (IWPs) in the Arctic tundra. However, any mapping of IWPs at regional to continental scales requires images acquired on different sensor platforms (particularly satellite) and a refined understanding of the performance stability of the method across sensor platforms through reliable evaluation assessments. In this study, we examined the transferability of a deep learning Mask Region-Based Convolutional Neural Network (R-CNN) model for mapping IWPs in satellite remote sensing imagery (~0.5 m) covering 272 km2 and unmanned aerial vehicle (UAV) (0.02 m) imagery covering 0.32 km2. Multi-spectral images were obtained from the WorldView-2 satellite sensor and pan-sharpened to ~0.5 m, and a 20 mp CMOS sensor camera onboard a UAV, respectively. The training dataset included 25,489 and 6022 manually delineated IWPs from satellite and fixed-wing aircraft aerial imagery near the Arctic Coastal Plain, northern Alaska. Quantitative assessments showed that individual IWPs were correctly detected at up to 72% and 70%, and delineated at up to 73% and 68% F1 score accuracy levels for satellite and UAV images, respectively. Expert-based qualitative assessments showed that IWPs were correctly detected at good (40–60%) and excellent (80–100%) accuracy levels for satellite and UAV images, respectively, and delineated at excellent (80–100%) level for both images. We found that (1) regardless of spatial resolution and spectral bands, the deep learning Mask R-CNN model effectively mapped IWPs in both remote sensing satellite and UAV images; (2) the model achieved a better accuracy in detection with finer image resolution, such as UAV imagery, yet a better accuracy in delineation with coarser image resolution, such as satellite imagery; (3) increasing the number of training data with different resolutions between the training and actual application imagery does not necessarily result in better performance of the Mask R-CNN in IWPs mapping; (4) and overall, the model underestimates the total number of IWPs particularly in terms of disjoint/incomplete IWPs. 

   more » « less
5. Automatic Utility Pole Inclination Angle Measurement Using Unmanned Aerial Vehicle and Deep Learning

   Zhu, Zanbo; Zhang, Jing; Alam, Md Morshedul; Tokgoz, Berna Eren; Hwang, Seokyon (May 2019, Proceedings of the Institute of Industrial and Systems Engineers Annual conference 2019)

   Measuring the inclination angles of utility poles of the electric power distribution lines is critical to maintain power distribution systems and minimize power outages, because the poles are very vulnerable to natural disasters. However, traditional human-based pole inspection methods are very costly and require heavy workloads. In this paper, we propose a novel pole monitoring system to measure the inclination angle of utility poles from images captured by unmanned aerial vehicle (UAV) automatically. A state-of-the-art deep learning neural network is used to detect and segment utility poles from UAV street view images, and computer vision techniques are used to calculate the inclination angles based on the segmented poles. The proposed method was evaluated using 64 images with 84 utility poles taken in different weather conditions. The pole segmentation accuracy is 93.74% and the average inclination angle error is 0.59 degrees, which demonstrate the efficiency of the proposed utility pole monitoring system. 

   more » « less