Search for: All records

Human action recognition is an important topic in artificial intelligence with a wide range of applications including surveillance systems, search-and-rescue operations, human-computer interaction, etc. However, most of the current action recognition systems utilize videos captured by stationary cameras. Another emerging technology is the use of unmanned ground and aerial vehicles (UAV/UGV) for different tasks such as transportation, traffic control, border patrolling, wild-life monitoring, etc. This technology has become more popular in recent years due to its affordability, high maneuverability, and limited human interventions. However, there does not exist an efficient action recognition algorithm for UAV-based monitoring platforms. This paper considers UAV-based video action recognition by addressing the key issues of aerial imaging systems such as camera motion and vibration, low resolution, and tiny human size. In particular, we propose an automated deep learning-based action recognition system which includes the three stages of video stabilization using the SURF feature selection and Lucas-Kanade method, human action area detection using faster region-based convolutional neural networks (R-CNN), and action recognition. We propose a novel structure that extends and modifies the InceptionResNet-v2 architecture by combining a 3D CNN architecture and a residual network for action recognition. We achieve an average accuracy of 85.83% for the entire-video-level recognition when applying our algorithm to the popular UCF-ARG aerial imaging dataset. This accuracy significantly improves upon the state-of-the-art accuracy by a margin of 17%. 

Due to Wildfire's huge destructive impacts on agriculture and food production, wildlife habitat, climate, human life and ecosystem, timely discovery of fires enable swift response to fires before they go out of control, in order to minimize the resulting damage and impacts. One of the emerging technologies for fire monitoring is deploying Unmanned Aerial Vehicles, due to their high flexibility and maneuverability, less human risk, and on-demand high quality imaging capabilities. In order to realize a real-time system for fire detection and expansion analysis, fast and high-accuracy image-processing algorithms are required. Several studies have shown that deep learning methods can provide the most accurate response, however the training time can be prohibitively long, especially when using online learning for constant refinement of the developed model. Another challenge is the lack of large datasets for training a deep learning algorithm. In this respect, we propose to use a pretrained mobileNetV2 architecture to implement transfer learning, which requires a smaller dataset and reduces the computational complexity while not compromising the accuracy. In addition, we conduct an effective data augmentation pipeline to simulate some extreme scenarios, which could promise the robustness of our approach. The testing results illustrate that our method maintains a high identification accuracy in different situations - original dataset (99.7%), adding Gaussian blurred (95.3%), and additive Gaussian noise (99.3%). 

Recently, using drones for forest fire management has gained a lot of attention from the research community due to their advantages such as low operation and deployment cost, flexible mobility, and high-quality imaging. It also minimizes human intervention, especially in hard-to-reach areas where the use of ground-based infrastructure is troublesome. Drones can provide virtual reality to firefighters by collecting ondemand high-resolution images with adjustable zoom, focus, and perspective to improve fire control and eliminate human hazards. In this paper, we propose a novel model for fire expansion as well as a distributed algorithm for drones to relocate themselves towards the front-line of an expanding fire field. The proposed algorithm comprises a light-weight image processing for fire edge detection that is highly desirable over computational expensive deep learning methods for resource-constrained drones. The positioning algorithm includes motions tangential and normal to fire frontline to follow the fire expansion while keeping minimum pairwise distances for collision avoidance and non-overlapping imaging. We proposed an action-reward mechanism to adjust the drones’ speed and processing rate based on the fire expansion rate and the available onboard processing power. Simulations results are provided to support the efficacy of the proposed algorithm. 

Dynamic network topology can pose important challenges to communication and control protocols in networks of autonomous vehicles. For instance, maintaining connectivity is a key challenge in unmanned aerial vehicle (UAV) networks. However, tracking and computational resources of the observer module might not be sufficient for constant monitoring of all surrounding nodes in large-scale networks. In this paper, we propose an optimal measurement policy for network topology monitoring under constrained resources. To this end, We formulate the localization of multiple objects in terms of linear networked systems and solve it using Kalman filtering with intermittent observation. The proposed policy includes two sequential steps. We first find optimal measurement attempt probabilities for each target using numerical optimization methods to assign the limited number of resources among targets. The optimal resource allocation follows a waterfall-like solution to assign more resources to targets with lower measurement success probability. This provides a 10% to 60% gain in prediction accuracy. The second step is finding optimal on-off patterns for measurement attempts for each target over time. We show that a regular measurement pattern that evenly distributed resources over time outperforms the two extreme cases of using all measurement resources either in the beginning or at the end of the measurement cycle. Our proof is based on characterizing the fixed-point solution of the error covariance matrix for regular patterns. Extensive simulation results confirm the optimality of the most alternating pattern with up to 10-fold prediction improvement for different scenarios. These two guidelines define a general policy for target tracking under constrained resources with applications to network topology prediction of autonomous systems 

In this paper, we propose a drone-based wildfire monitoring system for remote and hard-to-reach areas. This system utilizes autonomous unmanned aerial vehicles (UAVs) with the main advantage of providing on-demand monitoring service faster than the current approaches of using satellite images, manned aircraft and remotely controlled drones. Furthermore, using autonomous drones facilitates minimizing human intervention in risky wildfire zones. In particular, to develop a fully autonomous system, we propose a distributed leader-follower coalition formation model to cluster a set of drones into multiple coalitions that collectively cover the designated monitoring field. The coalition leader is a drone that employs observer drones potentially with different sensing and imaging capabilities to hover in circular paths and collect imagery information from the impacted areas. The objectives of the proposed system include: i) to cover the entire fire zone with a minimum number of drones, and ii) to minimize the energy consumption and latency of the available drones to fly to the fire zone. Simulation results confirm that the performance of the proposed system- without the need for inter-coalition communications- approaches that of a centrally-optimized system. 

This paper presents a novel mission-oriented path planning algorithm for a team of Unmanned Aerial Vehicles (UAVs). In the proposed algorithm, each UAV takes autonomous decisions to find its flight path towards a designated mission area while avoiding collisions to stationary and mobile obstacles. The main distinction with similar algorithms is that the target destination for each UAV is not apriori fixed and the UAVs locate themselves such that they collectively cover a potentially time-varying mission area. One potential application for this algorithm is deploying a team of autonomous drones to collectively cover an evolving forest wildfire and provide virtual reality for firefighters. We formulated the algorithm based on Reinforcement Learning (RL) with a new method to accommodate continuous state space for adjacent locations. To consider a more realistic scenario, we assess the impact of localization errors on the performance of the proposed algorithm. Simulation results show that the success probability for this algorithm is about 80% when the observation error variance is as high as 100 (SNR:-6dB). 

Managing energy consumption for computation and communication is a key requirement for flying ad hoc networks (FANET) to prolong the network lifetime. In many applications, the main role of drones is to collect imagery information and relay them to a ground station for further processing and decision making. In this paper, we present a predictive compression policy to maximize the end-to-end image quality penalized by communication and computation costs. The idea is to predict the number of remaining links to the destination for a given routing algorithm and use it to re-compress image frames at intermediate nodes such that the overall energy consumption is minimized. Numerical results confirm that the performance of this method is within 4% of the global optima and higher than the current fixed-rate policies with a significant margin. 

In this paper, we develop a distributed mechanism for spectrum sharing among a network of unmanned aerial vehicles (UAV) and licensed terrestrial networks. This method can provide a practical solution for situations where the UAV network may need external spectrum when dealing with congested spectrum or need to change its operational frequency due to security threats. Here we study a scenario where the UAV network performs a remote sensing mission. In this model, the UAVs are categorized to two clusters of relaying and sensing UAVs. The relay UAVs provide a relaying service for a licensed network to obtain spectrum access for the rest of UAVs that perform the sensing task. We develop a distributed mechanism in which the UAVs locally decide whether they need to participate in relaying or sensing considering the fact that communications among UAVs may not be feasible or reliable. The UAVs learn the optimal task allocation using a distributed reinforcement learning algorithm. Convergence of the algorithm is discussed and simulation results are presented for different scenarios to verify the convergence.