skip to main content

Explore Research Products in the NSF-PAR

Title: Distributed Cooperative Spectrum Sharing in UAV Networks Using Multi-Agent Reinforcement Learning
In this paper, we develop a distributed mechanism for spectrum sharing among a network of unmanned aerial vehicles (UAV) and licensed terrestrial networks. This method can provide a practical solution for situations where the UAV network may need external spectrum when dealing with congested spectrum or need to change its operational frequency due to security threats. Here we study a scenario where the UAV network performs a remote sensing mission. In this model, the UAVs are categorized to two clusters of relaying and sensing UAVs. The relay UAVs provide a relaying service for a licensed network to obtain spectrum access for the rest of UAVs that perform the sensing task. We develop a distributed mechanism in which the UAVs locally decide whether they need to participate in relaying or sensing considering the fact that communications among UAVs may not be feasible or reliable. The UAVs learn the optimal task allocation using a distributed reinforcement learning algorithm. Convergence of the algorithm is discussed and simulation results are presented for different scenarios to verify the convergence.  more » « less
Award ID(s):
1755984
NSF-PAR ID:
10133283
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
16th IEEE Annual Consumer Communications & Networking Conference (CCNC)
Page Range / eLocation ID:
1 to 6
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ( , Sensors)
    null (Ed.)
    Modern Public Safety Networks (PSNs) are assisted by Unmanned Aerial Vehicles (UAVs) to provide a resilient communication paradigm during catastrophic events. In this context, we propose a distributed user-centric risk-aware resource management framework in UAV-assisted PSNs supported by both a static UAV and a mobile UAV. The mobile UAV is entitled to a larger portion of the available spectrum due to its capability and flexibility to re-position itself, and therefore establish better communication channel conditions to the users, compared to the static UAV. However, the potential over-exploitation of the mobile UAV-based communication by the users may lead to the mobile UAV’s failure to serve the users due to the increased levels of interference, consequently introducing risk in the user decisions. To capture this uncertainty, we follow the principles of Prospect Theory and design a user’s prospect-theoretic utility function that reflects user’s risk-aware behavior regarding its transmission power investment to the static and/or mobile UAV-based communication option. A non-cooperative game among the users is formulated, where each user determines its power investment strategy to the two available communication choices in order to maximize its expected prospect-theoretic utility. The existence and uniqueness of a Pure Nash Equilibrium (PNE) is proven and the convergence of the users’ strategies to it is shown. An iterative distributed and low-complexity algorithm is introduced to determine the PNE. The performance of the proposed user-centric risk-aware resource management framework in terms of users’ achievable data rate and spectrum utilization, is achieved via modeling and simulation. Furthermore, its superiority and benefits are demonstrated, by comparing its performance against other existing approaches with regards to UAV selection and spectrum utilization. 
    more » « less
  2. ; ; ; ( , IEEE transactions on wireless communications)
    In dynamic spectrum access (DSA), secondary users (SU) should only be allowed to access a licensed band belonging to incumbent users (IU) when the quality-of-service (QoS) requirements of both IUs and SUs can be satisfied at the same time. However, IU’s location and its received interference strength are considered sensitive in many DSA systems which should not be revealed, making it very challenging to optimize the network utility subjected to satisfying the operation and security requirements of SUs and IUs. In this paper, we develop a secure and distributed SU transmit power control algorithm to solve this challenge. Our algorithm achieves optimal SU power control to maximize the sum of SU rates. The SINR-guaranteed coexistence between SUs and IUs are enabled to maintain effective communication, while no information is directly required from IUs. Local measurements of IU signals provided by Environmental sensing capability (ESC) also undergo a security masking process to ensure that IU location cannot be derived from its outputs. Convergence and stability properties of our algorithm and its privacy-protection strength are both theoretically analyzed and experimentally evaluated through simulations 
    more » « less
  3. ; ; ; ( , The 55th Annual Conference on Information Sciences and Systems)
    While there has been significant progress on statistical theories in the information community, there is a lack of studies in information-theoretic distributed resource allocation to maximize information gain. With advanced technologies of unmanned aerial vehicles (UAVs) in response to corresponding revised FAA regulations, this study focuses on developing a new framework for utilizing UAVs in incident management. As a result of new computing technologies, predictive decision-making studies have recently improved ERV allocations for a sequence of incidents; however, these ground-based operations do not simultaneously capture network-wide information. This study incorporates a real-time aerial view using UAVs with three key improvements. First, aerial observations update the status of the freeway shoulder, allowing an ERV to safely travel at full speed. Second, observing parameters of the congestion shockwave provides accurate measurements of the true impact of an incident. Third, real-time information can be gathered on the clearance progress of an incident scene. We automate UAV and ERV allocation while satisfying constraints between these vehicles using a distributed constraint optimization problem (DCOP) framework. To find the optimal assignment of vehicles, the proposed model is formulated and solved using the Max-Sum approach. The system utility convergence is presented for different scenarios of grid size, number of incidents, and number of vehicles. We also present the solution of our model using the Distributed Stochastic Algorithm (DSA). DSA with exploration heuristics outperformed the Max-Sum algorithm when probability threshold p=0.5 but degrades for higher values of p. 
    more » « less
  4. ; ; ; ; ; ; ( , 2021 18th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON))
    null (Ed.)
    To mitigate the long-term spectrum crunch problem, the FCC recently opened up the 6 GHz frequency band for unlicensed use. However, the existing spectrum sharing strategies cannot support the operation of access points in moving vehicles such as cars and UAVs. This is primarily because of the directionality-based spectrum sharing among the incumbent systems in this band and the high mobility of the moving vehicles, which together make it challenging to control the cross-system interference. In this paper we propose SwarmShare, a mobility-resilient spectrum sharing framework for swarm UAV networking in the 6 GHz band. We first present a mathematical formulation of the SwarmShare problem, where the objective is to maximize the spectral efficiency of the UAV network by jointly controlling the flight and transmission power of the UAVs and their association with the ground users, under the interference constraints of the incumbent system. We find that there are no closed-form mathematical models that can be used characterize the statistical behaviors of the aggregate interference from the UAVs to the incumbent system. Then we propose a data-driven three-phase spectrum sharing approach, including Initial Power Enforcement, Offline-dataset Guided Online Power Adaptation, and Reinforcement Learning-based UAV Optimization. We validate the effectiveness of SwarmShare through an extensive simulation campaign. Results indicate that, based on SwarmShare, the aggregate interference from the UAVs to the incumbent system can be effectively controlled below the target level without requiring the real-time cross-system channel state information. The mobility resilience of SwarmShare is also validated in coexisting networks with no precise UAV location information. 
    more » « less
  5. ; ; ( , 2019 IEEE 20th International Symposium on "A World of Wireless, Mobile and Multimedia Networks" (WoWMoM))
    The use of Millimeter-wave (mmWave) spectrum in cellular communications has recently attracted growing interest to support the expected massive increase in traffic demands. However, the high path-loss at mmWave frequencies poses severe challenges. In this paper, we analyze the potential coverage gains of using unmanned aerial vehicles (UAVs), as hovering relays, in integrated access and backhaul (IAB) mmWave cellular scenarios. Specifically, we utilize the WinProp software package, which employs ray tracing methodology, to study the propagation characteristics of outdoor mmWave channels at 30 and 60 GHz frequency bands in a Manhattan-like environment. In doing so, we propose the implementation of amplify-and-forward (AF) and decode-and-forward (DF) relaying mechanisms in the WinProp software. We show how the 3D deployment of UAVs can be defined based on the coverage ray tracing maps at access and backhaul links. Furthermore, we propose an adaptive UAV transmission power for the AF relaying. We demonstrate, with the aid of ray tracing simulations, the performance gains of the proposed relaying modes in terms of downlink coverage, and the received signal to interference and noise ratio (SINR). 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email