skip to main content

Explore Research Products in the NSF-PAR

This content will become publicly available on February 1, 2025

Title: Deep Reinforcement Learning for Interference Management in UAV-Based 3D Networks: Potentials and Challenges
Modern cellular networks are multi-cell and use universal frequency reuse to maximize spectral efficiency. This results in high inter-cell interference. This challenge is growing as cellular networks become three-dimensional with the adoption of unmanned aerial vehicles (UAVs). This is because the strength and number of interference links rapidly increase due to the line-of-sight channels in UAV communications. Existing interference management solutions require each transmitter to know the channel information of interfering signals, rendering them impractical due to excessive signaling overhead. In this article, we propose leveraging deep reinforcement learning for interference management to tackle this shortcoming. In particular, we show that interference can still be effectively mitigated even without knowing its channel information. We then discuss novel approaches to scale the algorithms with linear/sublinear complexity and decentralize them using multi-agent reinforcement learning. By harnessing interference, the proposed solutions enable the continued growth of civilian UAVs.  more » « less
Award ID(s):
2239524
NSF-PAR ID:
10528739
Author(s) / Creator(s):
; ; ; ;
Publisher / Repository:
IEEE
Date Published:
Journal Name:
IEEE Communications Magazine
Volume:
62
Issue:
2
ISSN:
0163-6804
Page Range / eLocation ID:
134 to 140
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; ( , 2021 18th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON))
    null (Ed.)
    To mitigate the long-term spectrum crunch problem, the FCC recently opened up the 6 GHz frequency band for unlicensed use. However, the existing spectrum sharing strategies cannot support the operation of access points in moving vehicles such as cars and UAVs. This is primarily because of the directionality-based spectrum sharing among the incumbent systems in this band and the high mobility of the moving vehicles, which together make it challenging to control the cross-system interference. In this paper we propose SwarmShare, a mobility-resilient spectrum sharing framework for swarm UAV networking in the 6 GHz band. We first present a mathematical formulation of the SwarmShare problem, where the objective is to maximize the spectral efficiency of the UAV network by jointly controlling the flight and transmission power of the UAVs and their association with the ground users, under the interference constraints of the incumbent system. We find that there are no closed-form mathematical models that can be used characterize the statistical behaviors of the aggregate interference from the UAVs to the incumbent system. Then we propose a data-driven three-phase spectrum sharing approach, including Initial Power Enforcement, Offline-dataset Guided Online Power Adaptation, and Reinforcement Learning-based UAV Optimization. We validate the effectiveness of SwarmShare through an extensive simulation campaign. Results indicate that, based on SwarmShare, the aggregate interference from the UAVs to the incumbent system can be effectively controlled below the target level without requiring the real-time cross-system channel state information. The mobility resilience of SwarmShare is also validated in coexisting networks with no precise UAV location information. 
    more » « less
  2. ; ; ; ( , ITU Journal on Future and Evolving Technologies)
    To integrate unmanned aerial vehicles (UAVs) in future large-scale deployments, a new wireless communication paradigm, namely, the cellular-connected UAV has recently attracted interest. However, the line-of-sight dominant air-to-ground channels along with the antenna pattern of the cellular ground base stations (GBSs) introduce critical interference issues in cellular-connected UAV communications. In particular, the complex antenna pattern and the ground reflection (GR) from the down-tilted antennas create both coverage holes and patchy coverage for the UAVs in the sky, which leads to unreliable connectivity from the underlying cellular network. To overcome these challenges, in this paper, we propose a new cellular architecture that employs an extra set of co-channel antennas oriented towards the sky to support UAVs on top of the existing down-tilted antennas for ground user equipment (GUE). To model the GR stemming from the down-tilted antennas, we propose a path-loss model, which takes both antenna radiation pattern and configuration into account. Next, we formulate an optimization problem to maximize the minimum signal-to-interference ratio (SIR) of the UAVs by tuning the up-tilt (UT) angles of the up-tilted antennas. Since this is an NP-hard problem, we propose a genetic algorithm (GA) based heuristic method to optimize the UT angles of these antennas. After obtaining the optimal UT angles, we integrate the 3GPP Release-10 specified enhanced inter-cell interference coordination (eICIC) to reduce the interference stemming from the down-tilted antennas. Our simulation results based on the hexagonal cell layout show that the proposed interference mitigation method can ensure higher minimum SIRs for the UAVs over baseline methods while creating minimal impact on the SIR of GUEs. 
    more » « less
  3. ; ; ; ( , 2019 IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN))
    We propose a mechanism for unlicensed LTE channel selection that not only takes into account interference to and from Wi-Fi access points but also considers other LTE operators in the unlicensed band. By collecting channel utilization statistics and sharing this information periodically with other unlicensed LTE eNBs, each eNB can improve their channel selection given their limited knowledge of the full topology. While comparing our algorithm to existing solutions, we find that the similarity between sensed Wi-Fi occupation at neighboring eNBs greatly impacts the performance of channel selection algorithms. To achieve better performance across diverse scenarios, we expand on our statistical channel selection formulation to include reinforcement learning, thereby balancing the shared contextual information with historical performance. We simulate operation in the unlicensed band using our channel selection algorithm and show how Wi-Fi load and inter-cell interference estimation can jointly be used to select transmission channels for all small cells in the network. Our approaches lead to an increase in user-perceived throughput and spectral efficiency across the entire band when compared to the greedy channel selection. 
    more » « less
  4. ; ; ( , IEEE INFOCOM 2023 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS))
    This paper considers the trajectory design problem for unmanned aerial vehicles (UAVs) via meta-reinforcement learning. It is assumed that the UAV can move in different directions to explore a specific area and collect data from the ground nodes (GNs) located in the area. The goal of the UAV is to reach the destination and maximize the total data collected during the flight on the trajectory while avoiding collisions with other UAVs. In the literature on UAV trajectory designs, vanilla learning algorithms are typically used to train a task-specific model, and provide near-optimal solutions for a specific spatial distribution of the GNs. However, this approach requires retraining from scratch when the locations of the GNs vary. In this work, we propose a meta reinforcement learning framework that incorporates the method of Model-Agnostic Meta-Learning (MAML). Instead of training task-specific models, we train a common initialization for different distributions of GNs and different channel conditions. From the initialization, only a few gradient descents are required for adapting to different tasks with different GN distributions and channel conditions. Additionally, we also explore when the proposed MAML framework is preferred and can outperform the compared algorithms. 
    more » « less
  5. ; ; ; ; ; ( , IEEE Journal on Selected Areas in Communications)
    Routing solutions for multi-hop underwater wireless sensor networks suffer significant performance degradation as they fail to adapt to the overwhelming dynamics of underwater environments. To respond to this challenge, we propose a new data forwarding scheme where relay selection swiftly adapts to the varying conditions of the underwater channel. Our protocol, termed CARMA for Channel-aware Reinforcement learning-based Multi-path Adaptive routing, adaptively switches between single-path and multi-path routing guided by a distributed reinforcement learning framework that jointly optimizes route-long energy consumption and packet delivery ratio. We compare the performance of CARMA with that of three other routing solutions, namely, CARP, QELAR and EFlood, through SUNSET-based simulations and experiments at sea. Our results show that CARMA obtains a packet delivery ratio that is up to 40% higher than that of all other protocols. CARMA also delivers packets significantly faster than CARP, QELAR and EFlood, while keeping network energy consumption at bay. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email