skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: QCell: Self-optimization of Softwarized 5G Networks through Deep Q-learning
With the unprecedented rise in traffic demand and mobile subscribers, real-time fine-grained optimization frameworks are crucial for the future of cellular networks. Indeed, rigid and inflexible infrastructures are incapable of adapting to the massive amounts of data forecast for 5G networks. Network softwarization, i.e., the approach of controlling “everything” via software, endows the network with unprecedented flexibility, allowing it to run optimization and machine learning-based frame- works for flexible adaptation to current network conditions and traffic demand. This work presents QCell, a Deep Q-Network- based optimization framework for softwarized cellular networks. QCell dynamically allocates slicing and scheduling resources to the network base stations adapting to varying interference con- ditions and traffic patterns. QCell is prototyped on Colosseum, the world’s largest network emulator, and tested in a variety of network conditions and scenarios. Our experimental results show that using QCell significantly improves user’s throughput (up to 37.6%) and the size of transmission queues (up to 11.9%), decreasing service latency.  more » « less
Award ID(s):
1925601
PAR ID:
10298727
Author(s) / Creator(s):
; ; ; ; ;
Date Published:
Journal Name:
IEEE Globecom 2021
Page Range / eLocation ID:
1-6
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Offloading cellular traffic to WiFi networks plays an important role in alleviating the increasing burden on cellular networks. However, excessive traffic offloading brings severe packet collisions into a WiFi network due to its contention-based medium access scheme, which significantly reduces the WiFi network’s throughput. In this paper, we propose DAO, a device-to-device (D2D) communications assisted traffic offloading scheme to improve the amount of traffic offloaded from cellular to WiFi in integrated cellular and WiFi networks. Specifically, in an integrated cellular-WiFi network, the cellular network exploits D2D communications in licensed cellular bands to aggregate traffic from cellular users before offloading it to the WiFi network to reduce the number of contending users in WiFi access. The traffic offloading process in DAO is formulated as an optimization problem that jointly takes into account the activations of aggregation nodes (ANs) and the connections between ANs and offloading users to maximize the offloaded traffic while guaranteeing the long-term data rates required by the offloading users. Extensive simulation results reveal the significant performance gain achieved by DAO over the existing schemes. 
    more » « less
  2. Given an urban development plan and the historical traffic observations over the road network, the Conditional Urban Traffic Estimation problem aims to estimate the resulting traffic status prior to the deployment of the plan. This problem is of great importance to urban development and transportation management, yet is very challenging because the plan would change the local travel demands drastically and the new travel demand pattern might be unprecedented in the historical data. To tackle these challenges, we propose a novel Conditional Urban Traffic Generative Adversarial Network (Curb-GAN), which provides traffic estimations in consecutive time slots based on different (unprecedented) travel demands, thus enables urban planners to accurately evaluate urban plans before deploying them. The proposed Curb-GAN adopts and advances the conditional GAN structure through a few novel ideas: (1) dealing with various travel demands as the "conditions" and generating corresponding traffic estimations, (2) integrating dynamic convolutional layers to capture the local spatial auto-correlations along the underlying road networks, (3) employing self-attention mechanism to capture the temporal dependencies of the traffic across different time slots. Extensive experiments on two real-world spatio-temporal datasets demonstrate that our Curb-GAN outperforms major baseline methods in estimation accuracy under various conditions and can produce more meaningful estimations. 
    more » « less
  3. null (Ed.)
    This paper studies congestion-aware route- planning policies for Autonomous Mobility-on-Demand (AMoD) systems, whereby a fleet of autonomous vehicles provides on- demand mobility under mixed traffic conditions. Specifically, we first devise a network flow model to optimize the AMoD routing and rebalancing strategies in a congestion-aware fashion by accounting for the endogenous impact of AMoD flows on travel time. Second, we capture reactive exogenous traffic consisting of private vehicles selfishly adapting to the AMoD flows in a user- centric fashion by leveraging an iterative approach. Finally, we showcase the effectiveness of our framework with a case- study considering the transportation sub-network in New York City. Our results suggest that for high levels of demand, pure AMoD travel can be detrimental due to the additional traffic stemming from its rebalancing flows, whilst the combination of AMoD with walking or micromobility options can significantly improve the overall system performance. 
    more » « less
  4. With the development of space-air-ground integrated networks, Low Earth Orbit (LEO) satellite networks are envisioned to play a crucial role in providing data transmission services in the 6G era. However, the increasing number of connected devices leads to a surge in data volume and bursty traffic patterns. Ensuring the communication stability of LEO networks has thus become essential. While Lyapunov optimization has been applied to network optimization for decades and can guarantee stability when traffic rates remain within the capacity region, its applicability in LEO satellite networks is limited due to the bursty and dynamic nature of LEO network traffic. To address this issue, we propose a robust Lyapunov optimization method to ensure stability in LEO satellite networks. We theoretically show that for a stabilizable network system, traffic rates do not have to always stay within the capacity region at every time slot. Instead, the network can accommodate temporary capacity region violations, while ensuring the long-term network stability. Extensive simulations under various traffic conditions validate the effectiveness of the robust Lyapunov optimization method, demonstrating that LEO satellite networks can maintain stability under finite violations of the capacity region. 
    more » « less
  5. Efficiently transferring data over long-distance, high-speed networks requires optimal utilization of available network bandwidth. One effective method to achieve this is through the use of parallel TCP streams. This approach allows applications to leverage network parallelism, thereby enhancing transfer throughput. However, determining the ideal number of parallel TCP streams can be challenging due to non-deterministic background traffic sharing the network, as well as non-stationary and partially observable network signals. We present a novel learning-based approach that utilizes deep reinforcement learning (DRL) to determine the optimal number of parallel TCP streams. Our DRL-based algorithm is designed to intelligently utilize available network bandwidth while adapting to different network conditions. Unlike rule-based heuristics, which lack generalization in unknown network scenarios, our DRL-based solution can dynamically adjust the parallel TCP stream numbers to optimize network bandwidth utilization without causing network congestion and ensuring fairness among competing transfers. We conducted extensive experiments to evaluate our DRL-based algorithm’s performance and compared it with several state-of-the-art online optimization algorithms. The results demonstrate that our algorithm can identify nearly optimal solutions 40% faster while achieving up to 15% higher throughput. Furthermore, we show that our solution can prevent network congestion and distribute the available network resources fairly among competing transfers, unlike a discriminatory algorithm. 
    more » « less