More Like this

1. Federated Neuroevolution O-RAN: Enhancing the Robustness of Deep Reinforcement Learning xApps

   https://doi.org/10.1109/MCOM.001.2400491  

   Kouchaki, Mohammadreza; Abdalla, Aly Sabri; Marojevic, Vuk (January 2025, IEEE Communications Magazine)

   The open radio access network (O-RAN) architecture introduces RAN intelligent controllers (RICs) to facilitate the management and optimization of the disaggregated RAN. Reinforcement learning (RL) and its advanced form, deep RL (DRL), are increasingly employed for designing intelligent controllers, or xApps, to be deployed in the near-real time (near-RT) RIC. These models often encounter local optima, which raise concerns about their reliability for RAN intelligent control. We therefore introduce Federated O-RAN enabled Neuroevolution (NE)-enhanced DRL (F-ONRL) that deploys an NE-based optimizer xApp in parallel to the RAN controller xApps. This NE-DRL xApp framework enables effective exploration and exploitation in the near-RT RIC without disrupting RAN operations. We implement the NE xApp along with a DRL xApp and deploy them on Open AI Cellular (OAIC) platform and present numerical results that demonstrate the improved robustness of xApps while effectively balancing the additional computational load. 

   more » « less
2. Actor-Critic Network for O-RAN Resource Allocation: xApp Design, Deployment, and Analysis

   https://doi.org/10.1109/GCWkshps56602.2022.10008713  

   Kouchaki, Mohammadreza; Marojevic, Vuk (December 2022, 2022 IEEE Globecom Workshops (GC Wkshps))

   Open Radio Access Network (O-RAN) has introduced an emerging RAN architecture that enables openness, intelligence, and automated control. The RAN Intelligent Controller (RIC) provides the platform to design and deploy network controllers. xApps are the applications that can leverage machine learning (ML) algorithms for near-real time control. Despite the opportunities provided by this new architecture, the progress of practical artificial intelligence (AI)-based solutions for network control and automation has been slow. There is a lack of end-to-end solutions for designing, deploying, and testing AI-based xApps in production-like network settings. This paper introduces an end-to-end O-RAN design and evaluation procedure using the latest O-RAN architecture and interface releases. We provide details on the development of a reinforcement learning (RL)-based xApp, considering two RL approaches and present numerical results to validate the xApp. 

   more » « less
3. Toward Next Generation Open Radio Access Networks--What O-RAN Can and Cannot Do!

   https://doi.org/10.1109/MNET.108.2100659  

   Abdalla, Aly S.; Upadhyaya, Pratheek S.; Shah, Vijay K.; Marojevic, Vuk (July 2022, IEEE Network)

   The open radio access network (O-RAN) describes an industry-driven open architecture and interfaces for building next generation RANs with artificial intelligence (AI) controllers. We circulated a survey among researchers, developers, and practitioners to gather their perspectives on O-RAN as a framework for 6G wireless research and development (R&D). The majority responded in favor of O-RAN and identified R&D of interest to them. Motivated by these responses, this paper identifies the limitations of the current O-RAN specifications and the technologies for overcoming them. We recognize end-to-end security, deterministic latency, physical layer real-time control, and testing of AI-based RAN control applications as the critical features to enable and discuss R&D opportunities for extending the architectural capabilities of O-RAN as a platform for 6G wireless. 

   more » « less
4. 3D-O-RAN: Dynamic Data Driven Open Radio Access Network Systems

   https://doi.org/10.1109/MILCOM55135.2022.10017706  

   Restuccia, Francesco; Blasch, Erik; Ashdown, Andrew; Ashdown, Jonathan; Turck, Kurt (November 2022, MILCOM 2022 - 2022 IEEE Military Communications Conference (MILCOM))

   This position paper introduces a Dynamic Data Driven Open Radio Access Network System (3D-O-RAN). The key objective of 3D-O-RAN is to support congested, contested and contaminated tactical settings where multimedia sensors, application constraints and operating wireless conditions may frequently change over space, time and frequency. 3D-O-RAN is compliant with the O-RAN specification for beyond 5G cellular systems to reduce costs and guarantee interoperability among vendors. Moreover, 3D-O-RAN integrates computational, sensing, and cellular networking components in a highly-dynamic, feedback-based, data-driven control loop. Specifically, 3D-O-RAN is designed to incorporate heterogeneous data into the network control loop to achieve a system-wide optimal operating point. Moreover, 3D-O-RAN steers the multimedia sensor measurement process in real time according to the required application needs and current physical and/or environmental constraints. 3D-O-RAN uses (i) a semantic slicing engine, which takes into account the semantic of the application to optimally compress the multimedia stream without losing in classification accuracy; (ii) a dynamic data driven neural network certification system that translates mission-level constraints into technical-level constraints on neural network latency/accuracy, and occupation of hardware/software resources. Realistic use-case scenarios of 3D-O-RAN in a tactical context demonstrate system performance. 

   more » « less
5. Deep reinforcement learning based voltage control revisited

   https://doi.org/10.1049/gtd2.13001  

   Nematshahi, Saeed; Shi, Di; Wang, Fengyu; Yan, Bing; Nair, Adithya (October 2023, IET Generation, Transmission & Distribution)

   Abstract Deep Reinforcement Learning (DRL) has shown promise for voltage control in power systems due to its speed and model‐free nature. However, learning optimal control policies through trial and error on a real grid is infeasible due to the mission‐critical nature of power systems. Instead, DRL agents are typically trained on a simulator, which may not accurately represent the real grid. This discrepancy can lead to suboptimal control policies and raises concerns for power system operators. In this paper, we revisit the problem of RL‐based voltage control and investigate how model inaccuracies affect the performance of the DRL agent. Extensive numerical experiments are conducted to quantify the impact of model inaccuracies on learning outcomes. Specifically, techniques that enable the DRL agent are focused on learning robust policies that can still perform well in the presence of model errors. Furthermore, the impact of the agent's decisions on the overall system loss are analyzed to provide additional insight into the control problem. This work aims to address the concerns of power system operators and make DRL‐based voltage control more practical and reliable. 

   more » « less