Title: Distributed Data-Driven Learning-Based Optimal Dynamic Resource Allocation for Multi-RIS-Assisted Multi-User Ad-Hoc Network
This study investigates the problem of decentralized dynamic resource allocation optimization for ad-hoc network communication with the support of reconfigurable intelligent surfaces (RIS), leveraging a reinforcement learning framework. In the present context of cellular networks, device-to-device (D2D) communication stands out as a promising technique to enhance the spectrum efficiency. Simultaneously, RIS have gained considerable attention due to their ability to enhance the quality of dynamic wireless networks by maximizing the spectrum efficiency without increasing the power consumption. However, prevalent centralized D2D transmission schemes require global information, leading to a significant signaling overhead. Conversely, existing distributed schemes, while avoiding the need for global information, often demand frequent information exchange among D2D users, falling short of achieving global optimization. This paper introduces a framework comprising an outer loop and inner loop. In the outer loop, decentralized dynamic resource allocation optimization has been developed for self-organizing network communication aided by RIS. This is accomplished through the application of a multi-player multi-armed bandit approach, completing strategies for RIS and resource block selection. Notably, these strategies operate without requiring signal interaction during execution. Meanwhile, in the inner loop, the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm has been adopted for cooperative learning with neural networks (NNs) to obtain optimal transmit power control and RIS phase shift control for multiple users, with a specified RIS and resource block selection policy from the outer loop. Through the utilization of optimization theory, distributed optimal resource allocation can be attained as the outer and inner reinforcement learning algorithms converge over time. Finally, a series of numerical simulations are presented to validate and illustrate the effectiveness of the proposed scheme.  more » « less
Award ID(s):
2128656
PAR ID:
10541307
Author(s) / Creator(s):
;
Publisher / Repository:
MDPI
Date Published:
Journal Name:
Algorithms
Volume:
17
Issue:
1
ISSN:
1999-4893
Page Range / eLocation ID:
45
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Joint device-to-device (D2D) and cellular communication is a promising technology for enhancing the spectral efficiency of future wireless networks. However, the interference management problem is challenging since the operating devices and the cellular users share the same spectrum. The emerging reconfigurable intelligent surfaces (RIS) technology is a potentially ideal solution for this interference problem since RISs can shape the wireless channel in desired ways. This paper considers an RIS-aided joint D2D and cellular communication system where the RIS is exploited to cancel interference to the D2D links and maximize the minimum signal-to-interference plus noise (SINR) of the device pairs and cellular users. First, we adopt a popular alternating optimization (AO) approach to solve the minimum SINR maximization problem. Then, we propose an interference cancellation (IC)-based approach whose complexity is much lower than that of the AO algorithm. We derive a representation for the RIS phase shift vector which cancels the interference to the D2D links. Based on this representation, the RIS phase shift optimization problem is transformed into an effective D2D channel optimization. We show that the AO approach can converge faster and can even give better performance when it is initialized by the proposed IC solution. We also show that for the case of a single D2D pair, the proposed IC approach can be implemented with limited feedback from the single receive device. 
    more » « less
  2. This paper explores reconfigurable intelligent surfaces (RIS) for mitigating cross-system interference in spectrum sharing applications. Unlike conventional reflect-only RIS that can only adjust the phase of the incoming signal, a hybrid RIS is considered that can configure the phase and modulus of the impinging signal by absorbing part of the signal energy. We investigate two spectrum sharing scenarios: (1) Spectral coexistence of radar and communication systems, where a convex optimization problem is formulated to minimize the Frobenius norm of the channel matrix from the communication base station to the radar receiver, and (2) Spectrum sharing in device-to-device (D2D) communications, where a max-min scheme that optimizes the worst-case signal-to-interference-plus-noise ratio (SINR) among the D2D links is formulated, and then solved through fractional programming. Numerical results show that with a sufficient number of elements, the hybrid RIS can in many cases completely eliminate the interference, unlike a conventional non-absorptive RIS. 
    more » « less
  3. Dynamic spectrum access (DSA) is regarded as one of the key enabling technologies for future communication networks. In this paper, we introduce a power allocation strategy for distributed DSA networks using a powerful machine learning tool, namely deep reinforcement learning. The introduced power allocation strategy enables DSA users to conduct power allocation in a distributed fashion without relying on channel state information and cooperations among DSA users. Furthermore, to capture the temporal correlation of the underlying DSA network environments, the reservoir computing, a special class of recurrent neural network, is employed to realize the introduced deep reinforcement learning scheme. The combination of reservoir computing and deep reinforcement learning significantly improves the efficiency of the introduced resource allocation scheme. Simulation evaluations are conducted to demonstrate the effectiveness of the introduced power allocation strategy. 
    more » « less
  4. Today, numerous machine learning (ML) applications offer continuous data processing and real-time data analytics at the edge of wireless networks. Distributed real-time ML solutions are highly susceptible to the so-called straggler effect caused by resource heterogeneity, which can be mitigated by various computation offloading mechanisms that severely impact communication efficiency, especially in large-scale scenarios. To reduce the communication overhead, we leverage device-to-device (D2D) connectivity, which enhances spectrum utilization and allows for efficient data exchange between proximate devices. In particular, we design a novel D2D-aided coded distributed learning method named D2D-CDL for efficient load balancing across devices. The proposed solution captures system dynamics, including data (time-varying learning model, irregular intensity of data arrivals), device (diverse computational resources and volume of training data), and deployment (different locations and D2D graph connectivity). To decrease the number of communication rounds, we derive an optimal compression rate, which minimizes the processing time. The resulting optimization problem provides suboptimal compression parameters that improve the total training time. Our proposed method is particularly beneficial for real-time collaborative applications, where users continuously generate training data. 
    more » « less
  5. The broadcasting nature of wireless signals may result in the task offloading process of mobile edge computing (MEC) suffering serious information leakage. As a novel technology, physical layer security (PLS) combined with reconfigurable intelligent surfaces (RIS) can enhance transmission quality and security. This paper investigates the MEC service delay problem in RIS-aided vehicular networks under malicious eavesdropping. Due to the lack of an explicit formulation for the optimization problem, we propose a deep deterministic policy gradient (DDPG)-based communication scheme to optimize the secure MEC service. It aims to minimize the maximum MEC service time while reducing eavesdropping threats by jointly designing the RIS phase shift matrix and computing resource allocation in real-time. Simulation results demonstrate that 1) the DDPG-based scheme can help the base station make reasonable actions to realize secure MEC service in dynamic MEC vehicular networks; 2) deploying RIS can dramatically reduce eavesdropping threats and improve the overall MEC service quality. 
    more » « less