The intelligent reflecting surface (IRS) is a technology for 6G-and-beyond [2], [3]. An IRS is a software-controlled metasurface, consisting of configurable meta-atoms with flexible reflection coefficients. By fine-tuning these meta-atoms, the IRS can change the wireless propagation environment, resulting in power savings, throughput increase, etc. Compared to a traditional antenna array with radio frequency (RF) chains for active relaying/beamforming, an IRS is made of low cost meta-surfaces, which consume low energy for tuning [4]. These benefits have motivated research on utilizing IRS in communications/signal processing literature. We aim to address three shortcomings of the current art, as discussed below.
A more comprehensive version of this paper is under review in IEEE Transactions on Wireless Communications [1]. C. G. Brinton and D. J. Love were supported in part by the National Spectrum Consortium (NSC) under Grant W15QKN-15-9-1004 and Office of Naval Research (ONR) under Grant N00014-21-1-2472. T. Kim was supported in part by the National Science Foundation (NSF) under Grants CNS1955561. coefficient design for communications has treated the reflection phase and attenuation of each meta atom independently [5]. In reality, the phase shift and attenuation are interdependent because the reflection behavior is determined by adjusting the tunable elements inside the meta-atoms, i.e., their controllable capacitance, as revealed in physics literature [6]. This interdependency has only been considered in a few works in the communications area [7].
2) Dependency between Meta-atoms' Reflection Coefficient and Incident Angle of Incoming Signals: Another practical consideration neglected in existing works is the dependency between the IRS reflection behavior and the incident angles of the electromagnetic (EM) waves [8], [9]. In [9], the authors propose an angle-dependent reflection coefficient model for each meta-atom using an equivalent circuit model. To the best of our knowledge, the angle-dependent property of the IRS reflection coefficient has not been incorporated into uplink/downlink signal transmission models for wireless communication systems.
3) Low Overhead Feedback Channel: The configuration of IRS meta-atoms is usually controlled via reception of some information from the base station (BS) through a feedback link. This link typically has a low data rate since its channel state information is unknown to the BS [4]. To reduce the feedback overhead for IRS control, some recent works have considered codebook structures [8], [10], where the codebook refers to a set of IRS reflection coefficients. In these works, the BS feeds back a specific codeword index to the IRS, using which the IRS recovers the desired reflection coefficients from the codebook. In doing so, these works directly design the IRS reflection coefficients and thus do not consider the practical IRS reflection behavior mentioned in Sec. I-A1, I-A2.
We propose a new methodology for IRS control that explicitly considers the above three practical design aspects. In doing so, we consider that the IRS is deployed in realistic multi-path, time-varying channels. These considerations render current optimization-based methods for IRS control [5], [7], which rely on channel estimation, impractical: it is difficult Specifically, we propose a novel adaptive codebook-based limited feedback protocol. We directly design the meta-atom capacitance values, instead of their reflection coefficients as in existing methods. With the codebook as a set of capacitance values for the meta-atoms, we develop two adaptive codebook design methods: (i) random adjacency (RA) and (ii) deep neural network policy-based IRS control (DPIC). These approaches only require the end-to-end channel from the user equipment (UE) to the BS, which can be readily measured in real-time.
We begin by formalizing IRS meta-atom reflection behavior in Sec. II-A. Then, we describe the signal model of IRS-assisted uplink communications in Sec. II-B.
An IRS is composed of two interconnected systems as shown in Fig. 1(a): a meta-surface and a control board. A meta-surface is an ultra-thin sheet composed of periodic sub-wavelength metal/dielectric structures, i.e., meta-atoms. Each meta-atom generally contains a semiconductor device, i.e., the tunable element, such as a positive-intrinsic-negative (PIN) diode and a variable capacitor (varactor) [6]. A control board, e.g., a field programmable gate array (FPGA) [11], adjusts the bias voltage applied to the semiconductor in each meta-atom and changes its capacitance. Given a range of potential bias voltage values, the capacitance C n [t] at meta-atom n at time t satisfies
where C min and C max depend on the semiconductor used.
Through tuning the capacitance of the meta-atoms, their impedance can be adjusted. However, the impedance is also dependent on the incident angle of the incoming EM wave [8], [9]. Both of these factors should be considered in IRS reflection behavior design. As an example, we provide the impedance and reflection coefficient of a meta-atom equipped with a varactor using its equivalent circuit model [9] depicted in Fig. 1(b). Let θ [t] denote the incident angle of the -th channel path to the IRS. 1 Under a far-field assumption where θ [t] is the same across all the meta-atoms, the impedance of meta-atom n can be described as [9] Z
,
where L T (•), C T (•), and R T (•) are the inductance, capacitance, and resistance of the top circuit layer in Fig. 1(b), respectively, L B (•) is the bottom layer inductance, C n [t] is the variable capacitance, and ω is the angular frequency of the EM waves.
Considering the impedance discontinuity between the free space impedance Z 0 ≈ 376.73 Ω and the meta-atom impedance
The expressions in ( 2)&(3) reveal two practical considerations for tuning the meta-atoms. First, the reflection attenuation
) are jointly controlled by the variable capacitance C n [t], as also reported in [7]. Thus, it is beneficial to design the variable capacitance instead of the reflection coefficient since some combinations of attenuation and phase shifts may not be feasible. Second, the reflection coefficient is a function of the incident angle θ [t], posing new challenges for applications of IRS in multi-path and time-varying channels, which will be discussed in Sec. III-A. While observed in [8], [9], this dependency has not yet been incorporated in the canonical signal model for IRS-assisted communications. In this paper, we incorporate these practical considerations into our signal model and methodology.
We consider IRS-assisted uplink communications with a UE, a BS, and an IRS, shown in Fig. 1. The UE posesses a single antenna, while the BS has N BS antennas. We assume a block fading channel model with time index t = 0, 1, ..., where channels are constant during each block. Let N IRS denote the number of IRS meta-atoms. We define the capacitance vector across the meta-atoms at time t as
We also formulate the reflection coefficient matrix
where the n-th diagonal
) is the reflection coefficient at meta-atom n, n ∈ {1, ..., N IRS }, given the incident angle θ [t].
We consider multi-path single tap channels and adopt a geometric channel model representation [12]. The channel from the UE to the IRS is described as
in which h UI (θ [t], t) is the -th path channel with incident angle θ [t] and L[t] is the number of paths. The received signal at the BS at time t is given by
where P ≥ 0 is the transmit power and
follows the complex Gaussian distribution CN (0, σ 2 I), where I denotes the identity matrix and σ 2 is the variance. In (7), the end-to-end compound channel
where
We first formulate the data rate maximization problem for IRS control and discuss the challenges associated with solving it in Sec. III-A. To address the challenges, we propose an adaptive codebook-based limited feedback protocol in Sec. III-B.
We formulate the data rate maximization at time t as maximize
where W is the channel bandwidth (in Hz). In other words, the objective is to adapt c[t] based on the time-varying channels. Operationally, we aim for ( 9)- (10) to be solved at the BS since the BS can obtain measurements and has abundant computing resources. The BS would then generate feedback information for the IRS. However, solving (9)-( 10) presents three key challenges. First, conventional optimization-based methods, relying on channel estimations, cannot be applied: the channel estimation requires the IRS reflection coefficients, which depend on the incident angles of incoming signals that cannot readily be measured (the IRS has no active sensors). Second, adaptive control of c[t] is necessary for efficient IRS operation in time-varying channels, which requires periodic information acquisition from the BS. Related to this is the third challenge: the feedback link has a low data rate [4]. The main contribution of our work is developing methodology to jointly address these challenges.
Motivated by the low overhead feedback requirement, we propose to exploit a codebook structure for IRS control, where the BS sends only a quantized codeword index to the IRS. Further, we consider adaptive design of this codebook based on channel variations. We denote the instantaneous codebook as
, where q m [t] ∈ R NIRS is the m-th codeword (capacitance vector) and M is the codebook size. The codebook is stored and updated at the IRS through its control board (see Fig. 1(a)). We propose a novel limited feedback protocol consisting of four steps conducted per each coherence time block t, depicted in Fig. 2:
Step 1. IRS channel sounding and reconfiguration. While the UE transmits pilot symbols, the IRS explores all of the M capacitance vectors, i.e., q m [t] ∈ C[t], m = 1, ..., M .
Step 2. Codeword selection at BS. The BS measures the effective channel h eff (q m [t], t) and calculates the data-rate R(q m [t], t) from (9), as IRS applies q m [t], m=1, ..., M . The BS obtains the codeword index
Step 3. Feedback to IRS and IRS final configuration. The BS feeds back m [t] to the IRS with log 2 M feedback bits. Then, the IRS tunes its meta-atoms with q
Step 4. Data transmission and IRS codebook update. The data transmission is conducted during the rest of the coherence time. During this period, the IRS obtains the next codebook C[t + 1] either locally or with assistance from the BS. The benefits of our protocol include its (i) simple procedure for IRS configuration, (ii) low-overhead feedback, and (iii) adaptation to dynamic channels. Careful design of the codebook C[t] is critical to obtaining high data rates, since the codewords {q m [t]} M m=1 are the solution candidates and the best one (q [t] in Step 3) among them is selected. 3 We next propose two adaptive codebook design approaches for selecting C[t + 1] in Step 4, which utilize the previous IRS decisions and responses, with the understanding that the channels are in practice correlated between consecutive coherence times.
For adaptive codebook design, we propose a low-overhead perturbation-based approach in Sec. IV-A and a deep neural network (DNN) policy-based approach in Sec. IV-B. Then, we present a group control strategy, and quantify the time overhead and average data rate over one channel coherence block in Sec. IV-C.
We first propose the random adjacency (RA) approach, which can be viewed as a random perturbation-based method [13] for codebook design. Since the optimization ( 9)-( 10) is conducted successively in time-correlated channels, the optimal solutions in adjacent time blocks are expected to be close to one another. The RA approach exploits this intuition by generating multiple solution candidates around the previous solution. The codebook resides and is updated at the IRS in this method.
Formally, the IRS obtains the codebook
, where the m-th codeword is updated by adding a random perturbation vector z m [t] ∈ R NIRS to the previous solution q [t] (obtained in Step 4 in Sec. III-B) as
where clip(•, [C min , C max ]) an element-wise clip function ensuring constraint (10). Each entry of z m [t] is generated from the uniform distribution U (-δ, δ), where δ is the maximum step size. The RA approach follows Step 1-Step 4 in Sec. III-B, while in Step 4 the codewords are updated by (12). Intuitively, the RA approach becomes more effective as the number of codewords M grows larger because more random points increase the chance of obtaining better codewords. However, M is limited due to the non-negligible IRS reconfiguration time and finite coherence time. This makes the performance of the RA approach restricted due to the nature of the randomness and motives us to develop our next codebook update algorithm.
We next propose a DNN policy-based IRS control (DPIC) approach, aiming to learn policies for updating the codewords. In DPIC, the codebook resides at the IRS, as in RA. However, the IRS now updates the codebook via information reception from the BS through the feedback link. We consider that each codeword is updated independently based on its prior deployments. Henceforth, without loss of generality, we focus on the updates of m-th codeword.
1) Low Overhead IRS Control via Direction Codebook: To conduct the low overhead codeword update, we introduce a fixed direction codebook D = {d k } K k=1 where d k ∈ R NIRS , k = 1, ..., K. The BS only transmits the index of a codeword in D to the IRS, which enables low feedback overhead for the codeword update. We assume that D is generated once at the beginning of the policy learning and shared at both the BS and IRS. The BS employs a learning architecture
2) IRS Control via Successive Decision Making: The learning architecture at the BS consists of a DNN policy that determines u m [t] and a quantization process that determines k m [t]. Our learning architecture consists of two phases: training phase and utilization phase. In the training phase, the BS aims to train the DNN policy to have an improved u m [t] over time, while in the utilization phase the BS exploits the trained DNN policy without additional training. In both phases, the BS first determines u m [t] with the DNN policy based on the current information (i.e., the codeword q m [t] in use and the effective channel h eff (q m [t], t)). Subsequently, the BS obtains k m [t] via a quantization process applied to u m [t] (described in Sec. IV-B4&IV-B5). The BS then feeds back k m [t] to the IRS, from which the IRS obtains the next codeword q m [t + 1] through (13). The next codeword affects the subsequent information at the BS (i.e., q m [t+1] and h eff (q m [t+1], t+1)). The codeword update can thus be formulated as a successive decision making process (Sec. IV-B3).
3) MDP for Codeword Update: We construct a Markov decision process (MDP) for the codeword update with the following state, action, and reward.
State. The state consists of information pertinent to the environment evolution, which we define as
where the real and imaginary parts of h eff (q m [t], t) are stored as separate state dimensions.
Action. The action is the direction vector u m [t]:
where each entry of the action is bounded to the maximum step size, i.e., [-δ, δ] ⊂ R. The action a m [t] is used to determine the index k m [t] based on different processes in the training (Sec. IV-B4) and utilization (Sec. IV-B5) phases. The next codeword q m [t + 1] is then obtained from k m [t] by (13).
Reward. The reward provides a measure of efficacy for policy learning. We define the reward as
where R(q m [t + 1], t + 1) denotes the data rate measured at time t+1 using codeword q m [t+1], and N clip,m [t] denotes the number of elements/dimensions in vector q m [t + 1] ∈ R NIRS that hit the clipping threshold in (13). N clip,m [t] is added as a penalty to avoid actions that result in the capacitance vectors violating (10).
We tailor a deep reinforcement learning (DRL) methodology to train the DNN policy with the formulated MDP. We assume that the BS trains M A different learning architectures, which are referred to as agents. Agent m ∈ {1, ..., M A } has the DNN policy π(s m [t]; w π,m ) : S → A where w π,m is the respective DNN weight parameters. We consider that agent m is trained with codeword m, m ∈ {1, ..., M A }.
The actual action of the agent m (i.e., d km[t] in ( 13)) is determined at the BS via the two following steps. First, the BS adds a random noise vector v m [t] to the output of the policy π(s m [t]; w π,m ) to have more diverse responses and avoid getting trapped in local optima [14]. The BS uses the clip function to confine the output result to the feasible action space. Second, the BS applies the quantization process using the direction codebook D, through which the BS determines the codeword index k m [t] ∈ {1, ..., K} with closest Euclidean distance. In other words,
To obtain π(s m [t]; w π,m ), we exploit the actor-critic network architecture of DRL. This architecture consists of an actor network π(s m [t]; w π,m ) and a critic network
The actor selects an action using a policy, and the critic evaluates/criticizes the action to guide the actor network to take better actions. However, using DNNs for reinforcement learning has been known to cause learning instability [14]. To stabilize the learning, we adopt the deep deterministic policy gradient (DDPG) approach [15] for training the actor-critic networks. The overall algorithm for training M A agents in the limited feedback protocol is given in Algorithm 1.
5) Utilization Phase with Trained DNN Policies: In the utilization phase, we utilize the M A trained agents to conduct codebook updates without additional training of the agents. The BS partitions M codewords among M A agents. We consider that codeword m ∈ {1, ..., M } is allocated to agent j
a single agent handles all M codeword updates, which we refer to as a single-agent DPIC (SDPIC). If M A > 1, multiple agents handle the M codeword updates, which we refer to as multiagent DPIC (MDPIC). Utilizing more agents often improves performance due to the ensemble learning principle [16].
The DPIC approach follows Step 1-Step 4 of the protocol in Sec. III-B. Additionally, in Step 2, the BS constructs s m [t] = {h eff (q m [t], t), q m [t]} and determines k m [t] in (17) by using v m [t] = 0 (no random noise addition) and w π,j[m] (instead of w π,m ). In Step 3, the BS additionally feeds back {k m [t]} M m=1 to the IRS for codebook updates, which incurs a total of log 2 M + M log 2 K feedback bits. In Step 4, the BS also updates the codebook. . Update e = max{ min, 0.99 e-1}, if e ≥ 1.
for t = 0, ..., Ntimestep -1 do 6:
Step 1. IRS channel sounding and reconfiguration. The IRS meta-atoms are tuned following {qm[t]} M A m=1 .
7:
Step 2. Inference at BS. Each agent m ∈ {1, ..., MA} computes rm[t -1] using ( 16), forms sm[t] in (14), and determines km[t] using (17) where vm[t] ∼ CN (0, eI).
Step 3. Feedback to IRS. The BS feeds back {km[t]} M A m=1 to the IRS. 9:
Step 4. IRS codebook update and BS training. The IRS obtains
from Bm, and updates wQ,m and wπ,m via the DDPG algorithm in [15]. We consider a group control [5], where IRS meta-atoms are partitioned into multiple groups and the same capacitance is applied for the meta-atoms belonging to the same group. This reduces the dimension of the design variables, and thus the BS can conduct the training/inference in a timely manner. We thus neglect the computation time overhead of our methods, and define the time overhead T p shown in Fig. 2 as
In ( 18), M T reconf denotes the total time for M IRS reconfiguration (in Step 1 in Sec. III-B) used in both RA and DPIC approaches, where T reconf is the time for each IRS reconfiguration, and T feedback denotes the time required for the feedback from the BS to the IRS (in Step 3), which is different for the RA and DPIC approach. For the RA approach, the feedback time is
where R feedback (bits/s/Hz) is the unit data rate for the feedback link. For the DPIC approach, the feedback time is
Lastly, T final denotes the execution time of the final IRS reconfiguration (in Step 3). If the selected index m [t] coincides with the last configuration in Step 1, the IRS does not need to change the configuration, i.e., T final = 0; otherwise T final =T reconf .
To measure the average data rate during one coherence block with coherence time T c , we define effective data rate as
where T c -T p is the actual data transmission time and q [t] ∈ C[t] is the selected codeword for the IRS configuration. The above metric captures the tradeoff between the data rate and the time overhead T p . As M increases, the data rate may increase due to having larger number of reconfigurations, while T p also increases leading to decreasing of T c -T p . In Sec. V, we evaluate the data rate and effective data rate under different M .
In this section, we describe the simulation setup in Sec. V-A, and then present and discuss the simulation results in Sec. V-B.
To emulate practical IRS reflection behavior, we recover Γ(C, θ) from the data in Fig. 4 and Table 1 of [9], where the ranges of C and θ are (C min , C max ) = (0.4, 2.7) pF and (0 o , 90 o ), respectively. We set f = 5.195 GHz and consider only azimuth coordinates as in [9]. We assume T c = 5 ms, N BS = 5 and N IRS = 200. We consider a group control with the number of groups N G = 10, where N IRS /N G = 20 metaatoms are controlled by each common capacitance. The BS antenna spacing is d BS = λ/2, and the IRS meta-atom spacing is d IRS = λ/10 where λ = c/f and c = 3 × 10 8 m/s. The BS and IRS are located at (0, 0) m and (90, 30) m, respectively. The initial UE position is randomly generated within the circle with radius 5 m at (100, 0) m. The UE is moving with the velocity v = 3 km/h and constant azimuth angle generated from U (0, 2π). We set P = 20 dBm, σ 2 = -80 dBm, W = 10 MHz, R feedback = 0.1 bits/s/Hz, and T reconf = 100µs [11]. 4 For RA, we set δ = (C max -C min )/5. For DPIC, we set N batch = 32 and |B m | = 5 × 10 5 , m ∈ {1, ..., M A }. For the DNNs, we consider a fully connected neural network with two hidden layers consisting of 400 and 300 neurons, respectively, with ReLU activation function. For the DNN policy, in the output layer the tanh function is employed, and the output is scaled by δ = (C max -C min )/4 to bound the actions. We set |D| = K = 2048, where each codeword in D is constructed by RVQ [17] ranging within [-δ, δ] NG . We employ the Adam optimizer for training. For MDP, we normalize the values as h eff (•) ← P/(σ 2 N BS N G )×h eff (•) and q m ← 10 12 ×q m in (14), a m ← 10 13 × a m in (15), and R(•) ← R(•)/W in (16).
We adopt a multi-path geometric channel model [12] for the IRS-BS, UE-BS, and UE-IRS channels. We model (i) the IRS-BS channel as Rician channel with K-factor of 5 [12], 10 non line-of-sight (NLoS) signal paths, and a path loss exponent (PLE) of 2, (ii) the UE-BS channel with only NLoS signals of 10 paths and a PLE of 3.75, and (iii) the UE-IRS channel with only NLoS signals of 10 paths and a PLE of 2.2. The small scale fading factors evolve according to a first-order Gauss-Markov process [18] with a time correlation coefficient of 0.95 (corresponding to speed of 3 km/h of the UE/scatterers), and the angle of each path varies over every coherence time by an amount generated from U (-0.1 o , 0.1 o ). 4 The reconfiguration time of IRS is determined by the characteristics of the control board and the internal communication between the control board and the meta-surface. The reconfiguration speed is typically a few kHz [11].
We train M A = 8 agents in the training phase with 1000 episodes, each with 500 timesteps (coherence blocks). Each episode has different realizations of the UE-IRS channel, UE-BS channel, IRS-BS channel, initial UE location, and UE moving direction. Our baseline is the RVQ codebook design [17]. Fig. 3(a) shows the effective data rate averaged over the timesteps. Each data point is a moving average over the previous 100 episodes. The performance of MDPIC is improved over time since the agents (specifically, the DNN policies) are trained to conduct better codebook updates.
We then evaluate our proposed algorithms in the utilization phase with 2000 episodes each with 30 timesteps. Fig. 3(b) depicts the average data rate over the utilization episodes. Our proposed schemes -RA, SPDIC, and MDPIC -update the codebook adaptively by using previous observations (i.e., previously used codeword and end-to-end channel) to improve the data rate over time. Our methods obtain their peak performance within 4-5 timesteps. Overall, as the number of IRS reconfiguration M increases, a higher data rate is achieved. The MDPIC yields better data rate compared to those of the SDPIC and RA due to the advantage of using multiple agents.
Fig. 3(c) shows the effective data rate along M . The effective data rate in (21) captures the tradeoff between the data rate and the time overhead discussed in Sec. IV-C. The MDPIC method shows the best performance in terms of effective data rate for any M , and has the highest value at M = 2. As M grows larger than 2, the increased time overhead outweighs the improvement of the data rate, leading to the decrease of the effective data rate.
In this paper, we introduced a novel signal model incorporating the practical IRS reflection behavior. To address the design challenges faced with IRS control under multi-path dynamic channels and low-overhead feedback requirements, we proposed the codebook-based limited feedback protocol. We proposed two adaptive codebook designs: the random adjacency and the deep neural network policy-based IRS control. Through simulations, we showed that the data rate and effective data rate performances are improved by the proposed schemes.
We are discussing the angle-dependent reflection model provided in[9] where only azimuth coordinates of the incident angle are considered.
We assume a narrowband system with a few tens of MHz in bandwidth. Then, we can approximate the reflection coefficients as constant across ω [8],[9].
M should be limited due to the finite coherence time and non-negligible IRS reconfiguration time. We consider that M is predetermined in the protocol.