High on wall on top RIS a) b) Tx SDR with Directional Ant. User Robot Rx SDR with Omni-direct Ant. Best Selected Beam Interference to 2 nd User Legend RIS Selected Beam Backhaul channel (observations) [θ 1 ,…, θ 4 ] Beam Steering c) user user AP AP-1 AP-2 Rotational mechanism Software-defined RIS Controller 1 Introduction Industry 4.0 is set to revolutionize manufacturing and industrial services through the digital transformation of the field, enabling real-time decision-making and automation [1]. Technologies such as the Internet of Things (IoT), artificial intelligence (AI), cloud connectivity, large-scale machine-to-machine communication (M2M), and networked, mobile robotics are central to the future of manufacturing [2]. These robots can collect and analyze data, making autonomous decisions with minimal human intervention. However, such a robot-enabled network architecture demands ultra-fast data transfer speeds, exceptional reliability, and minimal latency [3]. While network densification is a possible path to achieve these goals, it involves a significant cost overhead [4], and thus, network designers need to trade-off permanent infrastructure installations with reconfigurable platforms that can adapt to robot mobility over time. Aside of mobility, the harsh propagation environment within the factory floors increases blockages, results in limited coverage, and significant path loss. To overcome these challenges, recent studies have shown that programmable wireless environments that enable reconfigurability by shaping signal reflections can improve signal-to-noise ratio (SNR) [5] and expand coverage [6].
In REMARKABLE, we realize such a network architecture with low-mobility MRs and multiple wireless APs deployed in a robotic factory floor within a rich-scattering environment. Network reconfigurability for enhancing the APs' coverage is achieved by controlling the propagation environment using software-controlled reconfigurable intelligent surfaces (RISs) [7][8][9]. This ensures reliable connectivity for low-mobility MRs by creating a radio environment that adapts with the MR location, as in Fig. 1a-b. Yet, this requires a solution to the problem of adaptive beam selection in dynamic channel conditions between the AP and the MR, as depicted in Fig. 1c.
• Problem 1 (blockage and coverage): Consider Fig. 1a, showing a factory floor where the LoS signal is blocked by obstacles. In absence of LoS conditions, the MR relies on the strongest nonline-of-sight (NLoS) reflection to establish a communication link with the AP. NLoS multipath components can cause destructive interference due to uncontrollable phase reflections, leading to significant communication disruption. This results in significant received signal strength (RSS) fluctuations with small robot movements. This issue is exacerbated in single-antenna equipped MRs. Deploying multiple APs as in Fig. 1a, ensures LoS links and expands coverage but increases communication overhead and infrastructure complexity.
• Problem 2 (mobility and beam searching): Narrow beams formed via phased antenna arrays can mitigate propagation loss, as well as improve signal reception through increased SNR. Typically, these beams are formed by adjusting antenna element weights, with steering directivity and beamwidth defined through a codebook. The APs equipped with such capabilities exhaustively sweep over the beams in a codebook to discover the optimal beam with the highest signal strength. However, exhaustive beam searches create significant overhead, and MR mobility requires repeated searches to maintain connectivity.
• Problem 3 (interference): Even with APs forming highly directional beams towards MRs, in a heterogeneous environment with multiple APs, the close proximity of MRs can cause excessive interference , degrading network performance [10]. Therefore, beam selection must be judiciously performed, as some beam candidates may not be suitable for data transmission.
Our approach aims to tackle problems 1, 2, and 3 for MR connectivity in factory floor settings by achieving the following steps:
1) We design a passive beamformer using an RIS, a planar array of passive reflective elements, each configured to adjust the amplitude and phase of incident signals. This allows us to create various beam patterns, which are then used to form a beam codebook for the beam steering (see Fig. 1c). To address Problem 1 (see Fig. 1b), several practical challenges must be considered: (i) Instead of relying on Channel State Information (CSI), our approach uses a predefined codebook where each codeword corresponds to the weights of RIS elements. This is necessary because RIS elements are passive and lack radio chains, making traditional channel estimation impractical. Estimating each channel component-would be proportional to the number of reflective elements, would create extreme overhead. Therefore, we develop a method for creating the desired reflection beam pattern by using a non-uniform phase sampling technique, optimizing each element's reflection gain while considering incident and reflection signals. ii) In a planar RIS, edge elements of RIS contribute less to beamforming, limiting overall gain-especially in dynamic settings like mobile robots. To address this challenge, we propose a novel reconfigurable rotation mechanism that adjusts the pitch and roll angles along the RIS's local coordinate axes, effectively enhancing beam coverage and improving performance. Given a fixed AP location, this method requires reflective beam pattern synthesis w.r.t the new angular domain of RIS. Consequently, we generate a multi-level codebook, each level corresponding to a specific pair of rotational angles.
2) We study beam selection using codebooks derived from reflective beam-pattern synthesis. Our goal is to learn online the optimal beam from the RIS to the MR by casting the task as a kernelized multi-armed bandit (MAB), with each codeword as an arm. To address Problem 2 and Problem 3, we impose an interference constraint at a neighboring MR. The objective is to select beams that maximize RSS at the target MR subject to this constraint over a time-varying channel. We model cross-beam correlations with a Gaussian Process (GP) bandit and propose a primal-dual GP-Upper Confidence Bound (UCB) algorithm to balance exploration and exploitation while enforcing the interference constraint. To handle non-stationarity, we add an adaptive restart mechanism inspired by the bandit-overbandit framework, which dynamically tunes the restart interval from feedback. REMARKABLE is theoretically grounded and validated on a real RIS-enabled robotic testbed-unlike prior theoretical works [11,12], which remain untested in practice, and existing RIS implementations [5,13,14], which predominantly target static scenarios.
(1) We create an RIS codebook with beam patterns in multiple directions, enabling the online learning algorithm to find the best beam without channel estimation. Additionally, we introduce a reconfigurable rotational mechanism to expand RIS coverage. (2) We formulate beam selection for an MR as a primal-dual GP-UCB framework to maximize signal strength while avoiding interference. To address the time-varying or non-stationarity, we adopt "bandit over bandit" concept restart strategy, which adaptively forgets past data by tuning the restart interval via an adversarial bandit. Our method achieves sub-linear dynamic regret and constraint violation without prior knowledge of budget variations, safely learning beam selection even under a time-varying channel. (3) We show that we achieve Õ (𝐵 1/4 𝑇 3/4 ) regret and Õ (𝐵 1/4 𝑇 3/4 ) violation bound where 𝐵 indicates the total change in the environment (i.e., the change in the reward and the constraint, defined in Sec. 6.3.1) over 𝑇 time steps. We improve the existing bound of Õ (𝐵𝑇 3/4 ) dynamic regret and the dynamic violation bound achieved in [12]. (4) We demonstrate REMARKABLE in a real-world setting using USRP X310-B210 SDRs (Software Defined Radios), with MRs and a PCB-fabricated RIS, as shown in Fig. 1c. Our results show that REMARKABLE achieves 46.8% improved performance over classical methods with 94.2% selection accuracy. (5) We release the software pipeline for the online learning framework and the RIS configuration-orchestration software [15].
2 Related Work
• RIS & Smart Surface: RIS technology and similar concepts like metasurfaces have recently been proposed to enhance applications
(a) 0 4 8 12 16 20 Locations -90 -75 -60 Average Power (dBm) AP1-no_obstacle AP2-no_obstacle AP1-with_obstacle AP2-with_obstacle (b) such as security [16], virtual reality [17], localization and sensing [18], beamforming [5,19], and over-the-air aggregation [20]. Recent research has focused on optimizing transceivers and RIS phase-shifts to minimize signal distortion [19,20], especially with imperfect CSI estimation [21]. However, these methods assume knowledge of wireless channels, which is challenging due to RIS's limited processing capabilities. Additionally, some practical works [5,13,14,18,22] rely on real-time channel estimation, causing overhead proportional to system size and requiring fast feedback. In contrast, REMARKABLE uses a predefined RIS codebook, avoiding such overhead. Similar to our approach, other works consider configuring RIS elements with pre-defined coding patterns [20,23] and leveraging an extra degree of freedom by optimizing rotation of RIS plane/elements to improve system performance [24,25]. In fact, the work [11] proposes a hierarchical codebook-generating method using pattern synthesis, followed by a beam training method using the two-mainlobe codewords from the designed codebook for beam sweeping. Unlike these stationary setups that use exhaustive beam searching, REMARKABLE offers a novel method for faster beam selection, even considering mobility.
• Beam Selection with Bandit: Online Learning (OL), particularly MAB frameworks, has become prominent for beam selection due to its inherent ability to balance exploration and exploitation. Standard MAB frameworks utilized in beam selection cannot capture the correlation among beam directions. The authors in [26][27][28][29] leverage contextual information to exploit such correlations. However, these papers do not consider the time-varying channel and interference constraints that we considered, assuming quasi-static channels; thereby, driven models cannot capture time-varying channels, most likely mapped to real-world settings. A recent kernelized MAB approach [12] addresses time-varying, interference-constrained channels but neglects RIS settings and lacks real-world implementation. Our work explicitly incorporates RIS, demonstrates improved theoretical bounds compared to [12], and provides experimental validation in practical scenarios.
Before designing REMARKABLE, we conduct preliminary experiments in a factory floor use-case to investigate Problem 1.
• Experimental Setup: Consider a scenario where low-mobility MRs roam a factory floor to complete assigned tasks (see Fig. 5). We use a Turtlebot2 robot, which navigates the floor and stops at target locations to collect data. The data collection part is obtained at the 5GHz band with 20MHz bandwidth signal by SDR X310 radios equipped with omni-directional VERT2450 tx-rx antennas, where one of them is mounted on the Thurtlebot, while the other two are placed in designated areas in the environment as APs to communicate with the MR.
• Observation: We conducted two factory-floor experiments-one with obstacles and one without-while the MR navigated and RSS was measured from each AP at target locations. As shown in Fig. 2a, the AP1-MR channel exhibits frequency-selective fading that varies with MR position, whereas Fig. 2b shows AP2 providing better coverage in regions where AP1 is weak, even without obstacles. However, with obstacles, neither AP ensures reliable coverage, indicating the need for additional APs. This, in turn, introduces interference management challenges and increases overhead in terms of coordination and communication resources (e.g., bandwidth).
We aim to create a codebook of beam patterns by optimizing the phases of RIS's reflective elements to achieve the desired reflections. We start by looking at a scenario with a single AP and a single RIS. The RIS is a planar array with 𝑁 ×𝑁 passive reflective elements that can be configured for complex-valued amplitude and phase changes. Moreover, each generated codebook should consist of beams with predefined beam resolution and cover desired angular space.
With 𝑁 × 𝑁 layout RIS, we can derive the far-field reflection gain pattern of the surface w.r.t a specific target angle as:
where 𝑢 𝑖 = 𝑘𝑑 𝑥 𝑐𝑜𝑠𝜙 𝑖 𝑠𝑖𝑛𝜃 𝑖 and 𝑣 𝑖 = 𝑘𝑑 𝑦 𝑠𝑖𝑛𝜙 𝑖 𝑠𝑖𝑛𝜃 𝑖 are 𝑢-𝑣 space coordinates when the source signal contacts on different reflective elements from AP with azimuth and elevation angle of 𝜙 𝑖 and 𝜃 𝑖 . Similarly, the term 𝑢 𝑟 = 𝑘𝑑 𝑥 𝑐𝑜𝑠𝜙 𝑟 𝑠𝑖𝑛𝜃 𝑟 and 𝑣 𝑟 = 𝑘𝑑 𝑦 𝑠𝑖𝑛𝜙 𝑟 𝑠𝑖𝑛𝜃 𝑟 represent when the signal reflects from the surface towards the target with the azimuth and elevation angle, 𝜙 𝑟 and 𝜃 𝑟 , respectively. The reflective elements are placed in half wavelengths along the x and y directions, 𝑑 𝑥 = 𝑑 𝑦 = 𝜆/2, also 𝑘 = 2𝜋/𝜆, and 𝜆 is the wavelength of the operational frequency. Eventually, the signal path follows the incident angle and impacts the surface. Then, on the surface, it will be perturbed by the configurations of reflective elements. This is represented by the term of reflection coefficient, Γ 𝑚𝑛 = |Γ 𝑚𝑛 |𝑒 𝑗Φ 𝑚𝑛 , wherein the complex-valued amplitude and phase changes are applied to the incident signal. Assuming the reflection magnitude of all reflective elements is unity, i.e., |Γ 𝑚𝑛 | = 1. Additionally, we denote 𝐺 𝑚𝑛 (𝜙 𝑖 , 𝜃 𝑖 ) as radiated gain per reflective element defined as
Here, 𝐹 𝑒 is obtained from estimated full-wave simulation in the Ansys HFSS 3D electromagnetic simulator. By re-forming Eq.1, we can transform reflection pattern to:
(2)
where Φ 𝑠 𝑚𝑛 = Φ 𝑚𝑛 -(𝑚𝑢 𝑖 +𝑛𝑣 𝑖 ) is constant due to fixed locations of AP and the RIS. From the sampling theory for 2D periodic functions, the reflection array complex weights, 𝐴 𝑚𝑛 can be obtained from the samples of its radiation pattern |𝐹 (𝑢 𝑟 , 𝑣 𝑟 )| as follows: 𝐴 𝑚𝑛 (𝜙 𝑖 , 𝜃 𝑖 ) = (𝑁 -1)/2 ∑︁ 𝑝,𝑞=-(𝑁 -1)/2
In Eq.4, 𝑢 𝑝 , 𝑣 𝑞 are sampling points for the RIS, where 𝑢 𝑝 = 2𝜋𝑝/𝑁 and 𝑣 𝑞 = 2𝜋𝑞/𝑁 . Also, 𝑚 ′ = 𝑚-(𝑁 -1)/2 and 𝑛 ′ = 𝑛-(𝑁 -1)/2 for 𝑚, 𝑛 ∈ [0, 𝑁 -1]. In this method, we assign nonuniform phases to the radiation patterns of RIS at the sampling points via 𝐹 (𝑢 𝑝 , 𝑣 𝑞 ) = |𝐹 (𝑢 𝑝 , 𝑣 𝑞 )|𝑒 𝑗Φ 𝑝𝑞 [31], where Φ 𝑝𝑞 is the phase assigned to sampling points (𝑢 𝑝 , 𝑣 𝑞 ). Then, we can find optimized phase values needed to steer the beam in the intended direction by minimizing the Mean Square Error (MSE) function between desired, Î𝑚𝑛 , and generated, 𝐼 𝑚𝑛 , power of reflection array weights such as
𝑛=0 (𝐼 𝑚𝑛 -Î𝑚𝑛 ) 2 . We apply Gradient Descent (GD) [32] optimization method to minimize 𝑀𝑆𝐸. First, we define the gradient of 𝑀𝑆𝐸, 𝜕𝑀𝑆𝐸/𝜕Φ 𝑝𝑞 w.r.t non-uniform sampling points of the regenerated beam pattern, which is calculated by chain rule as in
Here, both derivatives in the chain are derived independently, then we can have:
𝑀 𝑚𝑛 𝑒 𝑗 Φ 𝑠 𝑚𝑛 +Φ 𝑓 (5) where 𝑀 𝑚𝑛 = 𝑎 𝑚𝑛 (𝑎 2 𝑚𝑛 -Î𝑚𝑛 ) and Φ 𝑓 = 𝑚 ′ 𝑢 𝑝 + 𝑛 ′ 𝑣 𝑝 . Since Eq.5 is a form of Fourier Transform, we utilize the Fast Fourier Transform (FFT) techniques to calculate gradients. Then at each iteration, new non-uniform phase samples are found as:
where 𝜂 𝑔 is the learning rate of the GD optimizer, determining the step size to converge it to the optimal point. After obtaining optimized non-uniform phase samples, we can find the phase distribution of the surface, Φ 𝑚𝑛 , by considering the amplitude and assigned phases, Φ 𝑝𝑞 , of the radiation pattern 𝐹 (𝑢 𝑝 , 𝑣 𝑞 ) through Eq.4. Fig. 3 compares the desired and generated pattern at steered angles of (40°,30°), showing that desired pattern can be achieved by our method, albeit with some increased side slopes due to the effect of quantization.
Here, we address RIS's limitation in terms of its angular coverage and propose a solution to mitigate this by integrating a reconfigurable rotational mechanism. 4b). We propose enhancing RIS coverage by adding a reconfigurable rotational mechanism to adjust its orientation through pitch and roll angles. The yaw angle is not used as the z-axis is the projection axis of RIS. Implementing this method necessitates rebuilding the codebook structure and reformulating the beam shaping and steering process. and
. Notably, we assume the placement of AP satisfies the far-field conditions [33]. Consequently, the incident angle on each reflective element is given as 𝜙 𝑖 and 𝜃 𝑖 for a planar array surface. Each element position vector defined as 𝑟 𝑚𝑛 = [𝑚𝑑 𝑥 , 𝑛𝑑 𝑦 , 0] ′ , 𝑟 𝑚𝑛 ∈ 𝑅 3 𝜎, 𝜎 = 𝜋 2 𝑏 -1 (7) Fig. 4e shows the output of quantization corresponding to Fig.4d. 4.3.2 Codebook Design. For 𝑅 rotational angles, the size of the codebook 𝑆 𝑟 , 𝑟 ∈ 𝑅 is given by
considering a 3D beam scanning. Here, 𝑖 is a correlation constant that increases correlation between neighboring beams in 𝐶 𝑟 , while 𝜅 denotes RIS angular coverage (e.g., 2𝜋/3 within [-𝜋/3, 𝜋/3] for our design). Each beam code in 𝐶 𝑟 consists of 𝑁 × 𝑁 codewords representing the required phase configurations of reflective elements. Implementing mechanical rotation introduces a controlled overhead (2.4 ms/angle). In our experiments, rotations occur infrequently-primarily when robot mobility substantially alters the optimal RIS orientation and demands broader angular coverage-thus balancing the overhead against the achieved coverage gains.
In REMARKABLE, we consider the scenario in Fig. 1c, involving an AP with 𝐾 antennas, a singleantenna MR, and an RIS with 𝐴 = 𝑁 × 𝑁 passive reflective elements. Each element adjusts the amplitude and phase of the incident signal. Before data transmission, optimal beam selection through AP-RIS and RIS-MR channels is needed for array gain and high throughput, assuming no direct AP-MR link due to the cluttered factory floor. Let 𝑤 𝑎 ∈ C represent the effect of RIS element 𝑎 on the reflected signal; 𝑊 ∈ 𝐶 be the beamforming weight vector from the predefined codebook 𝐶. The received signal at MR through the RIS for the transmitted pilot signal 𝑥 𝑡 at the 𝑡 𝑡ℎ time slot is:
Let 𝑾 𝒕 = diag[𝑤 1 , ..., 𝑤 𝑎 , ..., 𝑤 𝐴 ] ∈ C 𝐴×𝐴 , 𝒉 ′ ∈ C 𝐾 ×𝐴 , and 𝒉 ′′ ∈ C 𝐴×1 be complex-valued matrices and vectors, with elements representing channel coefficients between the AP and RIS element 𝑎 and between 𝑎 and MR, respectively. 𝑛 𝑡 ∈ C 𝐾 ×1 is the additive white Gaussian noise (AWGN), with 𝑛 𝑡 ∼ CN (0, 𝜎 2 𝑰 𝐾 ).
In our setup, the AP and RIS are fixed, while the robots are mobile. Thus, ℎ ′ is quasi-static channel with coherence time 𝑇 𝑆 , and ℎ ′′ is time-varying channel with 𝑇 𝑀 , where 𝑇 𝑆 ≫ 𝑇 𝑀 . Here, we adopt the time-varying geometric channel model [35] with 𝐿 multipath components between the RIS and MR. The multipath time-varying channel model at the 𝑝 𝑡ℎ time instance in 𝑇 𝑀 is:
where 𝛼 𝑙 ∼ CN (0, 1) is the complex channel gain of the 𝑙 𝑡ℎ path, 𝑣 𝑙 is the Doppler shift, and 𝒂(𝜙 𝑙 , 𝜃 𝑙 ) is the reflection steering vector in the direction of 𝜙 𝑙 and 𝜃 𝑙 , respectively. We can derive the received signal at 𝑝 𝑡ℎ time instance as 𝑌 𝑝 = 𝒉 ′ 𝑑𝑖𝑎𝑔(𝒉 ′′ 𝑝 )𝝎 𝒑 𝑿 + 𝑵 𝑝 , 𝑿 ∈ C 1×𝑇 𝑀 is transmitted signal sequence and 𝝎 𝒑 ∈ C 𝐴×1 . We assume MRs cannot estimate the cascaded channel 𝑯 𝑝 = 𝒉 ′ 𝑑𝑖𝑎𝑔(𝒉 ′′ 𝑝 ) from AP to MR over RIS; thereby, they only observe received signal power when the AP selects a beam from the RIS codebook 𝐶 and transmits a pilot signal. if the pilot signal 𝑋 is set to be 1, then the received signal power (RSP) can be expressed as F 𝑝 (𝝎 𝑝 ) = | √ 𝑠𝑯 𝑝 𝝎 𝑝 + 𝑵 𝑝 | 2 . Note that we convert the received signal power to the RSS by 10𝑙𝑜𝑔(F 𝑝 (𝝎 𝑝 )) for our experiments. The target user is MR M, but other MRs, denoted as 𝑖 ∈ I, should not experience interference from MR M. Received signal power F 𝑝,𝑖 (𝝎 𝑝 ) for 𝑖 𝑡ℎ MR is also effected by time-varying channel 𝑯 𝑝,𝑖 .
We aim to control the beamforming weight vector 𝝎 𝑝 to find the optimal beam that achieves the largest expected RSP, E[F 𝑝,𝑚 (𝝎 𝑝 )] = 𝑠 |𝑯 𝑝,𝑚 𝝎 𝑝 | 2 , for MR M while maintaining the expected maximum RSP, E[𝑚𝑎𝑥
for MRs in I less than a threshold 𝜌. Given that 𝐻 𝑝,𝑚 and 𝐻 𝑝,𝑖 are unknown and environment-dependent, we formulate the beam selection as an online constrained stochastic optimization problem. Let 𝑇 denote the time slots of equal duration for beam selection before transmitting data. In time slot 𝑝 ∈ 𝑇 , the AP selects a beamforming weight vector, 𝝎 𝑝 , from a set of candidate beams (arms in the bandit), and observes RSP, F 𝑝 (𝝎 𝑝 ), from all the MRs. Here, we define 𝑟 𝑝 (𝒘 𝑝 ) = F 𝑝,𝑚 (𝝎 𝑝 ) as observed reward and 𝑔 𝑝 (𝒘 𝑝 ) = 𝑚𝑎𝑥 𝑖 ∈ I {F 𝑝,𝑖 (𝝎 𝑝 )} as observed utility function, and both are time-varying. Sequentially selected beams and corresponding sequential rewards with utilities are presented as {𝒘 1 , 𝒘 2 , ..., 𝒘 𝑝 } and {(𝑟 1 (𝒘 1 ), 𝑔 1 (𝒘 1 )), ..., (𝑟 𝑝 (𝒘 𝑝 ), 𝑔 𝑝 (𝒘 𝑝 ))}, respectively. Our objective is to find a policy, 𝜋 ∈ Π, that maximizes the expected cumulative reward, i.e., expected RSP, while satisfying a constraint on the expected utility:
Here, the AP selects the beam vector based on selection probability through policy 𝜋 𝑝 . Note that such a policy can depend on the historical information.
We adopt GP bandits due to their proven effectiveness in beam alignment tasks [12], particularly over traditional multi-armed bandits. GP bandits capture spatial correlations among beams and adapt to time-varying channels-key for mobile scenarios with dynamic channel states. With this motivation, first, we introduce Gaussian process (GP) kernel to represent the reward and the constraints. Then, we define a constrained GP-bandit problem to determine the beamforming vector (cf,(Eq.10)). We next describe the base algorithm to address the static robot case, followed by our novel modification addressing the non-stationarity.
6.1.1 Gaussian Processes. Our RIS codebook contains beamforming vectors for various rotational angles; different beamforming vectors 𝝎, from different codebooks C 𝑟 ∈ 𝐶 can produce similar beams as the coverage sector of each codebook can partially overlap. This results in a high correlation between beamforming vectors (𝜔, 𝜔 ′ ). Since our reward 𝑟 𝑝 and utility functions 𝑔 𝑝 are unknown and non-linear, we use Gaussian Processes and their Reproducing Kernel Hilbert Space (RKHS) to model this correlation, as inspired by [12]. Note that MAB problem, where each beamforming vector is an orthogonal arm, cannot model such correlation.
We define a GP over 𝐶 as GP 𝐶 (𝜇 (•), 𝑘 (•, •)) that is completely specified by a mean function 𝜇 and covariance function (kernel) 𝑘 : ∀𝜔 ∈ 𝐶. We assume that the reward function without noise 𝑓 𝑝 and constrained utility function without noise 𝑧 𝑝 come from a GP, and perturbed with Gaussian noise: 𝑟 𝑝 = 𝑓 𝑝 (𝜔) + 𝑛 𝑝 , with 𝑛 𝑝 ∼ N (0, 𝜎 2 ) and 𝑓 𝑝 (•) ∼ GP (𝜇 𝑝 (•), 𝑘 (•, •)). Hence, if a beam vector 𝜔 is selected then 𝑟 𝑝 ∼ GP (𝜇 𝑝 (𝜔), 𝑘 (𝜔, •) + 𝜎 2 ). Similar argument holds for 𝑧 𝑝 and 𝑔 𝑝 = 𝑧 𝑝 (𝜔) + 𝑛 𝑝 , with kernel k. We use GP (0 𝑐 , 𝑘 (•, •)) as a prior distribution over 𝑓 𝑝 . Given a set of sampling points 𝐴 𝑇 = [𝜔 1 , ..., 𝜔 𝑇 ] within 𝐶, observed rewards
, where the mean and variance are:
with 𝑘 𝑝 (𝜔) = [𝑘 (𝜔 1 , 𝜔), ..., 𝑘 (𝜔 𝑇 , 𝜔)] 𝑇 , 𝐾 𝑝 = [𝑘 (𝜔, 𝜔 ′ )] 𝜔,𝜔 ′ ∈𝐴 𝑝 , and I is the identity matrix. Similarly, for the constraint 𝑧 𝑝 , we consider the posterior GP ( μ𝑝 , σ2 𝑝 ) where 𝑟 1:𝑝 is replaced by 𝑔 1:𝑝 , and the kernel 𝑘 is replaced by k.
We assume that 𝑓 𝑝 belongs to RKHS H 𝑘 . In particular, H 𝑘 is equipped with the kernel 𝑘 such that 𝑓 𝑝 (𝜔) = ⟨𝑓 𝑝 (•), 𝑘 (𝜔, •)⟩ H 𝑘 . Similarly, we assume that the constraint function 𝑧 𝑝 also belongs to H k , i.e., 𝑧 𝑝 (𝜔) = ⟨𝑧 𝑝 (•), k (𝜔, •)⟩ H k . Some examples of kernel functions are square exponential, Matern etc., [36].
Throughout the rest of this paper, we assume that the functions are bounded, i.e., ||𝑓 𝑝 (𝑥)|| H 𝑘 ≤ 𝐹 , and ||𝑔 𝑝 (𝑥)|| H k ≤ 𝐺 for all 𝑝. Such assumptions are also common in practice in the wireless communication [12,37].
6.1.3 Kernel Selection. We employ the Matern kernel to specify the RKHS in GP [36] as it shows the best performance:
Here, 𝑣 > 0 is the hyperparameter that controls the smoothness of the output, 𝑠 = 𝑑 (𝜔, 𝜔 ′ ) encodes the similarities between two arms with the Euclidean distance, 𝐵(𝑣) and Γ(𝑣) are the modified Bessel function and the gamma function, respectively.
We now discuss the base algorithm (inspired from [12]) which we use to find the beamforming vector in the static case. This algorithm also forms the basis in the non-stationary case as well where the robots are mobile.
). Since we do not know 𝑓 𝑝 and 𝑧 𝑝 , rather we are learning. We only get feedback (noisy) corresponding to selecting the beamforming vector 𝜔; we need to balance between the exploration and exploitation carefully. For the unconstrained GP-bandit, GP-UCB algorithm is proposed [36] where the idea is to select the points that have a higher mean estimate reward (exploitation) or have a higher posterior variance (exploration as it does not have enough information). Similarly, we maintain the upper confidence of the 𝑓 𝑝 (at time 𝑝) as the following term f 𝑝 (𝜔) = 𝜇 𝑝 -1 (𝜔) + 𝛽 𝑝 -1 𝜎 𝑝 -1 (𝜔). 𝛽 𝑝 -1 is the weight factor:
where 𝛾 is the information gainfoot_0 and the 𝛿 is the confidence parameter. Please see [12,36] for details.
Primal-Dual: Unlike the unconstrained version [36], we consider a constrained optimization problem. Similar to [12], we consider the Lagrangian of Eq.10 as E[𝑟 𝑝 (𝜔 𝑝 )] -𝜙 (E[𝑔 𝑝 (𝜔 𝑝 )] -𝜌).
We, then seek to solve for the Lagrangian:
Hence, unlike the unconstrained version, we have to develop the UCB for the Lagrangian for a given dual variable 𝜙. Since noise is zero-mean,
We, thus, only need to find the lower confidence bound of 𝑧 𝑝 as we already obtained UCB for 𝑓 𝑝 , for which, we use ẑ𝑝
With probability 1 -𝛿, 𝑓 𝑝 (𝜔) ≤ f 𝑝 (𝜔), and 𝑧 𝑝 (𝜔) ≥ ẑ𝑝 (𝜔) (from [12]) ensuring that if we use f 𝑝 and ẑ𝑝 , it will be indeed UCB. For the static channel 𝑓 𝑝 and 𝑧 𝑝 are drawn from the GP with time invariant mean. We decide to choose the solution at time 𝑝 as:
After selecting a beamforming vector based on Eq.15, all the posterior mean and variance, 𝜇 𝑝 (𝑥), 𝜎 𝑝 (𝑥) and μ𝑝 (𝑥), σ𝑝 (𝑥) are updated through Eq.12 based on the received value. Finally, we update dual variable 𝜙 with the gradient descent in the dual domain 𝜙 = max{𝜙 + 𝜂 ( ẑ𝑝 (𝜔 𝑝 ) -𝜌), 0}, where 𝜂 is the learning rate Initialization: Reward 𝑅 (𝑘 ) = 0, constraint value 𝐺 (𝑘 ) = 0.
Set the probability for arms 𝑗 = 1, . . . , 𝐽 as
Choose arm 𝑗 according to the probability 𝑝 ( 𝑗 ).
Set the restart interval 𝑊 = 2 𝑗 .
8:
Run the base algorithm with the restart interval 𝑊 . 9:
Collect the total reward 𝑅 (𝑘 ) and the constraint value 𝐺 (𝑘 ) across the interval 𝐼 .
10:
for 𝑗 = 1, . . . , 𝐽 do 11:
Update 𝑌 = min(max(𝑌 + 𝜂 ′ (𝐺 (𝑘 )/𝐼 -𝜌 ), 0), 𝑌 𝑚𝑎𝑥 )
We also clip the dual variable at 𝜙 𝑚𝑎𝑥 . Please see Algorithm 1 in [12] for details of the Base algorithm.
6.2.2 Learning Metric. : For the static-case, we are interested in minimizing the regret and the violation:
The regret measures the sub-optimality gap between the reward following the optimal policy 𝜋 * , and the reward following the policy 𝜋 𝑝 at time 𝑝. The violation measures the constraint violation at time 𝑝. Here, we seek to have sub-linear growth of 𝑅(𝑇 ) and 𝑉 (𝑇 ), i.e., 𝑅(𝑇 )/𝑇 → 0 as 𝑇 → ∞ as it will ensure that in most of the episodes, the policy is feasible and optimal. The following result signifies that the base algorithm indeed achieves the sub-linear regret and violation.
Proposition 1 ([12]). With probability 1 -𝛿, the base algorithm achieves 𝑅(𝑇 ) ≤ Õ (𝑇 1/2 ), 𝑉 (𝑇 ) ≤ Õ (𝑇 1/2 ).
We now discuss how we modify the base algorithm to address the non-stationarity. Our key contribution is adapting the 'bandit over bandit' approach inspired from [38] to the constrained GPbandit. We first quantify non-stationarity, discuss existing metricdependent methods, and finally introduce our novel approach that removes this dependency.
6.3.1 Time-varying Budget. Since our scenario is mobile, the channel conditions are time-varying. These time-varying channels affect reward/utility; thereby, 𝑓 𝑝 and 𝑧 𝑝 also vary over time. We assume that these variations are bounded by 𝐵 𝑓 and 𝐵 𝑧 . In particular, 𝐵 𝑓 :=
𝑇 -1 𝑝=1 max 𝑥 𝑓 𝑝 -𝑓 𝑝+1 H 𝑘 and 𝐵 𝑧 := 𝑇 -1 𝑝=1 max 𝑥 𝑧 𝑝 -𝑧 𝑝+1 H 𝑘 . The total combined variation budget would be 𝐵 = max{𝐵 𝑓 , 𝐵 𝑧 }. 6.3.2 Restart Strategy. Similar to [12], to combat non-stationary conditions, we adopt the restart strategy, which resets the kernels and forgets previous observations, no longer useful for deciding the new beamforming vector as perhaps the environment has changed. Restart enables efficient adaptation by discarding outdated observations in non-stationary environments. Note that instead of restart, one can employ sliding window, or weight-based algorithm. The key algorithmic contribution is selecting the restart interval 𝑊 which we explain in the following. 6.3.3 Unknown Variation Budget. Estimating the variation budgets 𝐵 𝑓 and 𝐵 𝑧 in real time is challenging and channel-dependent; thus, especially in mobile settings, the restart window 𝑊 should be learned adaptively. We propose Algorithm 1, a bandit-over-bandit scheme that treats a candidate set 𝐽 = {2 𝑗 } ⌊0.5 log 2 𝑇 ⌋ 𝑗=1
of restart intervals as outer-loop arms. We partition the horizon 𝑇 into epochs of length 𝐼 = √ 𝑇 . At epoch 𝑘, EXP3 [39] selects an arm 𝑗 from weights 𝑤 𝑗 (𝑘) (updated via Eq. ( 18)), yielding 𝑊 = 2 𝑗 ; the base algorithm (Sec. 6.2) then runs for 𝐼 steps with restarts every 𝑊 . We aggregate reward 𝑅(𝑘) and interference 𝐺 (𝑘) over the epoch, update arm probabilities using Eq. ( 17), and update the dual variable via 𝑌 ← Π [0,𝑌 max ] (𝑌 + 𝜂 ′ (𝐺 (𝑘)/𝐼 -𝜌)). Arms that induce low reward and/or higher constraint violation (see, (Eq.18)) receive lower weight and are selected less often in subsequent epochs.
•Regret and Violation: Since the optimal policy can change over time in a non-stationary environment, we evaluate our algorithm using dynamic regret Dy -R(𝑇 ) and constraint violation 𝑉 (𝑇 ). We define the dynamic regret as:
The constraint violation metric 𝑉 (𝑇 ) is still given by Eq.16. Compared to regret, bounding dynamic-regret is fundamentally more challenging. Nevertheless, we obtain sub-linear dynamic-regret and violation bound. Theorem 1. Algorithm 1 achieves with probability 1 -𝛿,
The proof is in our technical report [40] owing to space constraint. [12] achieves dynamic regret and violation bound of Õ (𝐵𝑇 3/4 ) (Theorem 1 there) where one can choose the worst estimated budget if the budget is unknown. In contrast, our bound achieves Õ (𝐵 1/4 𝑇 3/4 ). Hence, we improve the dependency of the budget.
Note that [12] indicates that even if 𝐵 = 𝑜 (𝑇 1/4 ), one can have linear regret and violation. In contrast, our result still achieves sub-linear regret and sub-liner violation bound as long as the time-varying budget grows sub-linearly. [12] has also achieved Õ (𝐵 1/4 𝑇 3/4 ) dynamic regret and violation, however, it requires the knowledge of the budget 𝐵 (Corollary 2 there). Our result achieves the same bound without the information of 𝐵.
For an online setting, it is norm to assume that 𝑇 is known. If 𝑇 is unknown, one can use the doubling 'trick' [41], which scales the regret and violation bound by log(𝑇 ). In particular, one can choose 𝑇 = 2 0 , 2 1 , 2 2 , . . . , and run the algorithm until reaching the 𝑇 . (a) (b) (c) (d) Figure 7: Validation of codebook design; a) measured beam pattern at (-35°,-19°), b) after changing the orientation of RIS ( MR was outside of the coverage at (65°,-13°)); c) Comparison of REMARKABLE's SNR performance against multi-AP scneraio; and d) Comparing the performance of REMARKABLE and exhaustive search (ES) w.r.t selection accuracy and beam selection time.
As this is a real-time experiment, obtaining the best policy while running the algorithm is not feasible. Consequently, we conduct two consecutive epochs: one for beam selection using the proposed approach and another utilizing an exhaustive search technique to find the best policy.
In this study, the MRs travel along a pre-determined path and send their observations (RSS) to the AP. We first conduct the experiment with our base algorithm (GP-UCB-C) (without restart strategy). The results (Fig. 6c-6d) show that GP-UCB-C has a significant increase in TA-regret (DynR(𝑇 )/𝑇 ) due to changes in the channel, specifically starting around 300, even if with non-violated constraint. We then apply the restart strategy GP-UCB-RE proposed in [12] with an upper estimate on the budget. While this restart strategy fails but still beats the baseline as it ultimately restarts and the TA-regret starts decreasing. Finally, our proposed approach Algorithm 1 (GP-UCB-A) has the best performance (Fig. 6c-6d) compared to other two baselines as it can quickly adapt to the change in the environment.
In particular, our proposed approach has the lowest TA-regret even when the environment changes while also finding feasible solution.
The last experiment corresponds to a time-varying channel with different channel coherence times as we adjust the speed of the MR from min to the allowed max speed as in [0.15𝑚/𝑠, 0.3𝑚/𝑠, 0.45𝑚/𝑠]. Fig. 6e presents that our approach Alg.1 adaptively selects the best optimum beams over time-varying channel quickly which is evident from its TA-regret performance.
Here, we first compare REMARKABLE's SNR performance with the classical multi-AP deployment, and then evaluate REMARKABLE against classical methods in terms of the beam selection time, and selection accuracy from Sec. 6.4. For benchmarking, REMARKABLE is compared with exhaustive search (ES)-the widely adopted baseline in mmWave protocols (e.g., IEEE 802.11ad/ay)-to highlight the efficiency of our approach without resorting to full codebook scans, rather than against complex learning methods such as reinforcement learning, which demands extensive training. As shown in Fig. 7c, the MR suffers an SNR drop when encountering obstacles or interference, whereas REMARKABLE maintains reliable links with consistent SNR. Despite RIS-induced cascaded path loss, RE-MARKABLE consistently achieves high SNR levels, underscoring the robustness of the design. Furthermore, we define a term of time slot as the end-to-end latency of each selected beam, comprising: (i) the beam searching latency of the employed algorithm, including both selection and real-time execution; (ii) feedback latency between the MR and AP, where the MR reports the observed RSS to the AP; and (iii) control latency between the AP and the RIS, where the AP transmits the selected codewords to the RIS. The control latency for transmitting a single codeword to the RIS is 208 µs, the average feedback latency over the wireless backhaul is 8.1 ms, and the beam searching latency is approximately 23.8 ms for codebook CB1 (higher for ES). When using ES, 32 time slots are needed to examine all beams from 𝐶𝐵1 and select the one with the highest RSS, resulting in a guaranteed overhead delay. In contrast, REMARK-ABLE averages 18 time slots (based on multiple experiments) to find the best beam with our adaptive algorithm GP-UCB-A 1 . After first initiated restart, GP-UCB-A 2 achieves adaptation in only 10 slots, whereas ES 2 must re-scan. Finally, Fig. 7d shows that REMARK-ABLE achieves a 46.8% improvement in beam selection time while maintaining 94.2% accuracy. Note that this study primarily focuses on low-mobility scenarios, while high-mobility cases are left for future work with further latency optimization. Time Complexity Analysis: The codebook design (see Sec 4) is performed offline, which significantly supports scalability of the hardware. As we use gradient-descent, its time complexity is O (𝑆𝐼 𝑁 2 log 𝑁 ), where 𝑆 is the codebook size, 𝑁 2 the number of RIS elements, and 𝐼 the number of gradient descent iterations. At runtime, the codebook size affects the online complexity of the bandit algorithm. Specifically, computing the mean and variance requires matrix inversion with worst-case complexity 𝑂 (𝑊 2 ), where 𝑊 is the restart interval. Evaluating GP-UCB over 𝑆 codebook cardinality results in overall complexity 𝑂 (𝑆𝑊 2 ). Since 𝑊 ≤ √ 𝑇 , the runtime remains linear in both 𝑆 and 𝑇 . Through empirical RIS measurements, the runtime is 25% fraction of the total end-to-end latency reported in Sec 8.4. This implies that the practical runtime overhead is suitable for real-world environments. Designing faster algorithms and potentially hardware-accelerated solutions remains an important future direction, especially in high-mobility settings.
We demonstrate optimal beam selection for robot connectivity under interference-constrained, time-varying channels. Our RIS codebook enables selection without channel estimation, and a reconfigurable mechanism extends coverage. Using an adaptive banditover-bandit restart strategy, REMARKABLE safely learns optimal beams in dynamic conditions, achieving 46.8% faster selection and 94.2% accuracy-outperforming classical methods. Characterizing the lower bound on the dynamic regret and violation is an important future work. Extending this framework to multiple robots and multiple RIS also constitutes an important future research direction.
For matern kernel, 𝛾 𝑇 = 𝑇 𝑑 (𝑑+1) /(2𝑣+𝑑 (𝑑+1) ) log(𝑇 )