skip to main content


Title: Reinforcement Learning of Millimeter Wave Beamforming Tracking over COSMOS Platform
Communication over large-bandwidth millimeter wave (mmWave) spectrum bands can provide high data rate, through utilizing highgain beamforming vectors (briefly, beams). Real-time tracking of such beams, which is needed for supporting mobile users, can be accomplished through developing machine learning (ML) models. While computer simulations were used to show the success of such ML models, experimental results are still limited. Consequently in this paper, we verify the effectiveness of mmWave beam tracking over the open-source COSMOS testbed. We particularly utilize a multi-armed bandit (MAB) scheme, which follows reinforcement learning (RL) approach. In our MAB-based beam tracking model, the beam selection is modeled as an action, while the reward of the algorithm is modeled through the link throughput. Experimental results, conducted over the 60-GHz COSMOS-based mobile platform, show that the MAB-based beam tracking learning model can achieve almost 92% throughput compared to the Genie-aided beams after a few learning samples.  more » « less
Award ID(s):
1816112
NSF-PAR ID:
10350753
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
WiNTECH'22: 16th ACM Workshop on Wireless Network Testbeds, Experimental evaluation & CHaracterization Proceedings
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Line-of-sight (LOS) is a critical requirement for mmWave wireless communications. In this work, we explore the use of access point (AP) infrastructure mobility to optimize indoor mmWave WiFi network performance based on the discovery of LOS connectivity to stations (STAs).We consider a ceiling-mounted mobile (CMM) AP as the infrastructure mobility framework. Within this framework, we present a LOS prediction algorithm based on machine learning (ML) that addresses the LOS discovery problem. The algorithm relies on the available network state information (e.g., LOS connectivity between STAs and the AP) to predict the unknown LOS connectivity status between the reachable AP locations and target STAs. We show that the proposed algorithm can predict LOS connectivity between the AP and target STAs with an accuracy up to 91%. Based on the LOS prediction algorithm, we then propose a systematic solution WiMove, which can decide if and where the AP should move to for optimizing network performance. Using both ns-3 based simulation and experimental prototype implementation, we show that the throughput and fairness performance of WiMove is up to 119% and 15% better compared with single static AP and brute force search. 
    more » « less
  2. Abstract

    Machine learning (ML) tools are able to learn relationships between the inputs and outputs of large complex systems directly from data. However, for time-varying systems, the predictive capabilities of ML tools degrade if the systems are no longer accurately represented by the data with which the ML models were trained. For complex systems, re-training is only possible if the changes are slow relative to the rate at which large numbers of new input-output training data can be non-invasively recorded. In this work, we present an approach to deep learning for time-varying systems that does not require re-training, but uses instead an adaptive feedback in the architecture of deep convolutional neural networks (CNN). The feedback is based only on available system output measurements and is applied in the encoded low-dimensional dense layers of the encoder-decoder CNNs. First, we develop an inverse model of a complex accelerator system to map output beam measurements to input beam distributions, while both the accelerator components and the unknown input beam distribution vary rapidly with time. We then demonstrate our method on experimental measurements of the input and output beam distributions of the HiRES ultra-fast electron diffraction (UED) beam line at Lawrence Berkeley National Laboratory, and showcase its ability for automatic tracking of the time varying photocathode quantum efficiency map. Our method can be successfully used to aid both physics and ML-based surrogate online models to provide non-invasive beam diagnostics.

     
    more » « less
  3. Abstract—Millimeter wave wireless spectrum deployments will allow vehicular communications to share high data rate vehicular sensor data in real-time.The highly directional nature of wireless links in millimeter spectral bands will require continuous channel measurements to ensure the transmitter (TX) and receiver (RX) beams are aligned to provide the best channel. Using real-world vehicular mmWave measurement data at 28GHz, we determine the optimal beam sweeping period, i.e. the frequency of the channel measurements,to align the RX beams to the best channel directions for maximizing the vehicle-to-infrastructure (V2I) throughput.We show that in a realistic vehicular traffic environment in Austin,TX, for a vehicle traveling at an average speed of 10.5mph,a beam sweeping period of 300 ms in future V2I communication standards would maximize theV2I throughput,using a system of four RX phased arrays that scanned the channel 360 degrees in the azimuth and 30 degrees above and below the boresight.We also investigate the impact of the number of active RX chains controlling the steerable phased arrays on V2I throughput. Reducing the number of RX chains controlling the phased arrays helps reduce the cost of the vehicular mmWave hardware while multiple RX chains, although more expensive,provide more robustness to beam direction changes at the vehicle,allowing near maximum throughput over a wide range of beam sweep periods.We show that the overhead of utilizing one RX chain instead of four leads to a10% drop in mean V2I throughput over six non-line- of-sight runs in real traffic conditions, with each run being 10 to 20 seconds long over a distance of 40 to 90 meters. Index Terms—mmWave;beam management;channel sound- ing; phased arrays;V2X;V2V;5G;sidelink 
    more » « less
  4. Utilizing millimeter-wave (mmWave) frequencies for wireless communication in mobile systems is challenging since it requires continuous tracking of the beam direction. Recently, beam tracking techniques based on channel sparsity and/or Kalman filter-based techniques were proposed where the solutions use assumptions regarding the environment and device mobility that may not hold in practical scenarios. In this paper, we explore a machine learning-based approach to track the angle of arrival (AoA) for specific paths in realistic scenarios. In particular, we use a recurrent neural network (R-NN) structure with a modified cost function to track the AoA. We propose methods to train the network in sequential data, and study the performance of our proposed solution in comparison to an extended Kalman filter based solution in a realistic mmWave scenario based on stochastic channel model from the QuaDRiGa framework. Results show that our proposed solution outperforms an extended Kalman filter-based method by reducing the AoA outage probability, and thus reducing the need for frequent beam search. 
    more » « less
  5. There is much interest in integrating millimeter wave radios (mmWave) into wireless LANs and 5G cellular networks to benefit from their multi-GHz of available spectrum. Yet, unlike existing technologies, e.g., WiFi, mmWave radios require highly directional antennas. Since the antennas have pencil-beams, the transmitter and receiver need to align their beams before they can communicate. Existing systems scan the space to find the best alignment. Such a process has been shown to introduce up to seconds of delay and is unsuitable for wireless networks where an access point has to quickly switch between users and accommodate mobile clients. This paper presents Agile-Link, a new protocol that can find the best mmWave beam alignment without scanning the space. Given all possible directions for setting the antenna beam, Agile-Link provably finds the optimal direction in logarithmic number of measurements. Further, Agile-Link works within the existing 802.11ad standard for mmWave LAN, and can support both clients and access points. We have implemented Agile-Link in a mmWave radio and evaluated it empirically. Our results show that it reduces beam alignment delay by orders of magnitude. In particular, for highly directional mmWave devices operating under 802.11ad, the delay drops from over a second to 2.5 ms. 
    more » « less