skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Reinforcement Learning of Millimeter Wave Beamforming Tracking over COSMOS Platform
Communication over large-bandwidth millimeter wave (mmWave) spectrum bands can provide high data rate, through utilizing highgain beamforming vectors (briefly, beams). Real-time tracking of such beams, which is needed for supporting mobile users, can be accomplished through developing machine learning (ML) models. While computer simulations were used to show the success of such ML models, experimental results are still limited. Consequently in this paper, we verify the effectiveness of mmWave beam tracking over the open-source COSMOS testbed. We particularly utilize a multi-armed bandit (MAB) scheme, which follows reinforcement learning (RL) approach. In our MAB-based beam tracking model, the beam selection is modeled as an action, while the reward of the algorithm is modeled through the link throughput. Experimental results, conducted over the 60-GHz COSMOS-based mobile platform, show that the MAB-based beam tracking learning model can achieve almost 92% throughput compared to the Genie-aided beams after a few learning samples.  more » « less
Award ID(s):
1816112
PAR ID:
10350753
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
WiNTECH'22: 16th ACM Workshop on Wireless Network Testbeds, Experimental evaluation & CHaracterization Proceedings
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Beamforming techniques are considered as essential parts to compensate severe path losses in millimeter-wave (mmWave) communications. In particular, these techniques adopt large antenna arrays and formulate narrow beams to obtain satisfactory received powers. However, performing accurate beam alignment over narrow beams for efficient link configuration by traditional standard defined beam selection approaches, which mainly rely on channel state information and beam sweeping through exhaustive searching, imposes computational and communications overheads. And, such resulting overheads limit their potential use in vehicle-to-infrastructure (V2I) and vehicle-to-vehicle (V2V) communications involving highly dynamic scenarios. In comparison, utilizing out-of-band contextual information, such as sensing data obtained from sensor devices, provides a better alternative to reduce overheads. This paper presents a deep learning-based solution for utilizing the multi-modality sensing data for predicting the optimal beams having sufficient mmWave received powers so that the best V2I and V2V line-of-sight links can be ensured proactively. The proposed solution has been tested on real-world measured mmWave sensing and communication data, and the results show that it can achieve up to 98.19% accuracies while predicting top-13 beams. Correspondingly, when compared to existing been sweeping approach, the beam sweeping searching space and time overheads are greatly shortened roughly by 79.67% and 91.89%, respectively which confirm a promising solution for beamforming in mmWave enabled communications. 
    more » « less
  2. There is much interest in integrating millimeter wave radios (mmWave) into wireless LANs and 5G cellular networks to benefit from their multi-GHz of available spectrum. Yet, unlike existing technologies, e.g., WiFi, mmWave radios require highly directional antennas. Since the antennas have pencil-beams, the transmitter and receiver need to align their beams before they can communicate. Existing systems scan the space to find the best alignment. Such a process has been shown to introduce up to seconds of delay and is unsuitable for wireless networks where an access point has to quickly switch between users and accommodate mobile clients. This paper presents Agile-Link, a new protocol that can find the best mmWave beam alignment without scanning the space. Given all possible directions for setting the antenna beam, Agile-Link provably finds the optimal direction in logarithmic number of measurements. Further, Agile-Link works within the existing 802.11ad standard for mmWave LAN, and can support both clients and access points. We have implemented Agile-Link in a mmWave radio and evaluated it empirically. Our results show that it reduces beam alignment delay by orders of magnitude. In particular, for highly directional mmWave devices operating under 802.11ad, the delay drops from over a second to 2.5 ms. 
    more » « less
  3. Millimeter Wave (mmWave) networks can deliver multi-Gbps wireless links that use extremely narrow directional beams. This provides us with a new opportunity to exploit spatial reuse in order to scale network throughput. Exploiting such spatial reuse, however, requires aligning the beams of all nodes in a network. Aligning the beams is a difficult process which is complicated by indoor multipath, which can create interference, as well as by the inefficiency of carrier sense at detecting interference in directional links. This paper presents BounceNet, the first many-to-many millimeter wave beam alignment protocol that can exploit dense spatial reuse to allow many links to operate in parallel in a confined space and scale the wireless throughput with the number of clients. Results from three millimeter wave testbeds show that BounceNet can scale the throughput with the number of clients to deliver a total network data rate of more than 39 Gbps for 10 clients, which is up to 6.6× higher than current 802.11 mmWave standards. 
    more » « less
  4. Utilizing millimeter-wave (mmWave) frequencies for wireless communication in mobile systems is challenging since it requires continuous tracking of the beam direction. Recently, beam tracking techniques based on channel sparsity and/or Kalman filter-based techniques were proposed where the solutions use assumptions regarding the environment and device mobility that may not hold in practical scenarios. In this paper, we explore a machine learning-based approach to track the angle of arrival (AoA) for specific paths in realistic scenarios. In particular, we use a recurrent neural network (R-NN) structure with a modified cost function to track the AoA. We propose methods to train the network in sequential data, and study the performance of our proposed solution in comparison to an extended Kalman filter based solution in a realistic mmWave scenario based on stochastic channel model from the QuaDRiGa framework. Results show that our proposed solution outperforms an extended Kalman filter-based method by reducing the AoA outage probability, and thus reducing the need for frequent beam search. 
    more » « less
  5. Abstract—Millimeter wave wireless spectrum deployments will allow vehicular communications to share high data rate vehicular sensor data in real-time.The highly directional nature of wireless links in millimeter spectral bands will require continuous channel measurements to ensure the transmitter (TX) and receiver (RX) beams are aligned to provide the best channel. Using real-world vehicular mmWave measurement data at 28GHz, we determine the optimal beam sweeping period, i.e. the frequency of the channel measurements,to align the RX beams to the best channel directions for maximizing the vehicle-to-infrastructure (V2I) throughput.We show that in a realistic vehicular traffic environment in Austin,TX, for a vehicle traveling at an average speed of 10.5mph,a beam sweeping period of 300 ms in future V2I communication standards would maximize theV2I throughput,using a system of four RX phased arrays that scanned the channel 360 degrees in the azimuth and 30 degrees above and below the boresight.We also investigate the impact of the number of active RX chains controlling the steerable phased arrays on V2I throughput. Reducing the number of RX chains controlling the phased arrays helps reduce the cost of the vehicular mmWave hardware while multiple RX chains, although more expensive,provide more robustness to beam direction changes at the vehicle,allowing near maximum throughput over a wide range of beam sweep periods.We show that the overhead of utilizing one RX chain instead of four leads to a10% drop in mean V2I throughput over six non-line- of-sight runs in real traffic conditions, with each run being 10 to 20 seconds long over a distance of 40 to 90 meters. Index Terms—mmWave;beam management;channel sound- ing; phased arrays;V2X;V2V;5G;sidelink 
    more » « less