skip to main content


Title: Second-Best Beam-Alignment via Bayesian Multi-Armed Bandits
Millimeter-wave (mm-wave) systems rely on narrow- beams to cope with the severe signal attenuation in the mm- wave frequency band. However, susceptibility to beam mis- alignment due to mobility or blockage requires the use of beam- alignment schemes, with huge cost in terms of overhead and use of system resources. In this paper, a beam-alignment scheme is proposed based on Bayesian multi-armed bandits, with the goal to maximize the alignment probability and the data-communication throughput. A Bayesian approach is proposed, by considering the state as a posterior distribution over angles of arrival (AoA) and of departure (AoD), given the history of feedback signaling and of beam pairs scanned by the base-station (BS) and the user- end (UE). A simplified sufficient statistic for optimal control is identified, in the form of preference of BS-UE beam pairs. By bounding a value function, the second-best preference policy is formulated, which strikes an optimal balance between exploration and exploitation by selecting the beam pair with the current second-best preference. Through Monte-Carlo simulation with analog beamforming, the superior performance of the second- best preference policy is demonstrated in comparison to existing schemes based on first-best preference, linear Thompson sampling, and upper confidence bounds, with up to 7%, 10% and 30% improvements in alignment probability, respectively.  more » « less
Award ID(s):
1642982
NSF-PAR ID:
10195594
Author(s) / Creator(s):
;
Date Published:
Journal Name:
IEEE Global Communications Conference (GLOBECOM)
Page Range / eLocation ID:
1 to 6
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Beam alignment is a critical aspect in millimeter wave (mm-wave) cellular systems. However, the inherent limitations of channel estimation result in beam alignment errors, which degrade the system performance. For systems with a large number of antennas at the base station, downlink channel estimation is performed using uplink pilot signals. The beam alignment errors, thus, depend on the user equipment (UE) transmit power, which needs to be managed properly as the UEs are battery powered. This paper investigates how the use of uplink power control for the transmission of pilot signals in a mm-wave network affects the downlink beam alignment errors, which depend on various link parameters. We use stochastic geometry and statistics of the Student's t -distribution to develop an analytical model, which captures the interplay between the uplink power control and downlink signal-to-noise ratio (SNR) coverage probability. Our results indicate that using uplink power control significantly reduces UE power consumption without adversely affecting the downlink SNR coverage. 
    more » « less
  2. This paper studies the effects of millimeter-wave (mm-wave) beam alignment errors on the downlink achievable rate of a heterogeneous network (HetNet), which consists of sub-6 GHz macro-cells and mm-wave small-cells. The alignment error is modeled as a function of the underlying mm-wave link parameters. The conventional maximum biased received power criterion, where the bias is used for mm-wave small-cells, is adopted for cell associations. By varying the value of the bias factor, we investigate the changes in the downlink rate coverage probability. Our simulation results indicate that high values (of the order of 30 dB) for the bias, while beneficial in the case of perfect alignment, are actually disadvantageous for the low-rate users in the case of imperfect beam alignment. The low-rate users are better served by a moderate value (of the order of 20 dB) of the bias when the beam alignment errors are accounted for. We also show that the above disparity can be narrowed down by increasing by mm-wave base station (BS) antennas and/or the mm-wave BS density. 
    more » « less
  3. We propose a novel analytical framework for evaluating the coverage performance of a millimeter wave (mmWave) cellular network where idle user equipments (UEs) act as relays. In this network, the base station (BS) adopts either the direct mode to transmit to the destination UE, or the relay mode if the direct mode fails, where the BS transmits to the relay UE and then the relay UE transmits to the destination UE. To address the drastic rotational movements of destination UEs in practice, we propose to adopt selection combining at destination UEs. New expression is derived for the signal-to-interference-plus-noise ratio (SINR) coverage probability of the network. Using numerical results, we first demonstrate the accuracy of our new expression. Then we show that ignoring spatial correlation, which has been commonly adopted in the literature, leads to severe over estimation of the SINR coverage probability. Furthermore, we show that introducing relays into a mmWave cellular network vastly improves the coverage performance. In addition, we show that the optimal BS density maximizing the SINR coverage probability can be determined by using our analysis. 
    more » « less
  4. To overcome the high pathloss and the intense shadowing in millimeterwave (mmWave) communications, effective beamforming schemes are required which incorporate narrow beams with high beamforming gains. The mm Wave channel consists of a few spatial clusters each associated with an angle of departure (AoD). The narrow beams must be aligned with the channel AoDs to increase the beamforming gain. This is achieved through a procedure called beam alignment (BA). Most of the BA schemes in the literature consider channels with a single dominant path while in practice the channel has a few resolvable paths with different AoDs, hence, such BA schemes may not work correctly in the presence of multi-path or at the least do not exploit such multi path to achieve diversity or increase robustness. In this paper, we propose an efficient BA schemes in presence of multi-path. The proposed BA scheme transmits probing packets using a set of scanning beams and receives the feedback for all the scanning beams at the end of probing phase from each user. We formulate the BA scheme as minimizing the expected value of the average transmission beamwidth under different policies. The policy is defined as a function from the set of received feedback to the set of transmission beams (TB). In order to maximize the number of possible feedback sequences, we prove that the set of scanning beams (SB) has an special form, namely, Tulip Design. Consequently, we rewrite the minimization problem with a set of linear constraints and reduced number of variables which is solved by using an efficient greedy algorithm. 
    more » « less
  5. null (Ed.)
    Purpose : Personalized screening guidelines can be an effective strategy to prevent diabetic retinopathy (DR)-related vision loss. However, these strategies typically do not capture behavior-based factors such as a patient’s compliance or cost preferences. This study develops a mathematical model to identify screening policies that capture both DR progression and behavioral factors to provide personalized recommendations. Methods : A partially observable Markov decision process model (POMDP) is developed to provide personalized screening recommendations. For each patient, the model estimates the patient’s probability of having a sight-threatening diabetic eye disorder (STDED) yearly via Bayesian inference based on natural history, screening results, and compliance behavior. The model then determines a personalized, threshold-based recommendation for each patient annually--either no action (NA), teleretinal imaging (TRI), or clinical screening (CS)--based on the patient’s current probability of having STDED as well as patient-specific preference between cost saving ($) and QALY gain. The framework is applied to a hypothetical cohort of 40-year-old African American male patients. Results : For the base population with TRI and CS compliance rates of 65% and 55% and equal preference for cost and QALY, NA is identified as an optimal recommendation when the patient’s probability of having STDED is less than 0.72%, TRI when the probability is [0.72%, 2.09%], and CS when the probability is above 2.09%. Simulated against annual clinical screening, the model-based policy finds an average decrease of 7.07% in cost/QALY (95% CI; 6.93-7.23%) and 15.05% in blindness prevalence over a patient’s lifetime (95% CI; 14.88-15.23%). For patients with equal preference for cost and QALY, the model identifies 6 different types of threshold-based policies (See Fig 1). For patients with strong preference for QALY gain, CS-only policies had an increase in prevalence by a factor of 19.2 (see Fig 2). Conclusions : The POMDP model is highly flexible and responsive in incorporating behavioral factors when providing personalized screening recommendations. As a decision support tool, providers can use this modeling framework to provide unique, catered recommendations. 
    more » « less