I. INTRODUCTION

Millimeter wave (mmWave) and Terahertz (THz) communication systems are essential components of 5G and beyond. To guarantee sufficient received signal power, these systems deploy large antenna arrays and use narrow beams. Due to the high cost and power consumption of high frequency RF chains, however, mmWave and THz communication systems can not use fully-digital architectures that assign an RF chain per antenna, and normally resort to analog-only or hybrid analog/digital architectures [1], [4]. Further, because of the hardware constraints on these large-scale MIMO systems, they typically adopt pre-defined single-lobe beamforming codebooks (such as DFT codebooks [2]) that scan all possible directions for both initial access and data transmission. These codebooks have two key limitations: (i) They incur high beam training overhead by scanning all possible directions even though many of these directions may never be used, and (ii) they normally have single-lobe beams which may not be optimal, especially in non-line-of-sight (NLOS) scenarios.

Contribution: In this paper, we consider hardwareconstrained large-scale MIMO systems and propose an artificial neural network based framework for learning environment aware beamforming codebooks. More specifically, we design a machine learning model that can learn how to adapt the patterns of the codebook beams based on the surrounding environment and user distribution. This is done by developing a novel complex-valued neural network architecture in which the weights directly model the beamforming/combining weights. Further, the developed neural network architecture accounts for the hardware constraints (such as phase-only, constant-modulus, and quantized-angle constraints) that are typically applied on the analog-only mmWave/THz transceiver architectures. We evaluate the performance of the proposed solution in both outdoor LOS and indoor NLOS communication scenarios. The simulation results highlight the capability of the proposed solution in learning multi-lobe beams that adapt to the environment geometry and to where the users are. Further, for the adopted NLOS simulation setup, the results show that the learned codebook can exceed the 64-beam DFT codebook with only 16 beams, which can lead to noticeable savings in the beam training overhead.

Prior work: Designing beamforming codebooks has been an important research topic for long time at both academia and industry (see [3] for a good overview). With largescale MIMO systems, however, the hardware limitations (especially at mmWave/THz) and the use of analog-only or hybrid transceiver architectures imposed new constraints on the codebook design problems. This motivated the development of new beamforming codebooks [1], [2], [4], [5]. These codebooks, however, are generally designed to have singlelobe narrow beams that cover all the angular directions and are not adaptive to the deployment, surrounding environment, and user distributions. This motivated the development of environment and hardware aware codebook learning strategies, which is the focus of this paper.

II. SYSTEM AND CHANNEL MODELS

In this paper, we consider a system model where a mmWave base station (BS), equipped with M antennas, is communicating with a single-antenna user. In the uplink transmission, if the user transmits a symbol x ∈ C to the BS, with the power constraint E |x| 2 = P , the received signal at the BS after combining can be expressed as

where h ∈ C M ×1 is the uplink channel vector between the mobile user and the BS, w is the receive combining vector at the BS and n ∼ N C 0, σ 2 n I is the receive noise vector at the BS. To reduce the cost and power consumption, the BS is assumed to employ analog-only transceiver architecture [6], where the beamforming/combining processing is implemented using analog phase shifters. To account for that, each element w m in the combining vector, m = 1, 2, ..., M , is assumed to have the structure w m = 1 √ M e jθm . We adopt a general geometric channel model for h [7]. Assume that the signal propagation between the mobile user and the BS consists of L paths. Each path has a complex gain α and an angle of arrival φ . Thus, the channel can be written as

where a(φ ) is the array response vector of the BS. The definition of a(φ ) depends on the array geometry.

III. PROBLEM DEFINITION

In this paper, we consider hardware-constrained large-scale MIMO systems and investigate the design of beamforming codebooks that are adaptive to the deployment scenario, environment geometry, user distributions, etc.-which we refer to as environment-aware beamforming codebooks.

Given the system and channel models in Section II, and due to the hardware constraints on the BS transceiver, the BSs normally adopt pre-defined beamforming codebooks. If W = {w 1 , ...., w N } denotes the BS beamforming codebook, then the maximum beamforming/combining gain of user u can be written as

Our objective in this paper is then to design the beamforming codebook W to maximize the maximum beamforming gain γ u averaged over the set of users served by this BS in the surrounding environment. Let H represent the set of channel vectors for the candidate users, then our objective can be written as

where |W| = N is the codebook size, and [N ] is the shorthand for representing set {1, 2, . . . , N }. The constraint ( 5) is imposed to uphold the phase-shifters constraint.

Constructing efficient beamforming codebooks has attracted significant interest from both academia and industry for the last two decades (see [3] for a good overview). With largescale MIMO systems, however, the hardware limitations, such as the constant modulus captured by (5) and the quantized phase shifters, add hard and non-convex constraints to the beam codebook optimization problems and make them difficult to solve [4]. In this paper, we make an initial step into leveraging the learning and optimization capabilities of neural networks to learn beam codebooks that are adaptive to the environment geometry, user distributions, etc., and that respect the hardware constraints on the large-scale MIMO systems. We believe that this direction of using neural networks to develop environment and hardware aware beam codebooks could lead in the future extensions to interesting solutions, especially when considering more complex system models such as interference-limited and multi-user MIMO.

IV. PROPOSED MACHINE LEARNING SOLUTION

In this section, we present our proposed beam codebook learning solution for hardware-constrained large-scale MIMO systems. First, we explain in Section IV-A the proposed neural network architecture and its relation to the optimization problem (4)-( 5), before going into the details of how a codebook is learned in Section IV-B.

A. Model Architecture

This proposed neural network architecture consists of three main components, as depicted in Fig. 1. Those components are the complex-valued fully-connected layer, the powercomputation layer and the max-pooling layer. A forward pass through these three layers is equivalent to evaluating the objective function of (4) over a single user's channel h.

1) Complex-valued fully-connected layer: The first layer consists of N neurons that are capable of performing complexvalued multiplication and summation. Each neuron, as shown in Fig. 1, learns one beamforming vector and performs inner product with the input channel vector. Formally, this is described by the following matrix multiplication

where W = [w 1 , . . . , w N ] ∈ C M ×N is the beamforming codebook matrix, h ∈ C M ×1 is a user's channel vector, and z ∈ C N ×1 is the vector of the combined received signal.

(6) could be re-written in the following equivalent real-valued block matrix form

where z r , z im are the real and imaginary parts of z, W r , W im are matrices containing the real and imaginary components of the elements of W, and similarly, h r , h im are the real and imaginary components of the channel vector h. Contrary to the norm in designing neural networks, the elements of the beamforming matrix W are not the weights of the fully-connected layer. Instead, they are derived from the actual neural network weights, which are the phases of the phased arrays. This is done through an embedded layer of phase-to-complex operations, as shown in Fig. 1. This layer transforms the phased arrays into unit-magnitude complex vectors by applying elements-wise cos and sin operations and scaling them by 1/ √ M as follows

where Θ = [θ 1 , . . . , θ N ] is an M × N matrix that includes the phases of the beamforming codebook, and

T , ∀n ∈ {1, . . . , N } is a single phase vector.

The use of this embedded layer is the network's way of learning beamforming vectors that respect the phase shifter constraint.

2) Power-computation layer: The output of the complexvalued fully-connected layer is then fed into the powercomputation layer. It performs element-wise absolute square operation and outputs a real-valued vector q given by q = [q 1 , q 2 , . . . ,

which contains the received power of each beamforming vector in the codebook.

3) Max-pooling layer: The power of the best beamforming vector is finally found by the last layer, the max-pooling layer. It performs the following element-wise max operation over the elements of q, that is

where g is the power of the best beamforming vector. This value is used to assess the quality of the codebook by comparing it to a desired receive power value. In the following subsection, we describe the details of the desired value and how the quality is assessed.

B. Learning Codebooks

With the learning architecture in mind, it is time to delve into the details of how a codebook is learned. Our proposed solution follows a supervised learning approach. In such approach, a machine learning model is trained using pairs of inputs and the desired responses, which constitute the dataset.

1) Dataset Description:

The inputs to the model are the users' channels as they are the communication quantity that drives the beamforming design process. For labels, there are many possible desired responses that could be used for training, and the choice between them should be made based on what the model needs to learn. In this paper, we adopt the equal gain combining (EGC) to construct the model label. This choice is based on the facts that EGC respects the phase shifters constraint and it is the beamforming strategy that achieves optimal beamforming gain when there are no restrictions on the codebook size or the quantized phase shifters. The EGC beamforming vector is constructed by using the phase component of every user's channel as f EGC = 1 √ M e ∠h , where ∠h stands for the vector of phases of a complex vector h. Using EGC beamforming vectors, the desired response for each user can be computed as follows

where • 1 is the L 1 norm of a vector. Putting the users' channels and their EGC gains together provides the training dataset S t . This is formally given by

where h u ∈ C M ×1 and p u ∈ R are the channel vector and EGC gain of the u-th user respectively, and U = |H| is the total number of collected data pairs.

2) Model Training: Using the set S t , the model is trained by undergoing multiple forward-backward cycles. In each cycle, a mini-batch of complex channel vectors and their EGC responses is sampled from the training set. The channels are fed sequentially to the model and a forward pass is performed as described in Section IV-A. For each channel vector in the batch, the model combines it with the currently available N beamforming vectors and outputs the power of the best beamforming vector for that channel. The quality of the best combiner is assessed by measuring how close its beamforming gain to that of the equal gain combiner, obtained by (11). A Mean Squared Error (MSE) loss is used as a metric to assess the quality of the codebook over the current mini-batch. Formally, it is defined as

where g b is the output of the max-pooling layer for the b-th data pair in the mini-batch, p b is the desired value achieved by (11) and B is the mini-batch size.

The error signal (derivative of the loss (13) with respect to each phase vector θ n ∈ Θ) is back propagated through the model to adjust the phases of the combining vectors [8] [9], making up what is usually referred to as the backward pass or backpropagation. This is formally expressed by the chain rule of differentiation Mathematically, ∂L ∂θn does not exist for two reasons: 1) the factor ∂g ∂q does not exist since the max-pooling function g is not differentiable, and 2) the factor ∂q ∂z does not satisfy the Cauchy-Riemann equations [10], meaning that q as a function of complex vector z is not complex differentiable (holomorphic). However, both issues could be resolved to enable backpropagation. The details of that are discussed in [11]. Computing the partial derivative of loss with respect to phase vector θ n allows the backward pass to modify the codebook Θ and make it adaptive to the environment. The update equation generally depends on the solver used to carry out the training, e.g., Stochastic Gradient Descent (SGD) and ADAptive Moment estimation (ADAM) to name two, but in its simplest form, it could be given by

where η is the optimization step size, commonly known as the learning rate in machine learning, and θ nnew , θ n old are the new and current n-th phase vector of the codebook respectively. Through the forward-backward cycles, the model learns to adapt its beamforming vectors according to the channels it experiences in the mini-batch. As mentioned in Section IV-A, a forward pass is equivalent to evaluating the objective function in (4). In a similar spirit, the backward pass through the architecture independently optimizes the beamforming vectors based on the group of channels that they receive best. It turns out that such group of channels that are best received by the same beamforming vector contribute only to the update of that beamforming vector and do not affect the others. This mainly comes as a result of the masking role that the max-pooling layer plays in the backward pass.

V. EXPERIMENTAL SETUP AND MODEL TRAINING

In this section, we describe the adopted scenarios, datasets and the machine learning parameters used in our simulation.

A. Scenario and Dataset

The performance of the proposed solution is evaluated on two different communication scenarios. The first is an outdoor scenario which addresses the situation when all users experience LOS connection with the base station, named LOS scenario. The second, on the other hand, is an indoor scenario and addresses the case when none of the users has a LOS connection, which is referred to as NLOS scenario. Both scenarios are for an operating frequency of 28GHz, and both are part of the scenarios in the DeepMIMO dataset [12] . Using the data generation script of DeepMIMO, two sets of channel data are generated, one for each scenario. Table I shows the parameters of channel data generation.

B. Model Training and Testing

The model is trained and tested by using both datasets described in Section V-A. Prior to any training session, the channels of each dataset is pre-processed to improve the training experience [9], which is a very common practice in machine learning. The pre-processing of choice for these experiments is data sample normalization. As in [7], [13], the channel normalization using the maximum absolute value in the training dataset helps the network undergo a stable and efficient training. Formally, the normalization factor is found as follows

where [h u ] m is the m-th element in the channel vector of the h u . Using the normalized channels, the model is trained on portion of the samples of the dataset and tested on the rest. For brevity, the data split percentage between training and testing along with other training hyper-parameters can be found at [14].

VI. SIMULATION RESULTS

In this section, we evaluate the performance of our proposed machine learning based beamforming codebook design solution on the scenarios described in Section V-A. The numerical results show that our proposed model can achieve performance that is comparable to EGC and adapt to different user distributions.

Fig. 3 shows the evaluation results of the proposed solution in a LOS scenario. With 64 beams, the learned codebook reaches more than 90% of the upper bound, which is achieved by the EGC where the number of beams in the codebook is equal to the number of users (and assuming unquantized phase shifters). Further, with 128 beams, the developed approach achieves more than 95% of the upper bound. Besides, Fig. 3 also exhibits the performance of the codebook with quantized phase shifters. It shows that with only 3-bit phase shifters, the learned codebook can still achieve more than 85% of the upper bound with 64 beams and 90% of the upper bound with 128 beams. This is very important and interesting for cases where the resolution of the analog phase shifters is limited. Fig. 4 shows the evaluation result of the proposed solution in a NLOS scenario. This is more interesting and challenging than the LOS case. In a NLOS scenario, each user may have multiple rays reflecting off some scatters and reaching the base station. As depicted in Fig. 2, the learned codebook with the developed approach is capable of learning multi-lobe beams that capture the dominant rays and gather more energy compared to classical DFT codebooks. Fig. 4 depicts the performance of the proposed solution in this NLOS environment. The learned codebook has almost the same performance or even outperforms a 64-beam DFT codebook with only 16 beams, which is one fourth in size. With 64 beams, the learned codebook achieves almost 90% of the upper bound and noticeable gain compared to the 64-beam DFT codebook.

VII. ACKNOWLEDGMENT

This work is supported by the National Science Foundation under Grant No. 1923676.

VIII. CONCLUSIONS AND DISCUSSIONS

In this paper, we developed a machine learning framework for learning environment and hardware aware beamforming codebooks for large-scale MIMO systems. In the developed solution, the neural networks incorporate the hardware constraints and learn how to adapt the beam shapes based on the surrounding environment and user distribution. Simulation results confirmed the capability of the proposed solution in learning beam codebooks that are optimized in the size and beam patterns. This leads to promising gains in terms of the achievable rates compared to classical DFT codebooks. In the future, it is interesting to extend the proposed machine learning framework to address more complicated scenarios such as multi-stream and multi-user communications.