An official website of the United States government
Federated learning (FL) offers many benefits, such as better privacy preservation and less communication overhead for scenarios with frequent data generation. In FL, local models are trained on end-devices and then migrated to the network edge or cloud for global aggregation. This aggregated model is shared back with end-devices to further improve their local models. This iterative process continues until convergence is achieved. Although FL has many merits, it has many challenges. The prominent one is computing resource constraints. End-devices typically have fewer computing resources and are unable to learn well the local models. Therefore, split FL (SFL) was introduced to address this problem. However, enabling SFL is also challenging due to wireless resource constraints and uncertainties. We formulate a joint end-devices computing resources optimization, task-offloading, and resource allocation problem for SFL at the network edge. Our problem formulation has a mixed-integer non-linear programming problem nature and hard to solve due to the presence of both binary and continuous variables. We propose a double deep Q-network (DDDQN) and optimization-based solution. Finally, we validate the proposed method using extensive simulation results.
more »
« less
Federated Learning over Wireless Networks: A Band-limited Coordinated Descent Approach
We consider a many-to-one wireless architecture for federated learning at the network edge, where multiple edge devices collaboratively train a model using local data. The unreliable nature of wireless connectivity, together with constraints in computing resources at edge devices, dictates that the local updates at edge devices should be carefully crafted and compressed to match the wireless communication resources available and should work in concert with the receiver. Thus motivated, we propose SGD-based bandlimited coordinate descent algorithms for such settings. Specifically, for the wireless edge employing over-the-air computing, a common subset of k-coordinates of the gradient updates across edge devices are selected by the receiver in each iteration, and then transmitted simultaneously over k sub-carriers, each experiencing time-varying channel conditions. We characterize the impact of communication error and compression, in terms of the resulting gradient bias and mean squared error, on the convergence of the proposed algorithms. We then study learning-driven communication error minimization via joint optimization of power allocation and learning rates. Our findings reveal that optimal power allocation across different sub-carriers should take into account both the gradient values and channel conditions, thus generalizing the widely used water-filling policy. We also develop sub-optimal distributed solutions amenable to implementation.
more »
« less
- Award ID(s):
- 2003111
- PAR ID:
- 10299414
- Date Published:
- Journal Name:
- IEEE INFOCOM 2021 - IEEE Conference on Computer Communications
- Page Range / eLocation ID:
- 1 to 10
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Wireless charging coupled with computation offloading in edge networks offers a promising solution for realizing power-hungry and computation intensive applications on user-devices. We consider a multi-access edge computing (MEC) system with collocated MEC server and base-station/access point (AP), each equipped with a massive MIMO antenna array, supporting multiple users requesting data computation and wireless charging. The goal is to minimize the energy consumption for computation offloading and maximize the received energy at the user from wireless charging. The proposed solution is a novel two-stage algorithm employing nested descent algorithm, primal-dual subgradient and linear programming techniques to perform data partitioning and time allocation for computation offloading and design the optimal energy beamforming for wireless charging, all within MEC-AP transmit power and latency constraints. Algorithm results show that optimal energy beamforming significantly outperforms other schemes such as isotropic or directed charging without beam power allocation. Compared to binary offloading, data partition in partial offloading leads to lower energy consumption and more charging time, leading to better wireless charging performance. The charged energy over an extended period of multiple time-slots both with and without computation offloading can be substantial. Wireless charging from MEC-AP thus offers a viable untethered approach for supplying energy to user-devices.more » « less
-
null (Ed.)Federated learning (FL) is a highly pursued machine learning technique that can train a model centrally while keeping data distributed. Distributed computation makes FL attractive for bandwidth limited applications especially in wireless communications. There can be a large number of distributed edge devices connected to a central parameter server (PS) and iteratively download/upload data from/to the PS. Due to limited bandwidth, only a subset of connected devices can be scheduled in each round. There are usually millions of parameters in the state-of-art machine learning models such as deep learning, resulting in a high computation complexity as well as a high communication burden on collecting/distributing data for training. To improve communication efficiency and make the training model converge faster, we propose a new scheduling policy and power allocation scheme using non-orthogonal multiple access (NOMA) settings to maximize the weighted sum data rate under practical constraints during the entire learning process. NOMA allows multiple users to transmit on the same channel simultaneously. The user scheduling problem is transformed into a maximum-weight independent set problem that can be solved using graph theory. Simulation results show that the proposed scheduling and power allocation scheme can help achieve a higher FL testing accuracy in NOMA based wireless networks than other existing schemes within the same learning time.more » « less
-
Over-the-air (OTA) federated learning (FL) has been well recognized as a scalable paradigm that exploits the waveform superposition of the wireless multiple-access channel to aggregate model updates simultaneously. Existing OTA-FL designs largely enforce zero-bias model updates by either assuming homogeneous wireless conditions (equal path loss across devices) or forcing zero bias updates to guarantee convergence. Under heterogeneous wireless scenarios, however, such unbiased designs are constrained by the worst-channel device and suffer from high variance in the updates. Moreover, prior analyses of biased OTA-FL largely focus on convex objectives, whereas most modern machine-learning models are highly non-convex. Motivated by these gaps, we study OTA-FL with stochastic gradient descent (SGD) for smooth non-convex objectives under wireless heterogeneity. We develop novel OTA-FL SGD updates that allow a structured, time-invariant model bias while facilitating reduced variance in the updates. We also derive a finite-time stationarity bound (expected time average squared gradient norm) that explicitly reveals a bias–variance trade-off. To optimize this trade-off, we pose a non-convex joint OTA power-control design and develop an efficient successive convex approximation (SCA) algorithm that requires only statistical CSI of the devices at the base station. Experiments on a non-convex image classification task validate the approach: the SCA-based design accelerates convergence via an optimized bias and improves generalization over prior OTA-FL baselines.more » « less
-
null (Ed.)Wireless charging coupled with computation offloading in edge networks offers a promising solution for realizing power-hungry and computation intensive applications on user devices. We consider a mutil-access edge computing (MEC) system with collocated MEC servers and base-stations/access points (BS/AP) supporting multiple users requesting data computation and wireless charging. We propose an integrated solution for wireless charging with computation offloading to satisfy the largest feasible proportion of requested wireless charging while keeping the total energy consumption at the minimum, subject to the MEC-AP transmit power and latency constraints. We propose a novel nested algorithm to jointly perform data partitioning, time allocation, transmit power control and design the optimal energy beamforming for wireless charging. Our resource allocation scheme offers a minimal energy consumption solution compared to other schemes while also delivering a higher amount of wirelessly transferred charge to the users. Even with data offloading, our proposed solution shows significant charging performance, comparable to the case of charging alone, hence showing the effectiveness of performing partial offloading jointly with wireless charging.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Conference Paper:
- https://doi.org/10.1109/INFOCOM42981.2021.9488818
