An official website of the United States government
Federated learning (FL) offers many benefits, such as better privacy preservation and less communication overhead for scenarios with frequent data generation. In FL, local models are trained on end-devices and then migrated to the network edge or cloud for global aggregation. This aggregated model is shared back with end-devices to further improve their local models. This iterative process continues until convergence is achieved. Although FL has many merits, it has many challenges. The prominent one is computing resource constraints. End-devices typically have fewer computing resources and are unable to learn well the local models. Therefore, split FL (SFL) was introduced to address this problem. However, enabling SFL is also challenging due to wireless resource constraints and uncertainties. We formulate a joint end-devices computing resources optimization, task-offloading, and resource allocation problem for SFL at the network edge. Our problem formulation has a mixed-integer non-linear programming problem nature and hard to solve due to the presence of both binary and continuous variables. We propose a double deep Q-network (DDDQN) and optimization-based solution. Finally, we validate the proposed method using extensive simulation results.
more »
« less
Federated Learning over Wireless Networks: A Band-limited Coordinated Descent Approach
We consider a many-to-one wireless architecture for federated learning at the network edge, where multiple edge devices collaboratively train a model using local data. The unreliable nature of wireless connectivity, together with constraints in computing resources at edge devices, dictates that the local updates at edge devices should be carefully crafted and compressed to match the wireless communication resources available and should work in concert with the receiver. Thus motivated, we propose SGD-based bandlimited coordinate descent algorithms for such settings. Specifically, for the wireless edge employing over-the-air computing, a common subset of k-coordinates of the gradient updates across edge devices are selected by the receiver in each iteration, and then transmitted simultaneously over k sub-carriers, each experiencing time-varying channel conditions. We characterize the impact of communication error and compression, in terms of the resulting gradient bias and mean squared error, on the convergence of the proposed algorithms. We then study learning-driven communication error minimization via joint optimization of power allocation and learning rates. Our findings reveal that optimal power allocation across different sub-carriers should take into account both the gradient values and channel conditions, thus generalizing the widely used water-filling policy. We also develop sub-optimal distributed solutions amenable to implementation.
more »
« less
- Award ID(s):
- 2003111
- PAR ID:
- 10299414
- Date Published:
- Journal Name:
- IEEE INFOCOM 2021 - IEEE Conference on Computer Communications
- Page Range / eLocation ID:
- 1 to 10
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Wireless charging coupled with computation offloading in edge networks offers a promising solution for realizing power-hungry and computation intensive applications on user-devices. We consider a multi-access edge computing (MEC) system with collocated MEC server and base-station/access point (AP), each equipped with a massive MIMO antenna array, supporting multiple users requesting data computation and wireless charging. The goal is to minimize the energy consumption for computation offloading and maximize the received energy at the user from wireless charging. The proposed solution is a novel two-stage algorithm employing nested descent algorithm, primal-dual subgradient and linear programming techniques to perform data partitioning and time allocation for computation offloading and design the optimal energy beamforming for wireless charging, all within MEC-AP transmit power and latency constraints. Algorithm results show that optimal energy beamforming significantly outperforms other schemes such as isotropic or directed charging without beam power allocation. Compared to binary offloading, data partition in partial offloading leads to lower energy consumption and more charging time, leading to better wireless charging performance. The charged energy over an extended period of multiple time-slots both with and without computation offloading can be substantial. Wireless charging from MEC-AP thus offers a viable untethered approach for supplying energy to user-devices.more » « less
-
null (Ed.)Federated learning (FL) is a highly pursued machine learning technique that can train a model centrally while keeping data distributed. Distributed computation makes FL attractive for bandwidth limited applications especially in wireless communications. There can be a large number of distributed edge devices connected to a central parameter server (PS) and iteratively download/upload data from/to the PS. Due to limited bandwidth, only a subset of connected devices can be scheduled in each round. There are usually millions of parameters in the state-of-art machine learning models such as deep learning, resulting in a high computation complexity as well as a high communication burden on collecting/distributing data for training. To improve communication efficiency and make the training model converge faster, we propose a new scheduling policy and power allocation scheme using non-orthogonal multiple access (NOMA) settings to maximize the weighted sum data rate under practical constraints during the entire learning process. NOMA allows multiple users to transmit on the same channel simultaneously. The user scheduling problem is transformed into a maximum-weight independent set problem that can be solved using graph theory. Simulation results show that the proposed scheduling and power allocation scheme can help achieve a higher FL testing accuracy in NOMA based wireless networks than other existing schemes within the same learning time.more » « less
-
null (Ed.)In this paper, we consider federated learning in wireless edge networks. Transmitting stochastic gradients (SG) or deep model's parameters over a limited-bandwidth wireless channel can incur large training latency and excessive power consumption. Hence, data compressing is often used to reduce the communication overhead. However, efficient communication requires the compression algorithm to satisfy the constraints imposed by the communication medium and take advantage of its characteristics, such as over-the-air computations inherent in wireless multiple-access channels (MAC), unreliable transmission and idle nodes in the edge network, limited transmission power, and preserving the privacy of data. To achieve these goals, we propose a novel framework based on Random Linear Coding (RLC) and develop efficient power management and channel usage techniques to manage the trade-offs between power consumption, communication bit-rate and convergence rate of federated learning over wireless MAC. We show that the proposed encoding/decoding results in an unbiased compression of SG, hence guaranteeing the convergence of the training algorithm without requiring error-feedback. Finally, through simulations, we show the superior performance of the proposed method over other existing techniques.more » « less
-
null (Ed.)Wireless charging coupled with computation offloading in edge networks offers a promising solution for realizing power-hungry and computation intensive applications on user devices. We consider a mutil-access edge computing (MEC) system with collocated MEC servers and base-stations/access points (BS/AP) supporting multiple users requesting data computation and wireless charging. We propose an integrated solution for wireless charging with computation offloading to satisfy the largest feasible proportion of requested wireless charging while keeping the total energy consumption at the minimum, subject to the MEC-AP transmit power and latency constraints. We propose a novel nested algorithm to jointly perform data partitioning, time allocation, transmit power control and design the optimal energy beamforming for wireless charging. Our resource allocation scheme offers a minimal energy consumption solution compared to other schemes while also delivering a higher amount of wirelessly transferred charge to the users. Even with data offloading, our proposed solution shows significant charging performance, comparable to the case of charging alone, hence showing the effectiveness of performing partial offloading jointly with wireless charging.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/INFOCOM42981.2021.9488818
