More Like this

1. Quantifying Individual Fairness in Continual Federated Learning

   https://doi.org/10.1145/3701716.3717646  

   Noor, Naima; Mashhadi, Afra (May 2025, ACM)

   Continual Federated Learning (CFL) is a distributed machine learning technique that enables multiple clients to collaboratively train a shared model without sharing their data, while also adapting to new classes without forgetting previously learned ones. This dynamic, adaptive learning process parallels the concept of founda- tion models in FL, where large, pre-trained models are fine-tuned in a decentralized, federated setting. While foundation models in FL leverage pre-trained knowledge as a starting point, CFL continu- ously updates the shared model as new tasks and data distributions emerge, requiring ongoing adaptation. Currently, there are limited evaluation models and metrics in measuring fairness in CFL, and ensuring fairness over time can be challenging as the system evolves. To address this challenge, this article explores temporal fairness in CFL, examining how the fairness of the model can be influenced by the selection and participation of clients over time. Based on individual fairness, we introduce a novel fairness metric that captures temporal aspects of client behavior and evaluates different client selection strategies for their impact on promoting fairness. 

   more » « less
2. Maximizing Global Model Appeal in Federated Learning

   Cho, Yae Jee; Jhunjhunwala, Divyansh; Li, Tian; Smith, Virginia; Joshi, Gauri (March 2024, Transactions on machine learning research)

   Bellet, Aurelien (Ed.)

   Federated learning (FL) aims to collaboratively train a global model using local data from a network of clients. To warrant collaborative training, each federated client may expect the resulting global model to satisfy some individual requirement, such as achieving a certain loss threshold on their local data. However, in real FL scenarios, the global model may not satisfy the requirements of all clients in the network due to the data heterogeneity across clients. In this work, we explore the problem of global model appeal in FL, which we define as the total number of clients that find that the global model satisfies their individual requirements. We discover that global models trained using traditional FL approaches can result in a significant number of clients unsatisfied with the model based on their local requirements. As a consequence, we show that global model appeal can directly impact how clients participate in training and how the model performs on new clients at inference time. Our work proposes MaxFL, which maximizes the number of clients that find the global model appealing. MaxFL achieves a 22-40% and 18-50% improvement in the test accuracy of training clients and (unseen) test clients respectively, compared to a wide range of FL approaches that tackle data heterogeneity, aim to incentivize clients, and learn personalized/fair models. 

   more » « less
3. Tackling Intertwined Data and Device Heterogeneities in Federated Learning with Unlimited Staleness

   https://doi.org/10.1609/aaai.v39i20.35405  

   Wang, Haoming; Gao, Wei (April 2025, Proceedings of the AAAI Conference on Artificial Intelligence)

   Federated Learning (FL) can be affected by data and device heterogeneities, caused by clients' different local data distributions and latencies in uploading model updates (i.e., staleness). Traditional schemes consider these heterogeneities as two separate and independent aspects, but this assumption is unrealistic in practical FL scenarios where these heterogeneities are intertwined. In these cases, traditional FL schemes are ineffective, and a better approach is to convert a stale model update into a unstale one. In this paper, we present a new FL framework that ensures the accuracy and computational efficiency of this conversion, hence effectively tackling the intertwined heterogeneities that may cause unlimited staleness in model updates. Our basic idea is to estimate the distributions of clients' local training data from their uploaded stale model updates, and use these estimations to compute unstale client model updates. In this way, our approach does not require any auxiliary dataset nor the clients' local models to be fully trained, and does not incur any additional computation or communication overhead at client devices. We compared our approach with the existing FL strategies on mainstream datasets and models, and showed that our approach can improve the trained model accuracy by up to 25% and reduce the number of required training epochs by up to 35%. 

   more » « less
4. Federated Few-shot Learning

   https://doi.org/10.1145/3580305.3599347  

   Wang, Song; Fu, Xingbo; Ding, Kaize; Chen, Chen; Chen, Huiyuan; Li, Jundong (August 2023, ACM)

   Federated Learning (FL) enables multiple clients to collaboratively learn a machine learning model without exchanging their own local data. In this way, the server can exploit the computational power of all clients and train the model on a larger set of data samples among all clients. Although such a mechanism is proven to be effective in various fields, existing works generally assume that each client preserves sufficient data for training. In practice, however, certain clients can only contain a limited number of samples (i.e., few-shot samples). For example, the available photo data taken by a specific user with a new mobile device is relatively rare. In this scenario, existing FL efforts typically encounter a significant performance drop on these clients. Therefore, it is urgent to develop a few-shot model that can generalize to clients with limited data under the FL scenario. In this paper, we refer to this novel problem as federated few-shot learning. Nevertheless, the problem remains challenging due to two major reasons: the global data variance among clients (i.e., the difference in data distributions among clients) and the local data insufficiency in each client (i.e., the lack of adequate local data for training). To overcome these two challenges, we propose a novel federated few-shot learning framework with two separately updated models and dedicated training strategies to reduce the adverse impact of global data variance and local data insufficiency. Extensive experiments on four prevalent datasets that cover news articles and images validate the effectiveness of our framework compared with the state-of-the-art baselines. 

   more » « less
5. Gradient Inversion with Generative Image Prior

   Jeon, Jiwnoo; Kim, Jaechang; Lee, Kangwook; Oh, Sewoong; Ok, Jungseul (January 2021, Advances in neural information processing systems)

   Ranzato, M.; Beygelzimer, A.; Liang, P.S.; Vaughan, J.W.; Dauphin, Y. (Ed.)

   Federated Learning (FL) is a distributed learning framework, in which the local data never leaves clients’ devices to preserve privacy, and the server trains models on the data via accessing only the gradients of those local data. Without further privacy mechanisms such as differential privacy, this leaves the system vulnerable against an attacker who inverts those gradients to reveal clients’ sensitive data. However, a gradient is often insufficient to reconstruct the user data without any prior knowledge. By exploiting a generative model pretrained on the data distribution, we demonstrate that data privacy can be easily breached. Further, when such prior knowledge is unavailable, we investigate the possibility of learning the prior from a sequence of gradients seen in the process of FL training. We experimentally show that the prior in a form of generative model is learnable from iterative interactions in FL. Our findings demonstrate that additional mechanisms are necessary to prevent privacy leakage in FL. 

   more » « less