As a promising approach to deal with distributed data, Federated Learning (FL) achieves major advancements in recent years. FL enables collaborative model training by exploiting the raw data dispersed in multiple edge devices. However, the data is generally non-independent and identically distributed, i.e., statistical heterogeneity, and the edge devices significantly differ in terms of both computation and communication capacity, i.e., system heterogeneity. The statistical heterogeneity leads to severe accuracy degradation while the system heterogeneity significantly prolongs the training process. In order to address the heterogeneity issue, we propose an Asynchronous Staleness-aware Model Update FL framework, i.e., FedASMU, with two novel methods. First, we propose an asynchronous FL system model with a dynamical model aggregation method between updated local models and the global model on the server for superior accuracy and high efficiency. Then, we propose an adaptive local model adjustment method by aggregating the fresh global model with local models on devices to further improve the accuracy. Extensive experimentation with 6 models and 5 public datasets demonstrates that FedASMU significantly outperforms baseline approaches in terms of accuracy (0.60% to 23.90% higher) and efficiency (3.54% to 97.98% faster).
Active learning is a subfield of machine learning that focuses on improving the data collection efficiency in expensive-to-evaluate systems. Active learning-applied surrogate modeling facilitates cost-efficient analysis of demanding engineering systems, while the existence of heterogeneity in underlying systems may adversely affect the performance. In this article, we propose the partitioned active learning that quantifies informativeness of new design points by circumventing heterogeneity in systems. The proposed method partitions the design space based on heterogeneous features and searches for the next design point with two systematic steps. The global searching scheme accelerates exploration by identifying the most uncertain subregion, and the local searching utilizes circumscribed information induced by the local Gaussian process (GP). We also propose Cholesky update-driven numerical remedies for our active learning to address the computational complexity challenge. The proposed method consistently outperforms existing active learning methods in three real-world cases with better prediction and computation time.
more » « less- Award ID(s):
- 2035038
- PAR ID:
- 10483241
- Publisher / Repository:
- ASME
- Date Published:
- Journal Name:
- Journal of Computing and Information Science in Engineering
- Volume:
- 23
- Issue:
- 4
- ISSN:
- 1530-9827
- Page Range / eLocation ID:
- 041009
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
-
Federated learning has emerged as an important paradigm for training machine learning models in different domains. For graph-level tasks such as graph classification, graphs can also be regarded as a special type of data samples, which can be collected and stored in separate local systems. Similar to other domains, multiple local systems, each holding a small set of graphs, may benefit from collaboratively training a powerful graph mining model, such as the popular graph neural networks (GNNs). To provide more motivation towards such endeavors, we analyze real-world graphs from different domains to confirm that they indeed share certain graph properties that are statistically significant compared with random graphs. However, we also find that different sets of graphs, even from the same domain or same dataset, are non-IID regarding both graph structures and node features. To handle this, we propose a graph clustered federated learning (GCFL) framework that dynamically finds clusters of local systems based on the gradients of GNNs, and theoretically justify that such clusters can reduce the structure and feature heterogeneity among graphs owned by the local systems. Moreover, we observe the gradients of GNNs to be rather fluctuating in GCFL which impedes high-quality clustering, and design a gradient sequence-based clustering mechanism based on dynamic time warping (GCFL+). Extensive experimental results and in-depth analysis demonstrate the effectiveness of our proposed frameworks.more » « less
-
more » « less
Abstract Modern microprocessor performance is limited by local hot spots induced at high frequency by busy integrated circuit elements such as the clock generator. Locally embedded thermoelectric devices (TEDs) are proposed to perform active cooling whereby thermoelectric effects enhance passive cooling by the Fourier law in removing heat from the hot spot to colder regions. To mitigate transient heating events and improve temperature stability, we propose a novel analytical solution that describes the temperature response of a periodically heated hot spot that is actively cooled by a TED driven electrically at the same frequency. The analytical solution that we present is validated by experimental data from frequency domain thermal reflectance (FDTR) measurements made directly on an actively cooled Si thermoelectric device where the pump laser replicates the transient hot spot. We herein demonstrate a practical method to actively cancel the transient temperature variations on circuit elements with TEDs. This result opens a new path to optimize the design of cooling systems for transient localized hot spots in integrated circuits.
-
more » « less
Summary A salient feature of data from clinical trials and medical studies is inhomogeneity. Patients not only differ in baseline characteristics, but also in the way that they respond to treatment. Optimal individualized treatment regimes are developed to select effective treatments based on patient's heterogeneity. However, the optimal treatment regime might also vary for patients across different subgroups. We mainly consider patients’ heterogeneity caused by groupwise individualized treatment effects assuming the same marginal treatment effects for all groups. We propose a new maximin projection learning method for estimating a single treatment decision rule that works reliably for a group of future patients from a possibly new subpopulation. Based on estimated optimal treatment regimes for all subgroups, the proposed maximin treatment regime is obtained by solving a quadratically constrained linear programming problem, which can be efficiently computed by interior point methods. Consistency and asymptotic normality of the estimator are established. Numerical examples show the reliability of the methodology proposed.
-
Cellular materials widely exist in natural biologic systems such as honeycombs, bones, and woods. With advances in additive manufacturing, research on cellular metamaterials is emerging due to their unique mechanical performance. However, the design of on-demand cellular metamaterials usually requires solving a challenging inverse design problem for exploring complex structure–property relations of microstructured representative volume elements (RVEs) in the design domain. Here, we propose an experience-free and systematic methodology for exploring a parametrized system for microstructures of cellular mechanical metamaterials using a multiobjective genetic algorithm (GA). Globally, by considering the importance of the initial population selection for a population-based heuristic optimization method, we study the impact of the populations initialized by the different sampling methods on the optimal solutions. Locally, we develop our method by using a micro-GA with a new searching strategy, which requires the standard genetic algorithm to be conditionally run for a sufficient number of times with a small population size during the global searching process. We have applied our method to explore optimal solutions for applications mapped on two different parameter spaces of the cellular mechanical metamaterials with periodic and nonperiodic RVEs effectively and accurately.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1115/1.4056567
