skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Sim-to-Real Transfer in Multi-agent Reinforcement Networking for Federated Edge Computing
Federated Learning (FL) over wireless multi-hop edge computing networks, i.e., multi-hop FL, is a cost-effective distributed on-device deep learning paradigm. This paper presents FedEdge simulator, a high-fidelity Linux-based simulator, which enables fast prototyping, sim-to-real code, and knowledge transfer for multi-hop FL systems. FedEdge simulator is built on top of the hardware-oriented FedEdge experimental framework with a new extension of the realistic physical layer emulator. This emulator exploits trace-based channel modeling and dynamic link scheduling to minimize the reality gap between the simulator and the physical testbed. Our initial experiments demonstrate the high fidelity of the FedEdge simulator and its superior performance on sim-to-real knowledge transfer in reinforcement learning -optimized multi-hop FL  more » « less
Award ID(s):
2003198 2008447
PAR ID:
10340786
Author(s) / Creator(s):
; ; ; ; ;
Date Published:
Journal Name:
2021 IEEE/ACM Symposium on Edge Computing (SEC)
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract We present methods for emulating the matter power spectrum by combining information from cosmological N-body simulations at different resolutions. An emulator allows estimation of simulation output by interpolating across the parameter space of a limited number of simulations. We present the first implementation in cosmology of multi-fidelity emulation, where many low-resolution simulations are combined with a few high-resolution simulations to achieve an increased emulation accuracy. The power spectrum’s dependence on cosmology is learned from the low-resolution simulations, which are in turn calibrated using high-resolution simulations. We show that our multi-fidelity emulator predicts high-fidelity counterparts to percent-level relative accuracy when using only 3 high-fidelity simulations and outperforms a single-fidelity emulator that uses 11 simulations, although we do not attempt to produce a converged emulator with high absolute accuracy. With a fixed number of high-fidelity training simulations, we show that our multi-fidelity emulator is ≃ 100 times better than a single-fidelity emulator at k ≤ 2 hMpc−1, and ≃ 20 times better at 3 ≤ k < 6.4 hMpc−1. Multi-fidelity emulation is fast to train, using only a simple modification to standard Gaussian processes. Our proposed emulator shows a new way to predict non-linear scales by fusing simulations from different fidelities. 
    more » « less
  2. Recent advances in on-policy reinforcement learning (RL) methods enabled learning agents in virtual environments to master complex tasks with high-dimensional and continuous observation and action spaces. However, leveraging this family of algorithms in multi-fingered robotic grasping remains a challenge due to large sim-to-real fidelity gaps and the high sample complexity of on-policy RL algorithms. This work aims to bridge these gaps by first reinforcement-learning a multi-fingered robotic grasping policy in simulation that operates in the pixel space of the input: a single depth image. Using a mapping from pixel space to Cartesian space according to the depth map, this method transfers to the real world with high fidelity and introduces a novel attention mechanism that substantially improves grasp success rate in cluttered environments. Finally, the direct-generative nature of this method allows learning of multi-fingered grasps that have flexible end-effector positions, orientations and rotations, as well as all degrees of freedom of the hand. 
    more » « less
  3. Simulating soft robots in cluttered environments remains an open problem due to the challenge of capturing complex dynamics and interactions with the environment. Fur- thermore, fast simulation is desired for quickly exploring robot behaviors in the context of motion planning. In this paper, we examine a particular class of inflated-beam soft growing robots called “vine robots,” and present a dynamics simulator that captures general behaviors, handles robot-object interactions, and runs faster than real time. The simulator framework uses a simplified multi-link, rigid-body model with contact constraints. To bridge the sim-to-real gap, we develop methods for fitting model parameters based on video data of a robot in motion and in contact with an environment. We provide examples of simulations, including several with fit parameters, to show the qualitative and quantitative agreement between simulated and real behaviors. Our work demonstrates the capabilities of this high-speed dynamics simulator and its potential for use in the control of soft robots. 
    more » « less
  4. Knowledge graph reasoning plays a pivotal role in many real-world applications, such as network alignment, computational fact-checking, recommendation, and many more. Among these applications, knowledge graph completion (KGC) and multi-hop question answering over knowledge graph (Multi-hop KGQA) are two representative reasoning tasks. In the vast majority of the existing works, the two tasks are considered separately with different models or algorithms. However, we envision that KGC and Multi-hop KGQA are closely related to each other. Therefore, the two tasks will benefit from each other if they are approached adequately. In this work, we propose a neural model named BiNet to jointly handle KGC and multi-hop KGQA, and formulate it as a multi-task learning problem. Specifically, our proposed model leverages a shared embedding space and an answer scoring module, which allows the two tasks to automatically share latent features and learn the interactions between natural language question decoder and answer scoring module. Compared to the existing methods, the proposed BiNet model addresses both multi-hop KGQA and KGC tasks simultaneously with superior performance. Experiment results show that BiNet outperforms state-of-the-art methods on a wide range of KGQA and KGC benchmark datasets. 
    more » « less
  5. Abstract Energy 3D printing processes have enabled energy storage devices with complex structures, high energy density, and high power density. Among these processes, Freeze Nano Printing (FNP) has risen as a promising process. However, quality problems are among the biggest barriers for FNP. Particularly, the droplet solidification time in FNP governs thermal distribution, and subsequently determines product solidification, formation, and quality. To describe the solidification time, physical-based heat transfer model is built. But it is computationally intensive. The objective of this work is to build an efficient emulator for the physical model. There are several challenges unaddressed: 1) the solidification time at various locations, which is a multi-dimensional array response, needs to be modeled; 2) the construction and evaluation of the emulator at new process settings need to be quick and accurate. We integrate joint tensor decomposition and Nearest Neighbor Gaussian Process (NNGP) to construct an efficient multi-dimensional array response emulator with process settings as inputs. Specifically, structured joint tensor decomposition decomposes the multi-dimensional array responses at various process settings into the setting-specific core tensors and shared low dimensional factorization matrices. Then, each independent entry of the core tensor is modeled with an NNGP, which addresses the computationally intensive model estimation problem by sampling the nearest neighborhood samples. Finally, tensor reconstruction is performed to make predictions of solidification time for new process settings. The proposed framework is demonstrated by emulating the physical model of FNP, and compared with alternative tensor (multi-dimensional array) regression models. 
    more » « less