More Like this

1. Bayesian Optimization for Task Offloading and Resource Allocation in Mobile Edge Computing

   https://doi.org/10.1109/IEEECONF56349.2022.10051868  

   Yan, Jia ; Lu, Qin ; Giannakis, Georgios B. ( October 2022 , Asilomar Conference on Signals Systems and Computers)

   Recent years have witnessed the emergence of mobile edge computing (MEC), on the premise of a costeffective enhancement in the computational ability of hardware-constrained wireless devices (WDs) comprising the Internet of Things (IoT). In a general multi-server multi-user MEC system, each WD has a computational task to execute and has to select binary (off)loading decisions, along with the analog-amplitude resource allocation variables in an online manner, with the goal of minimizing the overall energy-delay cost (EDC) with dynamic system states. While past works typically rely on the explicit expression of the EDC function, the present contribution considers a practical setting, where in lieu of system state information, the EDC function is not available in analytical form, and instead only the function values at queried points are revealed. Towards tackling such a challenging online combinatorial problem with only bandit information, novel Bayesian optimization (BO) based approach is put forth by leveraging the multi-armed bandit (MAB) framework. Per time slot, by exploiting temporal information, the discrete offloading decisions are first obtained via the MAB method, and the analog resource allocation variables are subsequently optimized using the BO selection rule. Numerical tests validate the effectiveness of the proposed BO approach. 

   more » « less
2. Online learning for multi-agent based resource allocation in weakly coupled wireless systems

   https://doi.org/10.1145/3492866.3549714  

   Song, Jianhan ; de Veciana, Gustavo ; Shakkottai, Sanjay ( October 2022 , ACM Mobihoc '22)

   We propose and evaluate a learning-based framework to address multi-agent resource allocation in coupled wireless systems. In particular we consider, multiple agents (e.g., base stations, access points, etc.) that choose amongst a set of resource allocation options towards achieving their own performance objective /requirements, and where the performance observed at each agent is further coupled with the actions chosen by the other agents, e.g., through interference, channel leakage, etc. The challenge is to find the best collective action. To that end we propose a Multi-Armed Bandit (MAB) framework wherein the best actions (aka arms) are adaptively learned through online reward feedback. Our focus is on systems which are "weakly-coupled" wherein the best arm of each agent is invariant to others' arm selection the majority of the time - this majority structure enables one to develop light weight efficient algorithms. This structure is commonly found in many wireless settings such as channel selection and power control. We develop a bandit algorithm based on the Track-and-Stop strategy, which shows a logarithmic regret with respect to a genie. Finally through simulation, we exhibit the potential use of our model and algorithm in several wireless application scenarios. 

   more » « less
3. An efficient algorithm for generalized linear bandit: Online stochastic gradient descent and thompson sampling

   Ding, Qin ; Hsieh, Cho-Jui ; Sharpnack, James ( January 2021 , Proceedings of Machine Learning Research)

   Banerjee, Arindam ; Fukumizu, Kenji (Ed.)

   We consider the contextual bandit problem, where a player sequentially makes decisions based on past observations to maximize the cumulative reward. Although many algorithms have been proposed for contextual bandit, most of them rely on finding the maximum likelihood estimator at each iteration, which requires 𝑂(𝑡) time at the 𝑡-th iteration and are memory inefficient. A natural way to resolve this problem is to apply online stochastic gradient descent (SGD) so that the per-step time and memory complexity can be reduced to constant with respect to 𝑡, but a contextual bandit policy based on online SGD updates that balances exploration and exploitation has remained elusive. In this work, we show that online SGD can be applied to the generalized linear bandit problem. The proposed SGD-TS algorithm, which uses a single-step SGD update to exploit past information and uses Thompson Sampling for exploration, achieves 𝑂̃ (𝑇‾‾√) regret with the total time complexity that scales linearly in 𝑇 and 𝑑, where 𝑇 is the total number of rounds and 𝑑 is the number of features. Experimental results show that SGD-TS consistently outperforms existing algorithms on both synthetic and real datasets. 

   more » « less
4. Deep Reinforcement Learning for Secure MEC Service in Vehicular Networks with Reconfigurable Intelligent Surfaces

   https://doi.org/10.1109/IEEECONF59524.2023.10476897  

   Wang, Haoyu ; Bai, Haowen ; Ju, Ying ; Swindlehurst, A Lee ( October 2023 , Proc. 57th Asilomar Conference on Signals, Systems, and Computers)

   The broadcasting nature of wireless signals may result in the task offloading process of mobile edge computing (MEC) suffering serious information leakage. As a novel technology, physical layer security (PLS) combined with reconfigurable intelligent surfaces (RIS) can enhance transmission quality and security. This paper investigates the MEC service delay problem in RIS-aided vehicular networks under malicious eavesdropping. Due to the lack of an explicit formulation for the optimization problem, we propose a deep deterministic policy gradient (DDPG)-based communication scheme to optimize the secure MEC service. It aims to minimize the maximum MEC service time while reducing eavesdropping threats by jointly designing the RIS phase shift matrix and computing resource allocation in real-time. Simulation results demonstrate that 1) the DDPG-based scheme can help the base station make reasonable actions to realize secure MEC service in dynamic MEC vehicular networks; 2) deploying RIS can dramatically reduce eavesdropping threats and improve the overall MEC service quality. 

   more » « less
5. MIRAS: Model-based Reinforcement Learning for Microservice Resource Allocation over Scientific Workflows

   https://doi.org/10.1109/ICDCS.2019.00021  

   Yang, Zhe ; Nguyen, Phuong ; Jin, Haiming ; Nahrstedt, Klara ( July 2019 , 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS))

   Microservice, an architectural design that decomposes applications into loosely coupled services, is adopted in modern software design, including cloud-based scientific workflow processing. The microservice design makes scientific workflow systems more modular, more flexible, and easier to develop. However, cloud deployment of microservice workflow execution systems doesn't come for free, and proper resource management decisions have to be made in order to achieve certain performance objective (e.g., response time) within constraint operation cost. Nevertheless, effective online resource allocation decisions are hard to achieve due to dynamic workloads and the complicated interactions of microservices in each workflow. In this paper, we propose an adaptive resource allocation approach for microservice workflow system based on recent advances in reinforcement learning. Our approach (1) assumes little prior knowledge of the microservice workflow system and does not require any elaborately designed model or crafted representative simulator of the underlying system, and (2) avoids high sample complexity which is a common drawback of model-free reinforcement learning when applied to real-world scenarios. We show that our proposed approach automatically achieves effective policy for resource allocation with limited number of time-consuming interactions with the microservice workflow system. We perform extensive evaluations to validate the effectiveness of our approach and demonstrate that it outperforms existing resource allocation approaches with read-world emulated workflows. 

   more » « less