More Like this

1. Bayesian Optimization for Online Management in Dynamic Mobile Edge Computing

   https://doi.org/10.1109/TWC.2023.3307875  

   Yan, Jia; Lu, Qin; Giannakis, Georgios B. (January 2024, IEEE Transactions on Wireless Communications)

   Recent years have witnessed the emergence of mobile edge computing (MEC), on the premise of a costeffective enhancement in the computational ability of hardwareconstrained wireless devices (WDs) comprising the Internet of Things (IoT). In a general multi-server multi-user MEC system, each WD has a computational task to execute and has to select binary (off)loading decisions, along with the analog-amplitude resource allocation variables in an online manner, with the goal of minimizing the overall energy-delay cost (EDC) with dynamic system states. While past works typically rely on the explicit expression of the EDC function, the present contribution considers a practical setting, where in lieu of system state information, the EDC function is not available in analytical form, and instead only the function values at queried points are revealed. Towards tackling such a challenging online combinatorial problem with only bandit information, novel Bayesian optimization (BO) based approaches are put forth by leveraging the multi-armed bandit (MAB) framework. Per time slot, the discrete offloading decisions are first obtained via the MAB method, and the analog resource allocation variables are subsequently optimized using the BO selection rule. By exploiting both temporal and contextual information, two novel BO approaches, termed time-varying BO and contextual time-varying BO, are developed. Numerical tests validate the merits of the proposed BO approaches compared with contemporary benchmarks under different MEC network sizes. 

   more » « less
2. Online learning for multi-agent based resource allocation in weakly coupled wireless systems

   https://doi.org/10.1145/3492866.3549714  

   Song, Jianhan; de Veciana, Gustavo; Shakkottai, Sanjay (October 2022, ACM Mobihoc '22)

   We propose and evaluate a learning-based framework to address multi-agent resource allocation in coupled wireless systems. In particular we consider, multiple agents (e.g., base stations, access points, etc.) that choose amongst a set of resource allocation options towards achieving their own performance objective /requirements, and where the performance observed at each agent is further coupled with the actions chosen by the other agents, e.g., through interference, channel leakage, etc. The challenge is to find the best collective action. To that end we propose a Multi-Armed Bandit (MAB) framework wherein the best actions (aka arms) are adaptively learned through online reward feedback. Our focus is on systems which are "weakly-coupled" wherein the best arm of each agent is invariant to others' arm selection the majority of the time - this majority structure enables one to develop light weight efficient algorithms. This structure is commonly found in many wireless settings such as channel selection and power control. We develop a bandit algorithm based on the Track-and-Stop strategy, which shows a logarithmic regret with respect to a genie. Finally through simulation, we exhibit the potential use of our model and algorithm in several wireless application scenarios. 

   more » « less
3. HPC Application Parameter Autotuning on Edge Devices: A Bandit Learning Approach

   https://doi.org/10.1109/HiPC62374.2024.00011  

   Hossain, Abrar; Badawy, Abdel-Hameed A; Islam, Mohammad A; Patki, Tapasya; Ahmed, Kishwar (December 2024, IEEE)

   The growing necessity for enhanced processing capabilities in edge devices with limited resources has led us to develop effective methods for improving high-performance computing (HPC) applications. In this paper, we introduce LASP (Lightweight Autotuning of Scientific Application Parameters), a novel strategy designed to address the parameter search space challenge in edge devices. Our strategy employs a multi-armed bandit (MAB) technique focused on online exploration and exploitation. Notably, LASP takes a dynamic approach, adapting seamlessly to changing environments. We tested LASP with four HPC applications: Lulesh, Kripke, Clomp, and Hypre. Its lightweight nature makes it particularly well-suited for resource-constrained edge devices. By employing the MAB framework to efficiently navigate the search space, we achieved significant performance improvements while adhering to the stringent computational limits of edge devices. Our experimental results demonstrate the effectiveness of LASP in optimizing parameter search on edge devices. 

   more » « less
4. Multiplayer Modeling via Multi-Armed Bandits

   Gray, Robert C; Zhu, Jichen; Ontañón, Santiago (January 2021, Proceedings of the 3rd IEEE Conference on Games)

   null (Ed.)

   This paper focuses on player modeling in multiplayer adaptive games. While player modeling has received a significant amount of attention, less is known about how to use player modeling in multiplayer games, especially when an experience management AI must make decisions on how to adapt the experience for the group as a whole. Specifically, we present a multi-armed bandit (MAB) approach for modeling groups of multiple players. Our main contributions are a new MAB frame- work for multiplayer modeling and techniques for addressing the new challenges introduced by the multiplayer context, extending previous work on MAB-based player modeling to account for new group-generated phenomena not present in single-user models. We evaluate our approach via simulation of virtual players in the context of multiplayer adaptive exergames. 

   more » « less
5. Bounded rationality in Las Vegas: finite automata play multi-armed bandits

   Liu, X.; Halpern, J.Y. (June 2020, Proceedings of the 36th Conference on Uncertainty in AI (UAI 2020))

   null (Ed.)

   While traditional economics assumes that humans are fully rational agents who always maximize their expected utility, in practice, we constantly observe apparently irrational behavior. One explanation is that people have limited computational power, so that they are, quite rationally, making the best decisions they can, given their computational limitations. To test this hypothesis, we consider the multi-armed bandit (MAB) problem. We examine a simple strategy for playing an MAB that can be implemented easily by a probabilistic finite automaton (PFA). Roughly speaking, the PFA sets certain expectations, and plays an arm as long as it meets them. If the PFA has sufficiently many states, it performs near-optimally. Its performance degrades gracefully as the number of states decreases. Moreover, the PFA acts in a ``human-like'' way, exhibiting a number of standard human biases, like an optimism bias and a negativity bias. 

   more » « less