More Like this

1. Adaptive Metareasoning for Bounded Rational Agents

   Svegliato, Justin; Zilberstein, Shlomo (July 2018, IJCAI-ECAI Workshop on Architectures and Evaluation for Generality, Autonomy and Progress in AI (AEGAP))

   In computational approaches to bounded rationality, metareasoning enables intelligent agents to optimize their own decision-making process in order to produce effective action in a timely manner. While there have been substantial efforts to develop effective meta-level control for anytime algorithms, existing techniques rely on extensive offline work, imposing several critical assumptions that diminish their effectiveness and limit their practical utility in the real world. In order to eliminate these assumptions, adaptive metareasoning enables intelligent agents to adapt to each individual instance of the problem at hand without the need for significant offline preprocessing. Building on our recent work, we first introduce a model-free approach to meta-level control based on reinforcement learning. We then present a meta-level control technique that uses temporal difference learning. Finally, we show empirically that our approach is effective on a common benchmark in meta-level control. 

   more » « less
2. Robotic Algorithm Service Contracts to Manage and Incentivize Adaptive Behavior

   https://doi.org/10.1109/BCCA66705.2025.11229721  

   Mallikarachchi, Sanjaya; Thammi, Phanindrateja; Commey, Daniel; Vitharana, Sandun S; Chintalapati, Meghana; Godage, Isuru S (October 2025, IEEE)

   This study presents a novel approach to incentivizing adaptive behavior in robotic systems by integrating reinforcement learning (RL) with blockchain-based access tokens via the Robotic Algorithm Service Contract (RASC). Unlike traditional methods, RASC leverages smart contracts to distribute access tokens, enabling users to invoke RL algorithms, such as those used for path planning. The robotic agents operate under predefined criteria within the smart contract, ensuring fair and transparent execution. Robotic agents are rewarded through an incentive model that evaluates task completion based on learned behaviors. Additionally, an oracle contract is introduced to supply real-world data, enhancing the adjudication process within the RASC framework. The proposed system has been rigorously validated through simulations and a prototype robot implementation, demonstrating its effectiveness in promoting adaptive and efficient robotic behavior. Furthermore, the incentive model introduced in this work serves as a strong proxy for real-world energy-efficient execution by ensuring that robots are only rewarded for effective learned behaviors while excluding unnecessary energy expenditure. This approach simplifies reward calculation, promotes fairness, and aligns economic incentives with optimal robotic performance. 

   more » « less
3. Reinforcement learning‐guided control strategies for CAR T‐cell activation and expansion

   https://doi.org/10.1002/bit.28753  

   Ferdous, Sakib; Shihab, Ibne Farabi; Chowdhury, Ratul; Reuel, Nigel F (September 2024, Biotechnology and Bioengineering)

   Abstract Reinforcement learning (RL), a subset of machine learning (ML), could optimize and control biomanufacturing processes, such as improved production of therapeutic cells. Here, the process of CAR T‐cell activation by antigen‐presenting beads and their subsequent expansion is formulated in silico. The simulation is used as an environment to train RL‐agents to dynamically control the number of beads in culture to maximize the population of robust effector cells at the end of the culture. We make periodic decisions of incremental bead addition or complete removal. The simulation is designed to operate in OpenAI Gym, enabling testing of different environments, cell types, RL‐agent algorithms, and state inputs to the RL‐agent. RL‐agent training is demonstrated with three different algorithms (PPO, A2C, and DQN), each sampling three different state input types (tabular, image, mixed); PPO‐tabular performs best for this simulation environment. Using this approach, training of the RL‐agent on different cell types is demonstrated, resulting in unique control strategies for each type. Sensitivity to input‐noise (sensor performance), number of control step interventions, and advantages of pre‐trained RL‐agents are also evaluated. Therefore, we present an RL framework to maximize the population of robust effector cells in CAR T‐cell therapy production. 

   more » « less
4. Continuous-Time Reward Machines

   https://doi.org/10.24963/ijcai.2025/563  

   Falah, Amin; Guha, Shibashis; Trivedi, Ashutosh (September 2025, International Joint Conferences on Artificial Intelligence Organization)

   Reinforcement Learning (RL) is a sampling-based method for sequential decision-making, in which a learning agent iteratively converges toward an optimal policy by leveraging feedback from the environment in the form of scalar reward signals.While timing information is often abstracted in discrete-time domains, time-critical learning applications—such as queuing systems, population processes, and manufacturing systems—are naturally modeled as Continuous-Time Markov Decision Processes (CTMDPs).Since the seminal work of Bradtke and Duff, model-free RL for CTMDPs has become well-understood. However, in many practical applications, practitioners possess high-quality information about system rates derived from traditional queuing theory, which learning agents could potentially exploit to accelerate convergence. Despite this, classical RL algorithms for CTMDPs typically re-learn these parameters through sampling.In this work, we propose continuous-time reward machines (CTRMs), a novel framework that embeds reward functions and real-time state-action dynamics into a unified structure. CTRMs enable RL agents to effectively navigate dense-time environments while leveraging reward shaping and counterfactual experiences for accelerated learning. Our empirical results demonstrate CTRMs' ability to improve learning efficiency in time-critical environments. 

   more » « less
5. Robust Deep Reinforcement Learning through Adversarial Loss

   Oikarinen, T.; Zhang, W.; Megretski, A.; Daniel, L.; Weng, T.-W. (December 2021, Advances in neural information processing systems)

   Recent studies have shown that deep reinforcement learning agents are vulnerable to small adversarial perturbations on the agent’s inputs, which raises concerns about deploying such agents in the real world. To address this issue, we propose RADIAL-RL, a principled framework to train reinforcement learning agents with improved robustness against lp-norm bounded adversarial attacks. Our framework is compatible with popular deep reinforcement learning algorithms and we demonstrate its performance with deep Q-learning, A3C and PPO. We experiment on three deep RL benchmarks (Atari, MuJoCo and ProcGen) to show the effectiveness of our robust training algorithm. Our RADIAL-RL agents consistently outperform prior methods when tested against attacks of varying strength and are more computationally efficient to train. In addition, we propose a new evaluation method called Greedy Worst-Case Reward (GWC) to measure attack agnostic robustness of deep RL agents. We show that GWC can be evaluated efficiently and is a good estimate of the reward under the worst possible sequence of adversarial attacks. All code used for our experiments is available at https://github.com/tuomaso/radial_rl_v2. 

   more » « less