Search for: All records

Robust Markov Decision Processes (MDPs) offer a promising framework for computing reliable policies under model uncertainty. While policy gradient methods have gained increasing popularity in robust discounted MDPs, their application to the average-reward criterion remains largely unexplored. This paper proposes a Robust Projected Policy Gradient (RP2G), the first generic policy gradient method for robust average-reward MDPs (RAMDPs) that is applicable beyond the typical rectangularity assumption on transition ambiguity. In contrast to existing robust policy gradient algorithms, RP2G incorporates an adaptive decreasing tolerance mechanism for efficient policy updates at each iteration. We also present a comprehensive convergence analysis of RP2G for solving ergodic tabular RAMDPs. Furthermore, we establish the first study of the inner worst-case transition evaluation problem in RAMDPs, proposing two gradient-based algorithms tailored for rectangular and general ambiguity sets, each with provable convergence guarantees. Numerical experiments confirm the global convergence of our new algorithm and demonstrate its superior performance. 

Current methods for producing cardiomyocytes from human induced pluripotent stem cells (hiPSCs) using 2D monolayer differentiation are often hampered by batch-to-batch variability and inef!cient puri!cation processes. Here, we introduce CM-AI, a novel arti!cial intelligence-guided laser cell processing platform designed for rapid, label-free puri!cation of hiPSC-derived cardiomyocytes (hiPSC-CMs). This approach signi!cantly reduces processing time without the need for chronic metabolic selection or antibody-based sorting. By integrating real-time cellular morphology analysis and targeted laser ablation, CM-AI selectively removes non-cardiomyocyte populations with high precision. This streamlined process preserves cardiomyocyte viability and function, offering a scalable and ef!cient solution for cardiac regenerative medicine, disease modeling, and drug disco