More Like this

1. Safe Policy Learning in Online Reinforcement Learning Using Augmented PPO and CMDPs for Robot Navigation

   Zhou, Z; Ho, KC; Sun, Z; Tran, T (October 2025, IEEE)

   This work focuses on online reinforcement learning (RL) in the presence of en- vironmental constraints. Specifically, we consider applications involving robot agents exploring in an environment where obstacles and unsafe zones are present, and the agents must maximize cumulative rewards and at the same time meet the environmental constraints. To address this challenge, we formulate the prob- lem using the constrained Markov Decision Process (CMDP) and incorporate the environmental constraint costs into the policy updates in the proposed Aug- mented Proximal Policy Optimization (APPO) algorithm. At each state and for each possible action, we apply a Variational Auto-Encoder (VAE) [1] to obtain a probabilistic estimate of the discounted cumulative future environmental con- straint costs and integrate them as a regularization term to the reward function. This augmented reward function updates the action-value functions within the APPO algorithm, which is trained by an efficient optimization scheme. Ex- perimental results demonstrate that our methodology enables robot agents to navigate within the safety-constrained regions effectively. 

   more » « less
2. A Spatio-Temporal Prediction and Planning Framework for Proactive Human–Robot Collaboration

   https://doi.org/10.1115/1.4063502  

   Flowers, Jared; Wiens, Gloria (December 2023, Journal of Manufacturing Science and Engineering)

   Pai Zheng (Ed.)

   Abstract A significant challenge in human–robot collaboration (HRC) is coordinating robot and human motions. Discoordination can lead to production delays and human discomfort. Prior works seek coordination by planning robot paths that consider humans or their anticipated occupancy as static obstacles, making them nearsighted and prone to entrapment by human motion. This work presents the spatio-temporal avoidance of predictions-prediction and planning framework (STAP-PPF) to improve robot–human coordination in HRC. STAP-PPF predicts multi-step human motion sequences based on the locations of objects the human manipulates. STAP-PPF then proactively determines time-optimal robot paths considering predicted human motion and robot speed restrictions anticipated according to the ISO15066 speed and separation monitoring (SSM) mode. When executing robot paths, STAP-PPF continuously updates human motion predictions. In real-time, STAP-PPF warps the robot’s path to account for continuously updated human motion predictions and updated SSM effects to mitigate delays and human discomfort. Results show the STAP-PPF generates robot trajectories of shorter duration. STAP-PPF robot trajectories also adapted better to real-time human motion deviation. STAP-PPF robot trajectories also maintain greater robot/human separation throughout tasks requiring close human–robot interaction. Tests with an assembly sequence demonstrate STAP-PPF’s ability to predict multi-step human tasks and plan robot motions for the sequence. STAP-PPF also most accurately estimates robot trajectory durations, within 30% of actual, which can be used to adapt the robot sequencing to minimize disruption. 

   more » « less
3. Map Space Belief Prediction for Manipulation-Enhanced Mapping

   Marques, Joao_Marcos_Correia; Dengler, Nils; Zaenker, Tobias; Mucke, Jesper; Wang, Shenlong; Bennewitz, Maren; Hauser, Kris (June 2025, Robotics Science and Systems)

   Searching for objects in cluttered environments requires selecting efficient viewpoints and manipulation actions to remove occlusions and reduce uncertainty in object locations, shapes, and categories. In this work, we address the problem of manipulation-enhanced semantic mapping, where a robot has to efficiently identify all objects in a cluttered shelf. Although Partially Observable Markov Decision Processes~(POMDPs) are standard for decision-making under uncertainty, representing unstructured interactive worlds remains challenging in this formalism. To tackle this, we define a POMDP whose belief is summarized by a metric-semantic grid map and propose a novel framework that uses neural networks to perform map-space belief updates to reason efficiently and simultaneously about object geometries, locations, categories, occlusions, and manipulation physics. Further, to enable accurate information gain analysis, the learned belief updates should maintain calibrated estimates of uncertainty. Therefore, we propose Calibrated Neural-Accelerated Belief Updates (CNABUs) to learn a belief propagation model that generalizes to novel scenarios and provides confidence-calibrated predictions for unknown areas. Our experiments show that our novel POMDP planner improves map completeness and accuracy over existing methods in challenging simulations and successfully transfers to real-world cluttered shelves in zero-shot fashion. 

   more » « less
4. FPGA-Based Velocity Estimation for Control of Robots with Low-Resolution Encoders

   Wu, Jie Ying; Chen, Zihan; Deguet, Anton; Kazanzides, Peter (October 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS))

   Robot control algorithms often rely on measurements of robot joint velocities, which can be estimated by measuring the time between encoder edges. When encoder edges occur infrequently, such as at low velocities and/or with low resolution encoders, this measurement delay may affect the stability of closed-loop control. This is evident in both the joint position control and Cartesian impedance control of the da Vinci Research Kit (dVRK), which contains several low-resolution encoders. We present a hardware-based method that gives more frequent velocity updates and is not affected by common encoder imperfections such as non-uniform duty cycles and quadrature phase error. The proposed method measures the time between consecutive edges of the same type but, unlike prior methods, is implemented for the rising and falling edges of both channels. Additionally, it estimates acceleration to enable software compensation of the measurement delay. The method is shown to improve Cartesian impedance control of the dVRK. 

   more » « less
5. Reduced Solar Quadrupole Moment Compensates for Lack of Asteroids in Long-term Solar System Integrations

   https://doi.org/10.3847/1538-3881/add5dc  

   Zeebe, Richard E; Kocken, Ilja J (June 2025, The Astronomical Journal)

   Abstract State-of-the-art long-term solar system integrations include several second order effects such as the Sun’s quadrupole momentJ₂and a contribution from asteroids (plus the Moon and general relativity). We recently showed that including 10 asteroids and a reducedJ₂in our astronomical solutions provides the best match with geologic data to −58 Myr. However, the rationale for the reducedJ₂remained ambiguous and may suggest that parameters for long-term integrations compatible with geologic observations are not fully compatible with our knowledge of the current solar system (specificallyJ₂). Here we show that a reducedJ₂compensates for a diminished asteroid population in long-term solar system integrations, which may appear surprising. We present an analysis and offer a mechanism for the long-term compensating effects ofJ₂and asteroid mass in the solar system (not planetary systems in general). Our analysis suggests that “differential effects” on specific secular frequencies involved in resonant terms (i.e., (g₄ − g₃) and (s₄ − s₃)), are critical in the long term, rather than short-term effects on the orbital elements of individual planetary orbits across the board. Also, our results indicate that if long-term integrations including the full asteroid population were computationally feasible, aJ₂value (within errors) compatible with our current knowledge of the solar system could be used. Attempts to improve the long-term accuracy of astronomical solutions by, e.g., tinkering with initial conditions using current/future astronomical observations are futile unless asteroid deficiencies in the solar system model are addressed. 

   more » « less