skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Attention:

The NSF Public Access Repository (PAR) system and access will be unavailable from 11:00 PM ET on Friday, May 16 until 2:00 AM ET on Saturday, May 17 due to maintenance. We apologize for the inconvenience.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Template Model Inspired Task Space Learning for Robust Bipedal Locomotion
This work presents a hierarchical framework for bipedal locomotion that combines a Reinforcement Learning (RL)-based high-level (HL) planner policy for the online generation of task space commands with a model-based low-level (LL) controller to track the desired task space trajectories. Different from traditional end-to-end learning approaches, our HL policy takes insights from the angular momentum-based linear inverted pendulum (ALIP) to carefully design the observation and action spaces of the Markov Decision Process (MDP). This simple yet effective design creates an insightful mapping between a low-dimensional state that effectively captures the complex dynamics of bipedal locomotion and a set of task space outputs that shape the walking gait of the robot. The HL policy is agnostic to the task space LL controller, which increases the flexibility of the design and generalization of the framework to other bipedal robots. This hierarchical design results in a learning-based framework with improved performance, data efficiency, and robustness compared with the ALIP model-based approach and state-of-the-art learning-based frameworks for bipedal locomotion. The proposed hierarchical controller is tested in three different robots, Rabbit, a five-link underactuated planar biped; Walker2D, a seven-link fully-actuated planar biped; and Digit, a 3D humanoid robot with 20 actuated joints. The trained policy naturally learns human-like locomotion behaviors and is able to effectively track a wide range of walking speeds while preserving the robustness and stability of the walking gait even under adversarial conditions.  more » « less
Award ID(s):
2144156
PAR ID:
10496139
Author(s) / Creator(s):
; ; ; ;
Publisher / Repository:
IEEE
Date Published:
ISBN:
978-1-6654-9190-7
Page Range / eLocation ID:
8582 to 8589
Format(s):
Medium: X
Location:
Detroit, MI, USA
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; (, Proceedings of the American Control Conference)
    Controller design for bipedal walking on dynamic rigid surfaces (DRSes), which are rigid surfaces moving in the inertial frame (e.g., ships and airplanes), remains largely underexplored. This paper introduces a hierarchical control approach that achieves stable underactuated bipedal walking on a horizontally oscillating DRS. The highest layer of our approach is a real-time motion planner that generates desired global behaviors (i.e., center of mass trajectories and footstep locations) by stabilizing a reduced-order robot model. One key novelty of this layer is the derivation of the reduced-order model by analytically extending the angular momentum based linear inverted pendulum (ALIP) model from stationary to horizontally moving surfaces. The other novelty is the development of a discrete-time foot-placement controller that exponentially stabilizes the hybrid, linear, time-varying ALIP. The middle layer translates the desired global behaviors into the robot’s full-body reference trajectories for all directly actuated degrees of freedom, while the lowest layer exponentially tracks those reference trajectories based on the full-order, hybrid, nonlinear robot model. Simulations confirm that the proposed framework ensures stable walking of a planar underactuated biped under different swaying DRS motions and gait types. 
    more » « less
  2. ; ; (, Journal of Dynamic Systems, Measurement, and Control)
    Abstract Bipedal locomotion over compliant terrain is an important and largely underexplored problem in the robotics community. Although robot walking has been achieved on some non-rigid surfaces with existing control methodologies, there is a need for a systematic framework applicable to different bipeds that enables stable locomotion over various compliant terrains. In this work, a novel energy-based framework is proposed that allows the dynamic locomotion of bipeds across a wide range of compliant surfaces. The proposed framework utilizes an extended version of the 3D dual spring-loaded inverted pendulum (Dual-SLIP) model that supports compliant terrains, while a bio-inspired controller is employed to regulate expected perturbations of extremely low ground-stiffness levels. An energy-based methodology is introduced for tuning the bio-inspired controller to enable dynamic walking with robustness to a wide range of low ground-stiffness one-step perturbations. The proposed system and controller are shown to mimic the vertical ground reaction force (GRF) responses observed in human walking over compliant terrains. Moreover, they succeed in handling repeated unilateral stiffness perturbations under specific conditions. This work can advance the field of biped locomotion by providing a biomimetic method for generating stable human-like walking trajectories for bipedal robots over various compliant surfaces. Furthermore, the concepts of the proposed framework could be incorporated into the design of controllers for lower-limb prostheses with adjustable stiffness to improve their robustness over compliant surfaces. 
    more » « less
  3. ; ; ; ; ; (, IEEE)
    The Angular-Momentum Linear Inverted Pendulum (ALIP) model is a promising motion planner for bipedal robots. However, it relies on two assumptions: (1) the robot has point-contact feet or passive ankles, and (2) the angular momentum around the center of mass, known as centroidal angular momentum, is negligible. This paper addresses the question of whether the ALIP paradigm can be applied to more general bipedal systems with complex foot geometry (e.g., flat feet) and nontrivial torso/limb inertia and mass distribution (e.g., non-centralized arms). In such systems, the dynamics introduce non-negligible centroidal momentum and contact wrenches at the feet, rendering the assumptions of the ALIP model invalid. This paper presents the ALIP planner for general bipedal robots with non-point-contact feet through the use of a task-space whole-body controller that regulates centroidal momentum, thereby ensuring that the robot's behavior aligns with the desired template dynamics. To demonstrate the effectiveness of our proposed approach, we conduct simulations using the Sarcos©Guardian® XO®robot, which is a hybrid humanoid/exoskeleton with large, offset feet. The results demonstrate the practicality and effectiveness of our approach in achieving stable and versatile bipedal locomotion. 
    more » « less
  4. (, IEEE International Conference on Robotics and Automation)
    This paper presents a novel framework for learning robust bipedal walking by combining a data-driven state representation with a Reinforcement Learning (RL) based locomotion policy. The framework utilizes an autoencoder to learn a low-dimensional latent space that captures the complex dynamics of bipedal locomotion from existing locomotion data. This reduced dimensional state representation is then used as states for training a robust RL-based gait policy, eliminating the need for heuristic state selections or the use of template models for gait planning. The results demonstrate that the learned latent variables are disentangled and directly correspond to different gaits or speeds, such as moving forward, backward, or walking in place. Compared to traditional template model-based approaches, our framework exhibits superior performance and robustness in simulation. The trained policy effectively tracks a wide range of walking speeds and demonstrates good generalization capabilities to unseen scenarios. 
    more » « less
  5. ; ; ; (, IEEE)
    This paper presents a novel framework for learning robust bipedal walking by combining a data-driven state representation with a Reinforcement Learning (RL) based locomotion policy. The framework utilizes an autoencoder to learn a low-dimensional latent space that captures the complex dynamics of bipedal locomotion from existing locomotion data. This reduced dimensional state representation is then used as states for training a robust RL-based gait policy, eliminating the need for heuristic state selections or the use of template models for gait planning. The results demonstrate that the learned latent variables are disentangled and directly correspond to different gaits or speeds, such as moving forward, backward, or walking in place. Compared to traditional template model-based approaches, our framework exhibits superior performance and robustness in simulation. The trained policy effectively tracks a wide range of walking speeds and demonstrates good generalization capabilities to unseen scenarios. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email