An official website of the United States government
This work presents a hierarchical framework for bipedal locomotion that combines a Reinforcement Learning (RL)-based high-level (HL) planner policy for the online generation of task space commands with a model-based low-level (LL) controller to track the desired task space trajectories. Different from traditional end-to-end learning approaches, our HL policy takes insights from the angular momentum-based linear inverted pendulum (ALIP) to carefully design the observation and action spaces of the Markov Decision Process (MDP). This simple yet effective design creates an insightful mapping between a low-dimensional state that effectively captures the complex dynamics of bipedal locomotion and a set of task space outputs that shape the walking gait of the robot. The HL policy is agnostic to the task space LL controller, which increases the flexibility of the design and generalization of the framework to other bipedal robots. This hierarchical design results in a learning-based framework with improved performance, data efficiency, and robustness compared with the ALIP model-based approach and state-of-the-art learning-based frameworks for bipedal locomotion. The proposed hierarchical controller is tested in three different robots, Rabbit, a five-link underactuated planar biped; Walker2D, a seven-link fully-actuated planar biped; and Digit, a 3D humanoid robot with 20 actuated joints. The trained policy naturally learns human-like locomotion behaviors and is able to effectively track a wide range of walking speeds while preserving the robustness and stability of the walking gait even under adversarial conditions.
more »
« less
An Energy-Based Framework for Robust Dynamic Bipedal Walking Over Compliant Terrain
Abstract Bipedal locomotion over compliant terrain is an important and largely underexplored problem in the robotics community. Although robot walking has been achieved on some non-rigid surfaces with existing control methodologies, there is a need for a systematic framework applicable to different bipeds that enables stable locomotion over various compliant terrains. In this work, a novel energy-based framework is proposed that allows the dynamic locomotion of bipeds across a wide range of compliant surfaces. The proposed framework utilizes an extended version of the 3D dual spring-loaded inverted pendulum (Dual-SLIP) model that supports compliant terrains, while a bio-inspired controller is employed to regulate expected perturbations of extremely low ground-stiffness levels. An energy-based methodology is introduced for tuning the bio-inspired controller to enable dynamic walking with robustness to a wide range of low ground-stiffness one-step perturbations. The proposed system and controller are shown to mimic the vertical ground reaction force (GRF) responses observed in human walking over compliant terrains. Moreover, they succeed in handling repeated unilateral stiffness perturbations under specific conditions. This work can advance the field of biped locomotion by providing a biomimetic method for generating stable human-like walking trajectories for bipedal robots over various compliant surfaces. Furthermore, the concepts of the proposed framework could be incorporated into the design of controllers for lower-limb prostheses with adjustable stiffness to improve their robustness over compliant surfaces.
more »
« less
- PAR ID:
- 10515996
- Publisher / Repository:
- American Society of Mechanical Engineers (ASME)
- Date Published:
- Journal Name:
- Journal of Dynamic Systems, Measurement, and Control
- Volume:
- 146
- Issue:
- 2
- ISSN:
- 0022-0434
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Low-friction foot/ground contacts present a particular challenge for stable bipedal walkers. The slippage of the stance foot introduces complexity in robot dynamics and the general locomotion stability results cannot be applied directly. We relax the commonly used assumption of nonslip contact between the walker foot and the ground and examine bipedal dynamics under foot slip. Using a two-mass linear inverted pendulum model, we introduce the concept of balance recoverability and use it to quantify the balanced or fall-prone walking gaits. Balance recoverability also serves as the basis for the design of the balance recovery controller. We design the within- or multi-step recovery controller to assist the walker to avoid fall. The controller performance is validated through simulation results and robustness is demonstrated in the presence of measurement noises as well as variations of foot/ground friction conditions. In addition, the proposed methods and models are used to analyze the data from human walking experiments. The multiple subject experiments validate and illustrate the balance recoverability concept and analyses.more » « less
-
This paper presents a comprehensive study on using deep reinforcement learning (RL) to create dynamic locomotion controllers for bipedal robots. Going beyond focusing on a single locomotion skill, we develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing. Our RL-based controller incorporates a novel dual-history architecture, utilizing both a long-term and short-term input/output (I/O) history of the robot. This control architecture, when trained through the proposed end-to-end RL approach, consistently outperforms other methods across a diverse range of skills in both simulation and the real world. The study also delves into the adaptivity and robustness introduced by the proposed RL system in developing locomotion controllers. We demonstrate that the proposed architecture can adapt to both time-invariant dynamics shifts and time-variant changes, such as contact events, by effectively using the robot’s I/O history. Additionally, we identify task randomization as another key source of robustness, fostering better task generalization and compliance to disturbances. The resulting control policies can be successfully deployed on Cassie, a torque-controlled human-sized bipedal robot. This work pushes the limits of agility for bipedal robots through extensive real-world experiments. We demonstrate a diverse range of locomotion skills, including: robust standing, versatile walking, fast running with a demonstration of a 400-meter dash, and a diverse set of jumping skills, such as standing long jumps and high jumps.more » « less
-
Abstract Understanding legged locomotion in various environments is valuable for many fields, including robotics, biomechanics, rehabilitation, and motor control. Specifically, investigating legged locomotion in compliant terrains has recently been gaining interest for the robust control of legged robots over natural environments. At the same time, the importance of ground compliance has also been highlighted in poststroke gait rehabilitation. Currently, there are not many ways to investigate walking surfaces of varying stiffness. This article introduces the variable stiffness treadmill (VST) 2, an improvement of the first version of the VST, which was the first treadmill able to vary belt stiffness. In contrast to the VST 1, the device presented in this paper (VST 2) can reduce the stiffness of both belts independently, by generating vertical deflection instead of angular, while increasing the walking surface area from 0.20m2 to 0.74m2. In addition, both treadmill belts are now driven independently, while high-spatial-resolution force sensors under each belt allow for measurement of ground reaction forces and center of pressure. Through validation experiments, the VST 2 displays high accuracy and precision. The VST 2 has a stiffness range of 13kN/m to 1.5MN/m, error of less than 1%, and standard deviations of less than 2.2kN/m, demonstrating its ability to simulate low-stiffness environments reliably. The VST 2 constitutes a drastic improvement of the VST platform, a one-of-its-kind system that can improve our understanding of human and robotic gait while creating new avenues of research on biped locomotion, athletic training, and rehabilitation of gait after injury or disease.more » « less
-
Dynamic legged locomotion is being explored as a means to maneuver on rugged and unstructured terrains. However, limited foot contact sensing capabilities often prohibit bipedal robots from being deployed on complex terrains. Locomotion over cluttered outdoor environments requires the contacting foot to be aware of terrain geometries, stiffness, and granular media properties. To achieve this, we designed a new soft contact pad integrated with a variety of embedded sensors, including tactile, acoustic, capacitive, and temperature sensors, as well as an accelerometer. In addition, we devised a terrain classification algorithm based on features extracted from those sensors and various real-world terrains. The classifier uses these features as inputs and classifies various terrains via Random Forests and a memory function. Our cross-validation tests demonstrate that the proposed classification algorithm achieves an accuracy of about 96.5%, manifesting the applicability of this foot sensing device to bipedal locomotion over diverse terrains.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1115/1.4064094
