A control system for bipedal walking in the sagittal plane was developed in simulation. The biped model was built based on anthropometric data for a 1.8 m tall male of average build. At the core of the controller is a deep deterministic policy gradient (DDPG) neural network that was trained in GAZEBO, a physics simulator, to predict the ideal foot placement to maintain stable walking despite external disturbances. The complexity of the DDPG network was decreased through carefully selected state variables and a distributed control system. Additional controllers for the hip joints during their stance phases and the ankle joint during toe-off phase help to stabilize the biped during walking. The simulated biped can walk at a steady pace of approximately 1 m/s, and during locomotion it can maintain stability with a 30 kg·m/s impulse applied forward on the torso or a 40 kg·m/s impulse applied rearward. It also maintains stable walking with a 10 kg backpack or a 25 kg front pack. The controller was trained on a 1.8 m tall model, but also stabilizes models 1.4–2.3 m tall with no changes.
more »
« less
Implementation of Deep Deterministic Policy Gradients for Controlling Dynamic Bipedal Walking
A control system for simulated two-dimensional bipedal
walking was developed. The biped model was built based on anthropometric
data. At the core of the control is a Deep Deterministic Policy
Gradients (DDPG) neural network that is trained in GAZEBO, a
physics simulator, to predict the ideal foot location to maintain stable
walking under external impulse load. Additional controllers for hip
joint movement during stance phase, and ankle joint torque during toeoff,
help to stabilize the robot during walking. The simulated robot can
walk at a steady pace of approximately 1m/s, and during locomotion it
can maintain stability with a 30N-s impulse applied at the torso. This
work implement DDPG algorithm to solve biped walking control problem.
The complexity of DDPG network is decreased through carefully
selected state variables and distributed control system.
more »
« less
- Award ID(s):
- 1739800
- NSF-PAR ID:
- 10073905
- Date Published:
- Journal Name:
- Living Machines: Conference on Biomimetic and Biohybrid Systems
- Page Range / eLocation ID:
- 276–287
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Our group is developing a cyber-physical walking system (CPWS) for people paralyzed by spinal cord injuries (SCI). The current CPWS consists of a functional neuromuscular stimulation (FNS) system and a powered lower-limb exoskeleton for walking with leg movements in the sagittal plane. We are developing neural control systems that learn to assist the user of this CPWS to walk with stability. In a previous publication (Liu et al., Biomimetics, 2019, 4, 28), we showed a neural controller that stabilized a simulated biped in the sagittal plane. We are considering adding degrees of freedom to the CPWS to allow more natural walking movements and improved stability. Thus, in this paper, we present a new neural network enhanced control system that stabilizes a three-dimensional simulated biped model of a human wearing an exoskeleton. Results show that it stabilizes human/exoskeleton models and is robust to impact disturbances. The simulated biped walks at a steady pace in a range of typical human ambulatory speeds from 0.7 to 1.3 m/s, follows waypoints at a precision of 0.3 m, remains stable, and continues walking forward despite impact disturbances and adapts its speed to compensate for persistent external disturbances. Furthermore, the neural network controller stabilizes human models of different statures from 1.4 to 2.2 m tall without any changes to the control parameters. Please see videos at the following link: 3D biped walking control .more » « less
-
This paper presents the design, modeling, analysis, and experimental results of a bipedal robotic system that utilizes two interconnected single degree-of-freedom leg mechanisms to produce stable forward locomotion and steering. The legs are composed of double four-bar mechanism connected in series that maintain a parallel orientation of a flat foot, relative to the biped body, that is actuated via a Reuleaux triangle cam-follower system to produce a desirable foot trajectory. The mechanical design of the leg mechanism is presented followed by kinematic analysis of the cam-follower system to select the optimal foot trajectory and synthesize the mechanism dimensions and produce a desired step height and step length. The concept of leg sequencing is then presented to maintain a constant body height above the ground and a constant forward walking velocity. Experimental results using an integrated prototype indicate that the proposed biped robot is capable of maintaining quasi-static stability during locomotion, maintaining a constant robot body height, maintaining a constant body orientation, move forward with a constant maximum velocity of 27.4 cm/s, and steer.more » « less
-
Robot assisted gait retraining is an increasingly common method for supporting restoration of walking function after neurological injury. Gait speed, an indicator of walking function, is correlated with propulsive force, a measure modulated by the posture of the trailing limb at push-off. With the ultimate goal of improving efficacy of robot assisted gait retraining, we sought to directly target gait propulsion, by exposing subjects to pulses of joint torque applied at the hip and knee joints to modulate push-off posture. In this work, we utilized a robotic exoskeleton to apply pulses of torque to the hip and knee joints, during individual strides, of 16 healthy control subjects, and quantified the effects of this intervention on hip extension and propulsive impulse during and after application of these pulses. We observed significant effects in the outcome measures primarily at the stride of pulse application and generally no after effects in the following strides. Specifically, when pulses were applied at late stance, we observed a significant increase in propulsive impulse when knee and/or hip flexion pulses were applied and a significant increase in hip extension angle when hip extension torque pulses were applied. When pulses were applied at early stance, we observed a significant increase in propulsive impulse associated with hip extension torque.more » « less
-
Abstract Background Few studies have systematically investigated robust controllers for lower limb rehabilitation exoskeletons (LLREs) that can safely and effectively assist users with a variety of neuromuscular disorders to walk with full autonomy. One of the key challenges for developing such a robust controller is to handle different degrees of uncertain human-exoskeleton interaction forces from the patients. Consequently, conventional walking controllers either are patient-condition specific or involve tuning of many control parameters, which could behave unreliably and even fail to maintain balance. Methods We present a novel, deep neural network, reinforcement learning-based robust controller for a LLRE based on a decoupled offline human-exoskeleton simulation training with three independent networks, which aims to provide reliable walking assistance against various and uncertain human-exoskeleton interaction forces. The exoskeleton controller is driven by a neural network control policy that acts on a stream of the LLRE’s proprioceptive signals, including joint kinematic states, and subsequently predicts real-time position control targets for the actuated joints. To handle uncertain human interaction forces, the control policy is trained intentionally with an integrated human musculoskeletal model and realistic human-exoskeleton interaction forces. Two other neural networks are connected with the control policy network to predict the interaction forces and muscle coordination. To further increase the robustness of the control policy to different human conditions, we employ domain randomization during training that includes not only randomization of exoskeleton dynamics properties but, more importantly, randomization of human muscle strength to simulate the variability of the patient’s disability. Through this decoupled deep reinforcement learning framework, the trained controller of LLREs is able to provide reliable walking assistance to patients with different degrees of neuromuscular disorders without any control parameter tuning. Results and conclusion A universal, RL-based walking controller is trained and virtually tested on a LLRE system to verify its effectiveness and robustness in assisting users with different disabilities such as passive muscles (quadriplegic), muscle weakness, or hemiplegic conditions without any control parameter tuning. Analysis of the RMSE for joint tracking, CoP-based stability, and gait symmetry shows the effectiveness of the controller. An ablation study also demonstrates the strong robustness of the control policy under large exoskeleton dynamic property ranges and various human-exoskeleton interaction forces. The decoupled network structure allows us to isolate the LLRE control policy network for testing and sim-to-real transfer since it uses only proprioception information of the LLRE (joint sensory state) as the input. Furthermore, the controller is shown to be able to handle different patient conditions without the need for patient-specific control parameter tuning.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1007/978-3-319-95972-6_29
