<?xml-model href='http://www.tei-c.org/release/xml/tei/custom/schema/relaxng/tei_all.rng' schematypens='http://relaxng.org/ns/structure/1.0'?><TEI xmlns="http://www.tei-c.org/ns/1.0">
	<teiHeader>
		<fileDesc>
			<titleStmt><title level='a'>MELP: Model Embedded Linear Policies for Robust Bipedal Hopping</title></titleStmt>
			<publicationStmt>
				<publisher>IEEE</publisher>
				<date>10/01/2023</date>
			</publicationStmt>
			<sourceDesc>
				<bibl> 
					<idno type="par_id">10496140</idno>
					<idno type="doi">10.1109/IROS55552.2023.10342023</idno>
					
					<author>Raghav Soni</author><author>Guillermo A. Castillo</author><author>Lokesh Krishna</author><author>Ayonga Hereid</author><author>Shishir Kolathaya</author>
				</bibl>
			</sourceDesc>
		</fileDesc>
		<profileDesc>
			<abstract><ab><![CDATA[Linear policies are the simplest class of policies that can achieve stable bipedal walking behaviors in both simulation and hardware. However, a significant challenge in deploying them widely is the difficulty in extending them to more dynamic behaviors like hopping and running. Therefore, in this work, we propose a new class of linear policies in which template models can be embedded. In particular, we show how to embed Spring Loaded Inverted Pendulum (SLIP) model in the policy class and realize perpetual hopping in arbitrary directions. The spring constant of the template model is learned in addition to the remaining parameters of the policy. Given this spring constant, the goal is to realize hopping trajectories using the SLIP model, which are then tracked by the bipedal robot using the linear policy. Continuous hopping with adjustable heading direction was achieved across different terrains in simulation with heading and lateral velocities of up to O.5m/ sec and 0.05m/ sec, respectively. The policy was then transferred to the hardware, and preliminary results (> 10 steps) of hopping were achieved.]]></ab></abstract>
		</profileDesc>
	</teiHeader>
	<text><body xmlns="http://www.tei-c.org/ns/1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink">
<div xmlns="http://www.tei-c.org/ns/1.0"><head>I. INTRODUCTION</head><p>Recent advances in actuation, computation, and sensing have made legged robots a viable technology for deploying in environments engineered for humans. These results are often conservative owing to the controlled settings of their deployed environments. However, the fullest potential of these platforms can be unleashed through their deployment as first responders in uncontrolled environments which require highly athletic and aggressive manoeuvres akin to their human counterparts, which is currently limited by existing control strategies. Established methods like Zero Moment Point (ZMP) <ref type="bibr">[18]</ref>, Linear Inverted Pendulum models <ref type="bibr">[5]</ref> and Hybrid Zero Dynamics (HZD) <ref type="bibr">[20]</ref> can provide robust walking but fail to extend to the expected aperiodic transient behaviours. Their success is also subject to rigorous tuning and gain scheduling <ref type="bibr">[9]</ref> for their real-world deployment.</p><p>Sophisticated techniques from optimal control, like trajectory optimization (TO), model predictive control (MPC) and whole body impulse control (WBIC), show great promise in realizing dynamic behaviours like jumps and backflips <ref type="bibr">[6]</ref>. Despite MPC's success, scaling it to acrobatic motions with longer aerial phases is challenging as their success and real-time deployment is conditional to the use of simplified dynamics <ref type="bibr">[6]</ref>, <ref type="bibr">[5]</ref> and shorter planning horizons. TO provides a workaround to overcome this limitation by generating libraries of trajectories utilizing the full order dynamics that can then be tracked online <ref type="bibr">[13]</ref> [12], <ref type="bibr">[14]</ref> but take longer to compute and restrict these techniques to be offline. The stochasticity and non-stationarity of real-world conditions demand the replanning or adaptation of these trajectories, which at present can only be achieved through a highlyengineered synergy between the offline full order TO and online reduced order MPCs and WBIC's <ref type="bibr">[4]</ref>, which requires rigorous modeling and tuning.</p><p>On the other end of the spectrum, data-driven methods have shown compelling results in synthesizing controllers for legged robots <ref type="bibr">[17]</ref>, <ref type="bibr">[2]</ref>. Their success can be attributed to their exposure to full-order dynamics, allowing them to learn unrestricted, efficient and novel behaviours, unlike their model-based equivalents. Additionally, they provide a more flexible framework to integrate and model the uncertainty observed in real-world conditions, thereby enhancing their sim-to-real transfer <ref type="bibr">[16]</ref>. <ref type="bibr">[10]</ref> recently showed that such policies for controlling a highly non-linear system typically parameterized as deep neural networks tend to linearize it such that a low-dimensional linear system can represent the entire high-dimensional closed-loop system, thus shining light on a major drawback of deep reinforcement learning based control, i.e. interpretability and safety. These findings complement our prior works <ref type="bibr">[8]</ref>, <ref type="bibr">[7]</ref> that learnt linear control policies for robust blind bipedal walking in simulation and showed direct transfer to hardware. Despite the policy's success in bipedal walking, the use of a hand-picked foot trajectory limits extending the framework to more athletic behaviours such as hopping. Selecting such task-specific trajectories is infeasible and hence calls for the utility of model-based planning. Thus, in this work, we propose to extend our design philosophy of learning linear control policies for realizing bipedal hopping through the integration of a dynamical model. Bipedal hopping is an ideal candidate for such a motion as it requires establishing an optimal balance of handling the body momentum and precise foot placements after significant flight phases. <ref type="bibr">[15]</ref> demonstrates agile hopping but at the cost of design simplifications of the deployed hardware, and hence cannot be readily extended to full-order humanoids. Hence, we propose alleviating such design limitations by learning the controller in simulation. Our contribution is twofold.</p><p>1) Providing a control framework for integrating reduced order models in model-free learning and realizing robust bipedal hopping.</p><p>2) A simple yet effective policy structure to integrate any number of diverse control strategies to automate the cumbersome parameter tuning (as found in model-based control) through training in simulation.</p><p>The paper is structured as follows: In Section II, we present the control framework and meticulously explain our design choices. Section III showcases the results and analysis followed by the conclusion in Section IV.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>II. METHODOLOGY</head><p>In this section, we describe our control framework and explain the working of the high-level linear policy in adherence with the low-level phase controller. The experiments were performed on the humanoid robot Digit. Digit is a 30 degrees of freedom (DoF) 3D biped developed by Agility Robotics, USA. The total weight of the robot is 48 kg, from which 22 kg corresponds to the upper body, and 13 kg to each leg. The kinematic structure of Digit is extensively discussed in our previous work <ref type="bibr">[7]</ref>.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>A. Overview of the Control Framework</head><p>In <ref type="bibr">[7]</ref>, we developed a policy that enabled robust walking by modulating an elliptical trajectory for the feet using a linear policy. This approach limits the realisation of more athletic behaviours since elliptical trajectories are restrictive in nature. In our experiments, extending <ref type="bibr">[7]</ref> to hopping, we found elliptical trajectories to be inadequate, causing the policies to fail. Gaits like hopping require more complex trajectories, which cannot be hand-picked. Therefore, we propose to integrate the linear policy with a model that plans the desired template motion online once every hopping cycle. This embedded model provides a reference trajectory to the linear policy for executing arbitrary motions, unlike fixed foot trajectory modulators with fixed gaits <ref type="bibr">[8]</ref>, thereby extending the framework. It is worth mentioning that the framework is independent of the chosen model and can be readily extended to diverse models as per the task and motion requirements. For instance, in our case, we use a SLIP model for hopping, whereas, for more complicated motions like backflips and parkour, we believe the approach could be extended to centroidal and full-order models. Thus the proposed hierarchical control framework, as shown in Fig. <ref type="figure">2</ref>, has a model-embedded linear policy at the high level and a phase controller, transforming the policy's actions at the low level. The learnt linear policy generates unscaled motor commands for the leg joints during the stance phase and desired foot placement in the flight phase. The policy's outputs are then transformed to the appropriate scale or target joint angles through inverse kinematics by the phase controller in accordance with the given phase. These controllers are further explained in detail as follows.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>B. High-Level Model Embedded Linear Policy</head><p>To generate motions with significant flight phases, we integrate the learnable linear policy with a Spring-Loaded Inverted Pendulum (SLIP) model as it is the fastest and simplest model that can generate trajectories for athletic behaviours like hopping and running in real-time <ref type="bibr">[3]</ref>. A new SLIP trajectory is generated at every touchdown with Digit's reduced current state as initial conditions. This trajectory is then used as a reference by the linear policy for the upcoming stance and flight phase. The holistic linear policy is parameterized by a single matrix with different rows pertaining to different outputs for stance and flight phases. By being a part of the same policy, different feedback terms are trained together and converge to an optimal consensus between trajectory tracking and torso balance.</p><p>1) Trajectory Generation: The SLIP model assumes a point mass m and a springy leg of length l 0 with spring constant k. The dynamics in polar coordinates for the SLIP model during stance, with the foot anchored at the origin, are given as:</p><p>(1)</p><p>Owing to the low inertia legs of Digit, the centre of mass can be reasonably approximated to be the base frame defined at the hip, as most of the mass is concentrated around the pelvis. Hence, we use the robot's base state as the state of the COM for SLIP. At every touchdown, the initial condition for the SLIP model is given as:</p><p>where X b and Z b are x and z coordinates of the base with respect to the feet, and &#7818;d b and &#379;d b are the desired longitudinal and vertical velocities, respectively. Due to the symmetry between the liftoff and touchdown velocities, we initialise the trajectories with the desired heading velocities.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>&#379;d</head><p>b is calculated based on the desired maximum jump height. The dynamics equations are numerically integrated over time to obtain a trajectory for the stance phase. It is worth noting that the above equation is for a 2D SLIP model where the motion is constrained to the sagittal plane, unlike the unconstrained real system in 3D. However, since the policy is trained to learn stabilizing behaviours for the fullorder system in simulation, it is unaffected by the fidelity of the template model, thereby proving its robustness and generality.</p><p>2) Observation Space: In <ref type="bibr">[7]</ref>, the effectiveness of using a reduced observation space was exhibited. Keeping up with the same philosophy, we formulate the observation space with torso orientation and trajectory tracking errors, allowing the linear policy to marginally deviate from the reference trajectories and trade-off accurate tracking for torso balance. Thus observation space is a 13-dimensional state vector defined as- </p><p>Where and &#10003; are the torso roll and pitch, and &#729; and &#10003; are the corresponding angular velocities. e &#8676; b = &#8676; d &#8676;, where &#8676; &#9999; { , x b , y b , z b , &#7819;b , &#7823;b , &#380;b }, are the current heading yaw, base positions and linear velocities with respect to the feet and</p><p>, are the corresponding desired terms which are obtained from the SLIP trajectory, and user commands. e &#7819;b is the error between the mean velocity of a hopping step and the desired velocity of the base along the heading direction. Akin to <ref type="bibr">[15]</ref>, x np is the nominal foot position calculated as x np = 0.5 &#7819;b T s , where T s is the time period for the last stance phase. However, by providing e &#7819;b and x np as inputs to the policy, we learn the relative weighting between the nominal and feedback terms through training and thereby enabling it to handle early contacts. A normalised constant (1) is passed for estimating the stiffness constant k for the SLIP model as it is independent of any of the system's state variables.</p><p>3) Action Space: The action space of the policy can be broadly classified into four parts: a) Torque actions for stance: During stance, the policy outputs unscaled torques for the hip (hr, hp) and knee (kp) motors. Since there is a significant difference in the magnitude of torques produced by position and velocity feedback, denoted by a p &#8676; and a v &#8676; respectively, where &#8676; &#9999; {hr, hp, kp}, we assign separate dedicated rows in the policy to output the corresponding torques and thereby keeping the matrix elements normalised. These torques are then appropriately scaled and added together to form the final joint torques. An additional action a l kp is also inferred from the policy for maintaining lateral stability. This action is added to the knee joint of one leg and subtracted from the other. The resulting torque difference accounts for nullifying the lateral disturbances.</p><p>b) Ankle transmission for stance: For the ankles, the policy provides the target roll and pitch values with respect to the current roll and pitch of the ankle. These actions are denoted as qd ap and qd ar . c) Target foot placement during flight: The policy estimates a fixed target foot position for the touchdown with dynamic adjustments to correct for external disturbances. The fixed target position is denoted as x f and the dynamic offsets along the x and y directions are denoted as xf and &#253;f . A target joint angle for the hip yaw motor, denoted by q d hy , is also estimated to control the desired heading yaw.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>d) Model parameter for SLIP:</head><p>The policy also estimates the stiffness constant k for the SLIP model. This is done to identify a spring constant that would closely mimic the dynamics in the template motion.</p><p>Hence, the action space is a 12-dimensional vector defined as-</p><p>4) Policy Matrix Sparsification: With <ref type="bibr">[7]</ref>, we observed that sparsification of the matrix could help reduce the overfitting of the policy to the simulation dynamics, speed up the training, and enhance the transfer to hardware. In our current work, the sparsification becomes even more critical as we combine actions for different phases in the same policy. For example, the flight phase touchdown position commands are not related to the stance phase's trajectory errors. Therefore, we learn only the relevant parameters that correlate an observation to action and set all the other terms to 0, as seen in Fig. <ref type="figure">3</ref>. We give a warm start to the training by initialising it with a sub-optimal policy that is able to hop for a step or two before falling. </p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>C. Low-Level Phase Controller</head><p>The hopping gait has two phases: stance and flight, which are separated by the events of touchdown and lift-off. At the low level, the phase controller is responsible for switching the control received from the policy between the different phases and processing the actions accordingly.</p><p>While landing from the flight, the touchdown is detected through the compression in a passive spring attached to the shin of the Digit's leg. When Digit reaches back to a height equal to that of the touchdown, we assume that the lift-off has occurred in accordance with the template model. The phase controller processes the actions from the policy depending upon the phase as follows.</p><p>Stance: During stance, the gait controller performs torque scaling and ankle regulation.</p><p>a) Torque Scaling: As discussed before, we scale the unscaled actions and do a weighted sum of the position, and velocity feedback torques as follows,</p><p>where &#8999; i is the motor torque, S p i and S v i are the scaling factors for position and velocity components, respectively. i &#9999; {hr, hp}. For the knee pitch, we have:</p><p>The value of F is defined as, F = 1 for the right leg, and F = 1 for the left leg. Hence, the left and the right knee can have a difference in torque, allowing for auxiliary and asymmetric lateral stability. b) Ankle Transmission: As the generated SLIP trajectories assume a point-foot and do not consider 2 DoF at the ankle, we need an explicit transmission for the ankle due to the non-linear relation between the desired angles and the ankle commands <ref type="bibr">[7]</ref>. The actuation at the ankle is crucial in maintaining Digit's balance during stance. The target roll and pitch for the ankle are obtained as follows.</p><p>q d ar = q ar + F(q d ar Q of f ar ) (8)</p><p>where q d ar and q d ap are the desired angles and Q of f ar = 0.366 rad and Q of f ap = 0.04 rad are the joint offsets in accordance to the kinematic assembly for the ankle joints. The desired roll and pitch commands for the ankles are transformed into the motor commands through ankle kinematics approximated using non-linear regression.</p><p>Flight: During the flight phase, the phase controller receives the target x foot position from the policy. We consider a straight-line trajectory for the feet from the position at the liftoff to the target touchdown. This trajectory is dynamically altered at each control step with foot placement residues, xf and &#253;f . Ideally, the trajectory is to be tracked in time T F , which is the average time of flight of the gait. At time t during the flight phase, the target foot positon is calculated as follows.</p><p>where S x f = 20, S xf = 10 and S &#253;f = 20 are the scaling factors for the actions from the policy, x i f is the feet position with respect to the base at liftoff, and Y of f f = 0.1m and Z of f f = 1.0m are the offsets for the feet with respect to the base.</p><p>The desired joint positions are then obtained using Inverse Kinematics (IK) and tracked with a joint-level PD controller. The ankle joints are kept passive and the target angle for hip yaw motors (q d hy ), is directly tracked with a PD controller. It is to be noted that all the scaling factors were constant for different simulation and hardware settings.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>D. Policy Training</head><p>Similar to our previous work, we use Augmented Random Search (ARS) for training the linear policy. Unlike generic ARS, we only search in a sub-space of the parameter space, R m&#8677;n . The linear policies trained using ARS are found to be effective for solving various continuous-control problems and require minimal tuning of hyper-parameters <ref type="bibr">[11]</ref>.</p><p>1) Reward Function: The reward function is formulated with the objective to keep the torso upright while having sufficiently long flight phases and simultaneously tracking the commanded heading velocities. It is defined as, r = G w1 ( ) + G w2 (&#10003;) + G w3 (e ) + G w4 (e &#7819;b ) + f r <ref type="bibr">(13)</ref> where f r = 0.2 in the flight phase and f r = 0.0 otherwise. The mapping G : R ! [0, 1] is the Gaussian kernel given by G wj (u) = exp( w j &#8677; u 2 ), w j &gt; 0. 2) Training Curriculum: We only train on flat ground across arbitrary heading velocity commands. The commanded heading velocity is uniformly sampled from the range [0.0, 0.2] and either remains constant throughout the episode or increases with a step of 0.1 m/sec up to 0.2 m/sec every 5 seconds. Alongside, we also train for commanded lateral velocities sampled from [ 0.05, 0.05] m/sec and commanded heading yaw varied randomly in a quasistatic way. The length of each episode is 15 seconds. An episode is terminated if the robot topples or the robot's height decreases below a certain threshold, or if the maximum episode length is reached. Additionally, we introduce noise in the observation, randomise the scaling factors in the phase controller and arbitrarily reduce the saturation limits for torques to further robustify the policy.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>III. RESULTS</head><p>This section presents the simulation results, sim-to-sim transfer of the policies, and the preliminary hardware experiments conducted. We use a custom gym environment with the MuJoCo physics engine <ref type="bibr">[19]</ref> for training our policies in simulation. We show the sim-to-sim transfer to the Agility Robotics (AR) Simulator, a simulation platform proprietary to AR. We also show preliminary sim-to-real transfer of the policies to the hardware.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>A. MuJoCo Simulation Results</head></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>1) Performance Analysis:</head><p>We train the policies on flat terrain for varied forward and lateral velocities, and yaw control. A 2D SLIP model is used to produce trajectories in the sagittal plane and the template motion is extended  and phase portraits settling into a stable limit cycle after disturbance from the uneven terrain (right)</p><p>to a complex 3D system like Digit, which is capable of performing lateral hops and adjusting the heading angle, by virtue of the policy design. The policy predicts actions for a single leg. As a result of the symmetric structure of Digit and most humanoid robots, the commands for one leg can be mirrored to the other leg. We are able to achieve accurate tracking of heading velocities from 0 m/sec up to 0.52 m/sec and lateral velocities from 0.05 m/sec up to 0.05 m/sec as shown in Fig. <ref type="figure">4</ref>. The heading yaw can be controlled freely, allowing the coverage of the complete 3D space with an omnidirectional heading. Maximum jumping height is increased with increasing longitudinal velocities to provide long enough flight phases for the feet to transition to the touchdown position. The maximum jumping height varies from 0.04 meters at 0 m/sec to 0.15 meters at 0.52 m/sec heading velocity. The main challenges in increasing the maximum jumping height were torso pitch disturbances and motor torque saturation at lower and higher longitudinal velocities, respectively. In adherence to the well-trained policy, the torso's oscillations were minimal and contained within the following ranges :</p><p>2 ( 1 , 1 ), &#10003; 2 ( 5 , +7 ), 2 ( 1 , 1 ). It can also be observed from Fig. <ref type="figure">6</ref> how the trained policy roughly establishes a proportional relationship between the heading velocity and the target foot placement during the flight phase.</p><p>2) Policy Generalisation and Robustness: We tested the policy trained on flat terrain under different terrain conditions, and it was found to be insusceptible to minor changes in the terrain. The policy performed well on the slopes ranging from 11 to 11 without any terrain feedback or explicit training. The changes in the phase portraits for the slopes as compared to the flat terrain is shown in Fig. <ref type="figure">5</ref> (left). The policy successfully handled step disturbances in the terrain of up to 7 cm in height. The limit cycles of the   leg joints can be seen reacting to the terrain disturbances in Fig. <ref type="figure">5</ref> (right). The policy also rejected longitudinal forces ranging from 40 N to 40 N and lateral forces ranging from 30 N to 30 N applied periodically during the stance phase. All the mentioned results are presented in the accompanying video submission.</p><p>3) Comparison with the Baseline: To further evaluate the performance of the trained policy, we compare it against two manually tuned controllers. 1) Raibert's Linear Hopping with foot placement and attitude correction. 2) Task-space SLIP trajectory tracking with look-up table for foot placement.</p><p>Both controllers work only with active ankle and torso regulations which are manually tuned. For the Raibert's hopping controller, as the natural structure of Digit doesn't resemble a single springy leg, we assume legs to function like SLIP template model and perform energy shaping to jump to the desired height. We compare the controllers on different metrics which act as indicators of robustness and reliability. The results are shown in I. It can be observed that the trained policy performs substantially better on all the metrics.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>B. Sim-to-sim transfer</head><p>To further evaluate the generalisation of the policy to different dynamic properties of the robot and the environment, we tested the policy in the AR simulator with no additional tuning to the policy parameters. The AR simulator is a high-fidelity real-time simulator that includes parameters corresponding to the actual hardware such as joint friction, communication delays, state estimation, etc.</p><p>In addition, testing the policy in the AR simulator results in an additional layer of safety that is important in the simto-real process to prevent non-expected behaviors on the real hardware.</p><p>We perform tests for velocity tracking and control of the heading angle, and robustness to challenging terrains. The performance of the learned policy for velocity tracking and control of heading angle are presented in Fig. <ref type="figure">7</ref>. The policy effectively tracked longitudinal velocities from 0 m/sec up to 0.3 m/sec and lateral velocities from 0.05 m/sec up to 0.05 m/sec. We notice that the maximum velocity reached by the policy in the AR simulator is lower than the one obtained in the custom MuJoco simulator, which is a common effect in sim-to-sim and sim-to-real transfer, specially for dynamic maneuvers <ref type="bibr">[1]</ref>. The combination of this commands allow the robot to jump in diagonal directions. In addition, the policy successfully tracked the continuous variations of the desired heading angle while jumping forward. Finally, we also tested the policy under different terrain conditions. The controller successfully traversed through challenging terrains, including slopes in a range of <ref type="bibr">[ 10,</ref><ref type="bibr">10]</ref> degrees. These results can be seen in the accompanying video submission. </p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>C. Preliminary hardware tests</head><p>We present some preliminary results of the testing of the leaned policy in the hardware of the Digit robot. Given the highly dynamic behaviors performed by the policy, there exists critical components that affect the performance of the policy in the real hardware, being one of them the state estimation algorithm used to compute the position and velocity of the robot.</p><p>Unfortunately, during the hardware tests, we found that the default state estimation algorithm included in the closedsource low-level API of the robot is sensitive to the takeoff and touchdown events. In particular, during the flight-phase and landing, there is a significant drift on the estimation of the vertical position and velocity of the robot's base. This estimation errors introduce significant disturbances to the policy input, causing high variance in the policy outputs, which results in instability of the whole system.</p><p>To address this problem, we use Forward Kinematics (FK) to compute the height of the robot's base during the double support phase and compensate for the drifting in the state estimation. Although the resulting motion is not highly dynamic like the one obtained in simulation, the robot is able to take jumps with prolonged phases of double support, in which the FK-based estimation of the robot's height is used to compensate for the drifting in the state estimator with the addition of a balancing controller to improve the stability of the stance phase.</p><p>Future work will focus on improving the state estimation algorithm to obtain reliable measurements of the robot's state that fully enables the deployment of the learned policy. The preliminary results for the hardware tests can be seen in the accompanying video submission.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>IV. CONCLUSION</head><p>In summary, we presented a linear policy integrated with the SLIP template model, capable of performing robust and continuous hopping motion in 3D space. We exhibit the generalisation of the linear policy through successful trials on adverse simulation conditions, sim-to-sim transfer to a high-fidelity real-time simulator and preliminary experiments on the hardware. The proposed control framework presents a generic approach for performing agile and dynamic manoeuvres. Although, in this work we use SLIP model to demonstrate a particular case of hopping, we believe our approach can be readily extended to more complex behaviours.</p><p>Future research directions include extensive testing on the hardware with improved state estimation and integration of more complex models in the linear policy for more dynamic behaviours like running. The preliminary results for the hardware tests can be seen in the accompanying video submission. An extended version of the video with additional experiments can be seen here: <ref type="url">https://youtu</ref>. be/iSK2JeBEPLw</p></div><note xmlns="http://www.tei-c.org/ns/1.0" place="foot" xml:id="foot_0"><p>Authorized licensed use limited to: The Ohio State University. Downloaded on March 20,2024 at 19:04:26 UTC from IEEE Xplore. Restrictions apply.</p></note>
		</body>
		</text>
</TEI>
