This is an open access article under the CC BY license: Al-Khwarizmi Engineering Journal Al-Khwarizmi Engineering Journal ISSN (printed): 1818 – 1171, ISSN (online): 2312 – 0789 Vol. 20, No. 3, September, (2024), pp. 20- 35 A Comprehensive Survey of Push Recovery Techniques for Standing and Walking Bipedal Robots Ibrahim Altameemi * *Robotics Engineering Department, Worcester Polytechnic Institute (WPI) *Email: ialtameemi@wpi.edu (Received 12 May 2024; Revised 23 July 2024; Accepted 29 July 2024; Published 1 September 2024) https://doi.org/10.22153/kej.2024.07.002 Abstract Bipedal robots, which mimic human or animal motion, are expected to perform numerous tasks, such as delivering healthcare, conducting search and rescue missions in dangerous environments and serving industrial applications. Bipedal robots are required to interact with objects or people in their surroundings while performing their planned tasks. The main challenge faced by these robots is their ability to maintain balance in the presence of disturbances, such as external pushing forces applied to them and uneven terrains. Therefore, a push recovery control system that enables these robots to preserve stability while executing their intended tasks must be developed. This study investigates several push recovery control algorithms for bipedal robots operating in static (standing) or dynamic (walking) modes when faced with disturbances. The study further assesses the literature based on three factors: 1) the dynamic model used to represent the robot’s behaviour, 2) the control methods and 3) the required sensors. Moreover, this review paper emphasises the challenges that must be tackled in future research. These issues include the ability of bipedal robots to adapt rapidly to changing conditions in dynamic scenarios, the substantial energy consumption they require and the delays that arise from the complex and nonlinear structure of their movements. Hence, this research suggests some recommendations for effectively tackling these difficulties. 1) Sensory feedback approaches with machine learning algorithms should be employed to develop adaptable balance control systems that quickly learn from and react to different disturbances in real time. 2) Control algorithms that optimally balance stability and energy efficiency, such as predictive control algorithms that emulate the natural reflexes of humans, should be developed. 3) Hierarchical control systems should be used to partition the balance control problem into smaller stages, thus reducing the latency issues related to solving complex nonlinear equations. Keywords: Push Recovery Control; Bipedal Robots, CoP; ZMP, Centroidal Moment Pivot; Dynamic Stability; LIPM; Whole-Body Dynamics 1. Introduction Ongoing and comprehensive research has been undertaken on bipedal robots because of their versatile capabilities in performing a wide range of tasks, including search and rescue missions, healthcare delivery and industrial applications. These tasks necessitate the engagement of bipedal robots with their surroundings, which encompass objects and people. The main challenge that these robots encounter is performing specified tasks while maintaining balance in the presence of external disturbances, such as a pushing force, uneven terrains or other factors. Hence, a push recovery control system that allows bipedal robots to preserve stability must be developed. Figure 1 demonstrates the push recovery control in which a humanoid robot counteracts a pushing force by executing a wider lateral step than existing models. This review paper aims to conduct a comprehensive examination of the current push recovery control techniques employed in bipedal mailto:ialtameemi@wpi.edu https://doi.org/10.22153/kej.2024.07.002 https://doi.org/10.22153/kej.2024.07.002 Ibrahim Altameemi Al-Khwarizmi Engineering Journal, Vol. 20, No.3, P.P. 20- 35 (2024) 21 robots under static (standing) or dynamic (walking) conditions while accounting for environmental disturbances. This research conducts a comparative analysis and categorisation of studies by focusing on three primary factors: the dynamic model employed to depict the robot’s movement, the method utilised for balance control and the sensors necessary for operation. This study examines and analyses previous research to identify the obstacles that prevent bipedal robots from performing actual tasks in real-world settings. These issues encompass the ability of bipeds to adjust quickly to unanticipated dynamic environments, the significant energy consumption and the nonlinearity and complexity of dynamics. Moreover, this study proposes recommendations for future research to tackle the challenges and improve the functionality of bipedal robots in performing real-world tasks. This review paper seeks to contribute to the continuous progress and innovation in bipedal robotics by clarifying previous accomplishments, highlighting obstacles and suggesting future research areas. Fig. 1. A humanoid robot recovers from a push (the red arrow) [1]. 2. Background 2.1. Principles of Balancing of Bipedal Robots The Zero Moment Point (ZMP) is the point where the sum of moments about the Centre of Pressure (CoP) is equal to zero, whereas the CoP is the point at which the resultant ground reaction force is normal [2-4]. A bipedal robot is stable if the ZMP remains within the boundaries of the support polygon. The ZMP and CoP coincide when they are inside the support polygon. Notably, CoP is always within the boundaries of the support polygon. If it reaches the edge, the foot will initiate rotation, thus causing the robot to be on the edge of balance. Meanwhile, the ZMP can go beyond the support polygon; in this case, it would be called a fictitious ZMP [5]. These terms are crucial stability criteria in bipedal robotics that guide the development of control techniques and algorithms for maintaining balance. The Centroidal Moment Pivot (CMP) point [6] denotes the point at which the ground reaction force vector must act to produce no torque about the Centre of Mass (CoM). As a result, the rate of change of the angular momentum is strongly correlated with the distance between CMP and ZMP. When the moment about the CoM is zero, the CMP coincides with the ZMP. However, when the CoM moment is nonzero, the extent of separation between the CMP and ZMP is equal to the magnitude of the horizontal component of the moment about the CoM divided by the normal component of the ground reaction force [6]. 2.2. Dynamics Modelling in Bipedal Robots Many dynamic modelling methods for bipedal robots provide significant insights at different levels of complexity. The Linear Inverted Pendulum Model (LIPM) [7,8] and its extensions, such as the LIPM with a Flywheel (LIPFM), help explain basic locomotion principles. Centroidal dynamics models [9] explain robot motion more accurately by considering mass distribution and its influence on movement. The maximum level of reliability is achieved through whole-body dynamics [10], which considers the complex interactions among all body parts. This approach allows for accurate simulations and precise control methods. 2.2.1. Simplified Models Linear Inverted Pendulum Model (LIPM): LIPM has been extensively employed as a simplified model for analysing the dynamics of robots in the field of push recovery control [7,8, 11,12]. In LIPM modelling, a set of multiple rigid bodies can be simplified and treated as a single mass. The total forces exerted on a system result in a change in linear momentum �̇�, which is equivalent to the acceleration of the system’s CoM (according to Newton’s law). At the same time, when the moments around the CoM are added together, the system’s angular momentum �̇� around the CoM changes. In the LIPM in the 𝑥𝑧 plane, the system is assumed to have zero angular momentum 𝐻𝐶𝑜𝑀,𝑦 = 0, and the rate of change of angular momentum �̇�𝐶𝑜𝑀,𝑦 is also zero. This condition implies that the forces exerted by the system do not create any rotational effects around Ibrahim Altameemi Al-Khwarizmi Engineering Journal, Vol. 20, No.3, P.P. 20- 35 (2024) 22 the CoM. Furthermore, in the LIPM, the height of the CoM is assumed to remain constant. The motion characteristics of the planar LIPM in the forward-backward direction, as shown in Figure 2 (a), can be mathematically represented as 𝑚�̈�𝐶𝑜𝑀 = 𝑚𝑔 𝑧0 (𝑥𝐶𝑜𝑀 − 𝑥𝐶𝑜𝑃) … (1) �̇�𝐶𝑜𝑃 = 𝑢 … (2) where 𝑥𝐶𝑜𝑀 is the CoM horizontal position at a constant height 𝑧0, 𝑚 is the system’s total mass, 𝑥𝐶𝑜𝑃 is the CoP position, 𝑔 is the gravitational acceleration, and 𝑢 is the control input for the CoP. As shown in Figure 2 (a), to satisfy the assumption that (�̇�𝐶𝑜𝑀,𝑦 = 0), the Ground Reaction Force (GRF) must point from the CoP in the direction of the CoM. Fig. 2. Simplified models (a) Linear Inverted Pendulum Model (LIPM), (b) Linear Inverted Pendulum Plus Flywheel Model (LIPFM). Linear Inverted Pendulum Plus Flywheel Model (LIPFM): The torque generated around the CoM, which is equal to the rate of change of angular momentum, can have a substantial effect on push recovery. The torque measured is a consequence of the rotating motion of the upper body, which includes the torso and the arms. Hence, in the simplified model depicted in Figure 2 (b), a flywheel positioned on the CoM should be employed as an approximation for the upper body. The system’s behaviour in the sagittal plane can be mathematically represented as follows: 𝑚�̈�𝐶𝑜𝑀 = 𝑚𝑔 𝑧0 (𝑥𝐶𝑜𝑀 − 𝑥𝐶𝑜𝑃) − 𝜏𝑦 𝑧0 … (3) 𝐼𝑦 �̈�𝑦 = 𝜏𝑦 … (4) where 𝜏𝑦 is the torque generated by the flywheel about the CoM, �̈�𝑦 is the acceleration of the flywheel, and 𝐼𝑦 is the inertia of the flywheel. Centroidal Dynamic Model: The CoM dynamic equation of bipedal robots can be formulated as [ 𝐴𝐶𝑜𝑀,1 𝐴𝐶𝑜𝑀,2 ] 𝐹𝐶𝑜𝑛𝑡𝑎𝑐𝑡 = 𝑏𝐶𝑜𝑀 … (5) where 𝐴𝐶𝑜𝑀,1 = [𝐼3×3 𝐼3×3 03×3 03×3] … (6) 𝐴𝐶𝑜𝑀,2 = [(𝑝𝑅 − 𝑐) (𝑝𝐿 − 𝑐) 𝐼3×3 𝐼3×3] … (7) 𝐹𝐶𝑜𝑛𝑡𝑎𝑐𝑡 = [ 𝐹𝑅 𝐹𝐿 𝑀𝑅 𝑀𝐿 ] … (8) 𝑏𝐶𝑜𝑀 = [ 𝑚(𝑔 + �̈�) �̇� ] … (9) 𝑝 is the position of the foot, 𝐹 is the foot contact force, 𝑀 is the foot contact moment, 𝐻 is the CoM angular momentum, and 𝑐 is the position of the CoM. The subscript 𝐿 represents the left foot, and 𝑅 represents the right foot [13] [14]. 2.2.2. Whole-body Dynamic Model Bipedal robots are floating base systems in which the robot’s base (torso or pelvis) is not fixed to the ground and is capable of moving flexibly [15]. The floating base dynamics must consider the base motion as a component of the overall system dynamics. The joint-space whole-body dynamics of floating-base bipedal robots can be described as follows: 𝑀(𝑞)�̈� + 𝑁(𝑞, �̇�) = 𝐵𝜏𝑞 + 𝐽(𝑞)𝑇𝐹𝑓 … (10) where 𝑞 ∈ 𝑅𝑛+6 denotes the generalised coordinates (𝑞 = [𝑞𝑏𝑎𝑠𝑒 , 𝑞𝑗𝑜𝑖𝑛𝑡𝑠]) and 𝑞𝑏𝑎𝑠𝑒 = [𝑥, 𝑦, 𝑧, ∅, 𝜃, 𝜑], which represent the position and orientation of the robot’s base. 𝑀(𝑞) ∈ 𝑅(𝑛+6)×(𝑛+6) is the rigid body dynamic inertial matrix, 𝑁(𝑞, �̇�) ∈ 𝑅𝑛+6 is the generalised force vector of all modelled forces including the Coriolis force, centrifugal force and gravity, 𝜏𝑞 ∈ 𝑅𝑛 is the driving torque vector, 𝐵 = [0𝑛×6 𝐼𝑛×𝑛] is the select matrix, 𝐼𝑛×𝑛 is the 𝑛-dimensional identity matrix, 𝐽(𝑞) ∈ 𝑅𝑘×(𝑛+6) is the force Jacobian matrix, 𝐹𝑓 ∈ 𝑅𝑘 is the foot contact force, and 𝑛 is the number of joints of the robot [14]. Additional details on the derivations of floating-base dynamics can be found in [15]. 2.3. Basic Balance Strategies Human balance investigations show that people have several strategies for dealing with disturbances [16-17]. Figure 3 shows three main strategies: ankle, hip and step. The ankle strategy limits hip and knee motion and uses ankle torque to move the CoM [18]. The ankle joint torque affects CoP balancing. This approach works well for minor disturbances, and the LIPM can simplify its analysis. The hip strategy is used for major Ibrahim Altameemi Al-Khwarizmi Engineering Journal, Vol. 20, No.3, P.P. 20- 35 (2024) 23 external disturbances [19]. Hip torque quickly creates a moment around the CoM, thus increasing ground reaction forces [20]. This assumption allows the LIPFM to analyse this strategy. When the disturbance increases and the ankle and hip strategies fail, a necessary step must be taken to prevent tipping over. Some studies [21,22] have integrated these three strategies to develop a more robust response to disturbances. Fig. 3. Three basic balancing strategies [23]. 3. Push Recovery Control 3.1. Push Recovery Control Techniques for Standing Bipedal Robots [24,25] demonstrate the modelling and application of ankle and hip strategies on a bipedal robot that is exposed to an unexpected pushing force while maintaining an upright standing position. A basic planar dynamical model in the sagittal plane represents both strategies. The bipedal robot is supposed to maintain balance by having a stationary support foot and vertical reaction forces that pass through the CoM. Thus, the authors of [24,25] achieve balance by ensuring that the wrenches exerted on the support foot, which are caused by the movements of the links, are reduced to zero. In the hip strategy, the reaction null-space method is utilised to acquire all joint accelerations/velocities that are necessary to maintain balance. The experiments validate the robot’s ability to respond immediately, which is similar to that of a human. The study by [23] presents a proposed method for a bipedal robot to regain balance by utilising the knee joint. A Virtual Model Control (VMC) system utilising a spring and damper is employed to achieve a compliant response for the ankle strategy. The reaction null-space method is employed to establish the correlation between the ankle and knee joints. The simulation findings demonstrate that the robot’s ability to maintain balance is enhanced when the ankle and knee joints are engaged, even in the face of significant external forces as opposed to relying solely on the ankle strategy when the knee joint is immobilised. Model Predictive Control (MPC) has two types: simplified-model MPC and whole-body MPC. In the first type, simplified dynamical models are used in the MPC to predict the control sequences over a receding horizon into the future. Although a simplified-model MPC can execute the desired tasks in real time at a high rate, it limits the capability of bipedal robots to behave naturally and precisely. Hence, other researchers use whole-body MPC, which allows robots to perform tasks with high accuracy. The main drawback of the whole- body MPC is the optimisation problem, which is computationally expensive. In [26], a convex Model Hierarchy Predictive Control (MHPC) is developed to stabilise a bipedal robot called BRUCE in its normal upright position following a push. The MHPC employs the simplified model (centroidal dynamics) in the short term while relying on whole-body dynamics in the long term. This approach allows for the benefits of quick execution and precise actions. The optimisation problem is expressed as a Quadratic Program (QP) with two terms (the first term represents the operational space tasks, such as the upright posture of the body and the trajectory of the swing leg, while the second term aims to achieve the desired CoM position and momenta). Experimental results show the ability of BRUCE to maintain balance and return to an upright posture in a natural and compliant manner after being pushed with a constant pushing force. [20] proposes a push recovery control system that utilises foot positioning. The robot is equipped with an Attitude and Heading Reference System (AHRS) that monitors the CoM’s position and velocity changes in response to the push. The robot is represented mathematically as an inverted pendulum of constant length, and the length of the stride is calculated to counterbalance the kinetic energy increase that occurs upon impact. Fig. 4. Illustration of the convex MHPC framework [26]. Ibrahim Altameemi Al-Khwarizmi Engineering Journal, Vol. 20, No.3, P.P. 20- 35 (2024) 24 The study by [27] introduces a balance controller that enables a humanoid to regain stability after experiencing significant disturbances while maintaining an upright stance. The study employs a combination of a linear quadratic regulator and computed torque control to determine the desired position of the CoP. Next, a method including integral control is suggested to regulate the position of the CoP precisely. A practical hierarchical push recovery approach is introduced in [28]. The approach comprises low- level controllers that execute three push recovery processes, which focus on the ankle, hip and step. To select from these strategies, a high-level controller uses the reinforcement learning approach in stochastic policy gradients. The high- level controller utilises sensory information and the present state of the low-level controllers to select and implement the most suitable strategy. The study by [29] introduces a model-driven learning approach that enables a humanoid robot to maintain balance by adjusting its step length. The study employs a basic central pattern generation system to generate a walking motion. This system takes the amplitude vector of the leg swing as an input and creates regular signals for the leg joints. During the moment of support exchange, the system measures the trunk inclination and its rate and simultaneously estimates the step length and trunk position. The balance controller utilises these two variables for training. To maintain balance, the authors of [29] combine the simple proportional controller of the pendulum cart model with footstep error to generate an appropriate adjustment of the step length. This modification is incorporated into the gradient function. Next, the gradient function is employed to train the balance control function to determine the swing leg amplitude. [22] proposes a partition-aware controller for balanced basins. This controller is enhanced to incorporate the captured-stepping approach, which aims to reestablish balance. The balanced state boundaries in the ankle and hip strategy are actively utilised as explicit criteria for selecting among the ankle, hip and captured-stepping methods. Thus, they provide the current estimated CoM state of the robot. At all times, the ankle subcontroller is active. Simultaneously, the activation of the hip and captured-stepping subcontrollers occurs only when the estimated CoM state exits the balanced basins for the ankle and hip, respectively. This study formulates capturability by taking into account the whole- body system dynamics of a humanoid robot to establish suitable conditions for balance recovery steps. In [11], the ankle strategy is presented as a means of maintaining stability for a humanoid robot that experiences external pushing forces. The ZMP/CoP position can be regulated to zero by changing the reference joint position through a joint position PD controller. The simulation findings demonstrate that the simulated bipedal robot can counteract external shocks of up to 10 N, thus resulting in a reduction of oscillation by 60%. In [30], a push recovery control method is developed to determine when and where to take a step to prevent falling. A new term called capture point is defined, which is a location on the ground that the bipedal robot must move to maintain stability. To derive this point, the concept of orbital energy is applied to LIPM. However, if LIPFM is employed to represent the dynamics, multiple points can be calculated. In this case, a new term called capture region is defined. A capture region is a set of capture points that the robot can step on to preserve stability. Simulation results demonstrate that a bipedal robot can maintain stability by taking a single step in response to a pushing force. The work [13] introduces Push Recovery- Model Predictive Control (PR-MPC) as an approach to generate complete-body step recovery movements following a significant disturbance. The objective of the PR-MPC is to achieve the stationary position of the CoM just above the centroid of the support polygon. A quadratic programming problem is formulated to determine control inputs that will subsequently be inserted into the discrete representation of the LIPM to construct the trajectory of the CoM. Low-gain PD joint tracking control is employed in combination with feedforward torques generated by Dynamic Balance Force Control (DBFC) for tracking desired trajectories. The findings demonstrate that the humanoid robot, known as Sarcos, possesses the capability to regain stability and counteract disruptions through the execution of a single step. The virtual leg model depicted in Figure 5 is introduced in [12]. This model is characterised by connecting a virtual link between two virtual revolute joints, which is precisely positioned between the left and right hip joints and left and right ankle joints in the frontal plane. In their study, the authors of [12] utilise a PD controller to counteract the external force and restore the robot to the upright posture. Ibrahim Altameemi Al-Khwarizmi Engineering Journal, Vol. 20, No.3, P.P. 20- 35 (2024) 25 Fig. 5. Virtual leg model [12]. The author in [31] presents DBFC as a model- based method for obtaining the required joint torques of the whole body. This computation is derived from the movement of the CoM and the forces encountered at the contact locations. The contact forces that govern the rate of change of linear and angular momenta are determined through a constrained optimisation problem. A unified Model Predictive Control (MPC) system regulates the position of the capture point by adjusting the ZMP and the CMP using ankle and hip techniques [32]. The ZMP is utilised to regulate the capture point when it falls within the support polygon. The CMP is employed when the capture point is located outside of the support polygon. The study [33] introduces a novel control method to regulate the position of the CoM and the orientation of the trunk of a bipedal robot compliantly. The objective is to calculate the necessary torque exerted at the CoM to restore the body’s position despite external disturbances. Next, the force distribution framework is employed to allocate the needed wrench at predetermined contact locations through a constrained optimisation problem, as shown in Figure 6. Fig. 6. Proposed balancing controller in [33]. [34] presents an innovative technique for humanoid robots to recover from a push by using the rotational dynamics. The process begins by calculating the Centroidal Angular Momentum (CAM) reference in real time using the magnitude and direction of the pushing forces. Then, a quadratic optimisation problem is developed, including the CAM reference as an input. This problem aims to generate feasible whole-body motion by optimising the adjusted velocities, including the linear velocity of the CoM, the angular velocity of the hip and the translational and rotational velocities of the right and left foot. The final torque required for the robot’s desired motion is determined using the passivity-based whole- body controller provided in [35] in the last stage of the process. The experiments demonstrate the humanoid robots’ potential to maintain balance and prevent falling when standing on either one or two legs. Table 1 presents a comparison of the investigated studies that examine standing stability. The comparison is based on three key factors: the dynamical model used, the push recovery technique employed and the sensors required. 3.2. Push Recovery Control Techniques for Walking Bipedal Robots [36] introduces a push detector that relies on the CoM error. It also suggests two control strategies Ibrahim Altameemi Al-Khwarizmi Engineering Journal, Vol. 20, No.3, P.P. 20- 35 (2024) 26 to deal with pushing forces that may arise while walking effectively. The initial control method is the hip and ankle strategy, which is effective only when a force is detected before the foot is lifted off the ground. This approach produces a rotational force at the hip joint to counteract the rotational motion caused by external forces. It generates significant rotational forces at the ankle joint to generate opposing forces. Consequently, the robot would increase its step length to regain equilibrium. However, if the push is sensed during the leg’s swing phase, the knee approach is employed to flex the knee joint and slow down the movement of the upper trunk. The simulation results demonstrate the biped’s capability to regain stability after experiencing significant external forces while walking. [37] introduces a trajectory generation system, which includes a push recovery trajectory generator to deal with disturbances effectively. The walking trajectory generator utilises preview control to construct the planned CoM trajectory for regular walking. The authors of [37] enhance the generic walking trajectory by incorporating a push recovery trajectory generator that mimics human behaviour in the presence of disturbances, as shown in Figure 7. To ensure stability in the case of a push, the trajectory of the swing leg is modified to mitigate the angular momentum generated by the push on the CoM. The robot’s upper body bends simultaneously to counterbalance the disturbance caused by human reactions to external pushes. A human-like motion control pipeline is shown in Figure 8. [38] describes a push recovery control implemented via walking phase modification. It presents two controllers. The first is the normal walking pattern generator, which generates normal walking in the absence of pushing using LIPM. The system executes the push recovery controller, which is the second controller, as soon as the push occurs, as illustrated in Figure 9. Acceleration (occurring when CoM velocity increases) and deceleration (occurring when CoM velocity decreases) are the two phases of this controller. The concept of orbital energy is utilised to decide whether the robot is experiencing an external push. The leg must promptly land to transition to the deceleration phase if the push occurs during the acceleration phase. This step is required to prevent the further acceleration of the CoM velocity and to calculate a new CoM position in a manner that minimises the orbital energy. The simulation outcomes demonstrate the efficacy of the research [38] regarding recovery from a push while walking. Fig. 7. Proposed push recovery trajectory generation in [37]. The implementation of a two-level dynamic walking controller on a real robot named BRUCE is demonstrated [39]. The Divergent Component of Motion (DCM)-based high-level footstep planner is the initial controller that determines the position and duration of the upcoming steps. DCM is an enhancement to LIPM that incorporates a dynamic reference trajectory for the CoM. This modification improves the stability and naturalness of the motion. The whole-body control is the low- level controller responsible for determining the joint torques needed to create planned foot contact. It also regulates other task-space behaviours, including CoM height, swing leg position and torso orientation. Controllers for real-time applications utilise small-scale quadratic programming to guarantee optimality. The findings indicate that BRUCE can successfully recover by adjusting the position of each footstep and the duration of each step in the subsequent few movements. Ibrahim Altameemi Al-Khwarizmi Engineering Journal, Vol. 20, No.3, P.P. 20- 35 (2024) 27 Fig. 8. Human-like motion control pipeline [37]. Fig. 9. Push recovery control system [38]. The work [40] introduces a push recovery controller designed to improve the walking capabilities of bipedal robots. The K-means algorithm is modified in this study to identify the ZMP trajectories as either safe or not safe. After selecting the safe ZMP trajectory, Inverse Ibrahim Altameemi Al-Khwarizmi Engineering Journal, Vol. 20, No.3, P.P. 20- 35 (2024) 28 Kinematics (IK) is used to determine the joint angles necessary to track the trajectory. [41] presents a control technique for restoring orientational force using angular momentum control. This restoration is achieved by utilising the virtual mass-ellipsoid inverted pendulum model shown in Figure 10, which was initially presented in [42]. A novel objective function is introduced for push recovery during walking, which utilises enhanced Intrinsically Model Predictive Control (IS-MPC). The results demonstrate the robot’s ability to maintain stability when walking over rough terrain. Fig. 10. Virtual-mass-ellipsoid Inverted Pendulum Model [41]. The study [43] presents a controller designed to handle a sudden and severe push that happens when a humanoid robot is walking. The robot exhibits typical walking behaviour. When a push is recognised based on the linear velocity of its body, the push recovery control mechanism is activated. The algorithm initially computes the necessary steps for recovery and subsequently uses the preview control method described in [44] to construct the trajectory of the CoM. To mimic the deceleration phase of the LIPM (which involves a loss of energy), the authors of [44] adjust the ZMP reference by shifting it a certain distance behind the foot. IK is derived to compute the joint variables required to make the robot follow the updated ZMP reference. The aim of [45] is to develop a technique for producing locomotion patterns for humanoid robots that are simultaneously dynamic (capable of adjusting to changing circumstances) and versatile (capable of dealing with diverse scenarios). The authors of [45] suggest employing a combination of two models: the 3D actuated Spring-loaded Inverted Pendulum (3D-aSLIP) to simulate the robot’s physical dynamics and the Hybrid Linear Inverted Pendulum (HLIP) as a control mechanism to assist in producing stepping patterns. The stepping controller based on HLIP is designed to disturb the periodic walking motion of a 3D-aSLIP to provide dynamic and versatile walking, as shown in Figure 11. Fig. 11. Overview of the approach presented in [45]. The approach presented in [46] can be seen as an expansion of stepping controllers in [47] [46] [48] and [49], which use the Step-to-step (S2S) dynamics approximation to strategise footsteps for the stabilisation of bipedal walking. The process begins by implementing the Hybrid-LIP stepping technique described in [47] to produce a sufficient amount of waking data for learning. Next, the discrete horizontal CoM state and the actual step size in the S2S dynamics are obtained. The stepping controllers, based on System-level Synthesis (SLS), are designed using the learned dynamics of the S2S process. These controllers consider bounded push disturbances and kinematic constraints. The study by [50] proposes a method for detecting pushes based on changes in the angle and angular velocity of the robot’s torso. It also suggests a strategy for avoiding falls by adjusting the positioning of the robot’s feet. The rescue step size is determined by calculating the difference between the initial and final kinetic energy of the Ibrahim Altameemi Al-Khwarizmi Engineering Journal, Vol. 20, No.3, P.P. 20- 35 (2024) 29 LIPM, which takes into account that the final CoM will be located on the line connecting the original CoM and the capture point. The work [51] presents a closed-loop feedback control approach that utilises an accelerometer and gyroscope to enable a cost-effective humanoid robot to maintain balance and recover from external forces while walking. The control method consists of three stages. In the first stage, falling is detected and classified into three categories (hard push, medium push and light push) using gyroscope and accelerometer sensors. In the second stage, the appropriate response (CoP balancing, CMP balancing and step-out strategy) is selected based on the strength and direction of the push. During stage 3, nine walking parameters, such as step length, step pace, hip-pitch position and ankle-pitch position, are modified according to the magnitude and direction of the applied pushing forces. In [1], a method is proposed to enhance the dynamic stability of humanoid robots. The nominal footstep position and time are inputs to the DCM planner, which are generated by the foot position planner. With the implementation of the DCM planner, the nominal DCM trajectory and foot position are generated. Using nominal values of foot and DCM trajectories in conjunction with observations of the actual DCM (desired CoM), the authors propose a stepwise adaptor. This integration allows for the evaluation of adapted foot trajectories, footstep position and time. To compute the required torques needed to achieve these updated parameters, a whole-body QP control is implemented. To generate walking trajectories, the CoM trajectories must remain aligned with the ZMP trajectory and not deviate from it. Nevertheless, not all ZMP trajectories can be executed without the CoM diverging. In [52], a novel method for generating online walking trajectories is introduced. This method incorporates nondivergence constraints for ZMP-CoM. This approach allows for the integration of walking direction and speed adjustment, as well as recovery from unknown external forces, in a unified manner. [53] proposes a mathematical model called the Virtual Force Linear Inverted Pendulum Model (VFLIPM), which causes a bipedal robot to change its gait pattern according to external disturbances. A natural ZMP reference is utilised to generate the dynamic gait pattern. Once the push is detected using the ZMP threshold, a fuzzy controller is developed to modify the gait pattern and VFLIPM parameters (lean angle, stride length and virtual mass of VFLIPM) based on the impact. Table 2 presents a comparison of the investigated studies that specifically examine stability during walking. The walking pattern generation and push recovery strategy are emphasised for each study in addition to the dynamical models and sensors employed. Table 1 Comparison of Push Recovery Control Techniques for Standing Bipedal Robots. Reference Dynamic Model Control Method Implementation Platform Sensor Usage [24] [25] Two-link planar robot VMC with null-space method (ankle and hip strategy) Real humanoid (HOAP-2) Joint encoders [23] Two-link planar robot VMC with null-space method (ankle and knee strategy) Biped (simulation) Joint encoders [30] LIPM and LIPFM Step position control based on orbital energy concept (step strategy) Two-link robot (simulation) - [26] Whole-body dynamics for short horizon and centroidal dynamics for long horizon Model Hierarchy Predictive Control (MHPC) (whole- body motion strategy) Real biped (BRUCE) IMU and contact foot sensors [20] LIPM Foot placement control using kinetic reduction (step strategy) Biped (simulation) AHRS to measure angle and angular velocity [27] Two-link planar robot Integral control of CoP position (hip strategy) Biped (simulation) - [28] LIPM High-level control uses reinforcement learning to select among (ankle, hip Real humanoid (DARwIn-HP) IMU and joint encoders Ibrahim Altameemi Al-Khwarizmi Engineering Journal, Vol. 20, No.3, P.P. 20- 35 (2024) 30 and step strategies) [29] Simple pendulum cart model Step position control using model-driven learning approach (step strategy) Real humanoid (Copedo) Joint encoders [21] [22] Centroidal dynamics Whole body capturability (ankle, hip and step strategies) Real humanoid (DARwIn-OP) Joint encoders [13] LIPM and centroidal dynamics Step position control using Push Recovery-MPC (PR- MPC) (step strategy) Real humanoid (Sarcos) Force sensors [12] LIPM CoM position control using PD controller (ankle and hip strategy) NAO humanoid (simulation) Joint encoders [32] LIPFM Capture point control using MPC (ankle and hip strategy) SURENA III humanoid (simulation) - [11] LIPM ZMP position control using PD controller (ankle strategy) Biped (simulation) Foot force sensors and joint encoders [31] Centroidal dynamics Dynamics Balance Force Control (DBFC) (step strategy) Real humanoid (sarcos) IMU and foot force sensors [33] Centroidal dynamics CoM position and trunk orientation control using contact force optimisation (whole-body motion strategy) Real biped (DLR- Biped) Joint torques sensors [34] Whole-body dynamic model Whole-body balance control-based on Centroidal Angular Momentum (CAM) (whole-body motion) Real humanoid robot (TORO) IMU, joint encoders and force sensor at the stick to measure the pushing force Table 2 Comparison of Push Recovery Control Techniques for Walking Bipedal Robots. Reference Dynamic Model Control Method Implementation Platform Sense Usage [36] LIPM for normal walking Step length control using ankle and hip strategy for double support phase and trunk velocity control using knee strategy for swing phase 21 DoF Biped (simulation) Foot force sensor and IMU [37] LIPM for normal walking CoM trajectory generation with preview control for normal walking and push recovery trajectory generator modifies swing leg trajectory Real biped IMU and Force/Torque sensors [14] Whole-body dynamics Optimal ground reaction forces Biped (simulation) IMU and foot force sensors [38] LIPM Walking phase modification (transition between acceleration and deceleration phases) Biped (simulation) Accelerometer, body inclination sensor and foot force sensors [39] LIPM for footstep planner and whole- body dynamics for whole body control Two-level controllers (Divergent Component of Motion-based high level footstep planner and low-level Whole-body Control to compute the joint torques) Real biped (BRUCE) IMU, joint encoders and force sensors [40] LIPM Feasible and stable ZMP trajectory based on the K-Mean algorithm. Simulation Body sensor based on Human Capture Device (HCMD) [41] Virtual mass ellipsoid Intrinsically Model Predictive Humanoid HRP4 - Ibrahim Altameemi Al-Khwarizmi Engineering Journal, Vol. 20, No.3, P.P. 20- 35 (2024) 31 inverted pendulum model Control (IS-MPC) for angular momentum control (simulation) [43] LIPM Preview control for normal walking and phase change for push recovery Humanoid (simulation) Accelerometer sensor [45] 3D actuated Spring- loaded Inverted Pendulum (3D-aSLIP) Stepping controller based on Hybrid Linear Inverted Pendulum Model Atlas humanoid (simulation) IMU and joint encoders [46] Hybrid Linear Inverted Pendulum Model Step-to-step based stepping controller Real biped (Cassie) IMU and joint encoders [50] LIPM Foot placement modification based on kinetic energy error of the upper body Small scale humanoid (simulation) AHRS and foot force sensors [51] LIPM for CoP balancing and LIPFM for CMP balancing CoP balancing, CMP balancing and step strategy Real humanoid (Polaris) IMU [1] LIPM Footstep position and time based on Divergent Component of Motion planner Humanoid (iCub) (simulation) Joint and body encoders [52] LIPM Online walking trajectories with non-divergence constraints for ZMP-CoM Real biped (HRP3L-JSK) 6-Axis force sensor, accelerometer and Fiber Optic Gyroscope (FOG) [53] Virtual Force LIPM (VFLIPM) to change the ZMP and CoM to eliminate disturbances from environment A fuzzy dynamic gait pattern generator Real robot (David Junior II) Eight pressure sensors and IMU 4. Conclusion This study conducted a thorough examination of the innovative strategies and methodologies used to ensure the stability of bipedal robots in the presence of external disturbances. This work began by categorising previous studies into two categories: those that apply to standing robots and those that achieve stability while walking. Subsequently, it systemically analysed and compared current methods, which focused on three key aspects: the dynamic model, sensor integration and push recovery control techniques. Researchers in this field would benefit from this systematic classification by efficiently comparing and understanding previous studies as well as identifying research gaps that require future attention. Numerous studies have focused on joint control strategies in the context of standing, which requires the activation of specific joints (e.g., the ankle, knee, or hip) to maintain balance in the presence of external pushing forces. When implementing joint control strategies, researchers frequently use simplified dynamical models, such as LIPM and LIPFM, to simulate the robot’s behaviour. However, these simplified models fail to account for certain body parts, such as the arms and the head, which leads to unnatural responses. Consequently, previous research has attempted to generate whole-body motion to recover a push using a variety of techniques, including momentum-based balance control. As the level of disturbance increases, the robot must step to maintain stability. The capture point, which is based on the orbital energy concept, is the most systematic and efficient method in the literature for determining the step position (step strategy). Walking pattern generation, push detection and push recovery control are essential elements for push recovery control during walking. The ZMP- based approach consistently generates a walking pattern that maintains the ZMP within the support polygon, thereby promoting stable walking. The literature suggests various approaches to detect exerted pushing forces, such as measuring the difference in the CoM trajectory or analysing the kinetic energy of the robot’s body. Following a push detection, different push control systems use either reactive or predictive controllers to modify the robot’s posture and adjust the position of its footsteps, as demonstrated in previous research. Despite these advancements, some obstacles still need to be addressed: 1) Current methods often struggle to cope with unpredictable dynamic environments while also preserving real-time adaptation. Ibrahim Altameemi Al-Khwarizmi Engineering Journal, Vol. 20, No.3, P.P. 20- 35 (2024) 32 2) Current balance techniques exhibit promise in enhancing stability but with the drawback of increased energy use. 3) The dynamics of bipedal robots are complex and nonlinear, which results in increased computational requirements and latency issues. In the future, researchers should focus on combining sensory feedback methods with machine learning algorithms to make balance control systems that are adaptable and can quickly learn from and react to different disturbances in real time. Previous studies have demonstrated the effectiveness of current methods in maintaining balance in static and dynamic situations. However, the high energy consumption associated with their use is a significant drawback. Hence, future research must prioritise the development of control algorithms that effectively balance stability and energy efficiency. One possible way to enable bipedal robots to anticipate and respond to disruptions while minimising energy usage is to use predictive control algorithms that replicate humans’ natural reflexes. Several research areas can be improved to accommodate the complexity of bipedal robot dynamics. The development of hybrid control methods that incorporate model- based and data-driven approaches can significantly simplify the dynamics issue. Moreover, using hierarchical control architectures, which break the balance control problem down into simpler steps, can help reduce the time delays that come up when solving complicated nonlinear equations. To summarise, although significant advancements have been achieved in the domain of balance control for bipedal robots, ongoing innovation and interdisciplinary research are crucial to address current constraints and exploit the capabilities of these systems fully. References [1] M. Shafiee, G. Romualdi, S. Dafarra, F. J. Chavez and a. D. Pucci, "Online DCM trajectory generation for push recovery of torque-controlled humanoid robots," in IEEE- RAS 19th International Conference on Humanoid Robots (Humanoids), 2019. [2] M. VUKOBRATOVIĆ and B. Borovac, "Zero-moment point — thirty five years of its life," International Journal of Humanoid Robotics, vol. 01, p. 157–173, 2004. [3] A. Goswami, " Postural stability of biped robots and the foot-rotation indicator (FRI) point," The International Journal of Robotics Research, vol. 18, p. 523–533, 1999. [4] M. Vukobratović and J. Stepanenko, "On the stability of anthropomorphic systems," Mathematical Biosciences, vol. 15, p. 1–37, 1972. [5] A. Goswami and P. Vadakkepat, Humanoid Robotics: A Reference., Springer, 2020. [6] M. B. Popovic, A. Goswami and H. Herr, "Ground reference points in legged locomotion: Definitions, biological trajectories and Control Implications," The International Journal of Robotics Research, vol. 24, p. 1013–1032, 2005. [7] S. Kajita, F. Kanehiro, K. Kaneko, K. Yokoi and H. Hirukawa, "The 3D linear inverted pendulum mode: A simple modeling for a biped walking pattern generation," in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2001. [8] Z. Li, C. Zhou, H. Dallali, N. G. Tsagarakis and D. G. Caldwell, "Comparison study of two inverted pendulum models for balance recovery," in IEEE-RAS International Conference on Humanoid Robots, 2014. [9] D. E. Orin, A. Goswami and S.-H. Lee , "Centroidal Dynamics of a humanoid robot," v, vol. 35, p. 161–176, 2013. [10] E. Dantec, M. Naveau, P. Fernbach, N. Villa, G. Saurel, O. Stasse, M. Taix and N. Mansard, "Whole-body model predictive control for biped locomotion on a torque- controlled humanoid robot," in IEEE-RAS 21st International Conference on Humanoid Robots (Humanoids), 2022. [11] M. Folgheraiter, A. Yessaly, G. Kaliyev, A. Yskak, S. Yessirkepov, A. Oleinikov and G. Gini, "Computational Efficient Balance Control for a lightweight biped robot with sensor based ZMP estimation," in 18th International Conference on Humanoid Robots (Humanoids), 2018. [12] P. Ghassemi, M. T. Masouleh and A. Kalhor, "Push recovery for nao humanoid robot," in Second RSI/ISM International Conference on Robotics and Mechatronics (ICRoM), 2014. [13] B. J. Stephens and C. G. Atkeson, "Push recovery by stepping for humanoid robots Ibrahim Altameemi Al-Khwarizmi Engineering Journal, Vol. 20, No.3, P.P. 20- 35 (2024) 33 with force controlled joints," in 10th IEEE- RAS International Conference on Humanoid Robots, 2010. [14] J. Li, Z. Yuan, S. Dong, J. Zhang and X. Sang , "External Force Observer aided push recovery for torque-controlled biped robots," Autonomous Robots, vol. 46, p. 553–568, 2022. [15] R. Featherstone, Rigid Body Dynamics Algorithms Roy Featherstone, New York, NY: Springer, 2007. [16] F. Horak and L. Nashner, "Central programming of postural movements: adaptation to altered support-surface configurations," Journal of Neurophysiology, vol. 55, p. 1369–1381, 1986. [17] B. E. Maki and W. E. Mcllroy, "The Role of Limb Movements in Maintaining Upright Stance: The ‘Change-in-Support’ Strategy," Physical Therapy, vol. 77, p. 488–507, 1997. [18] J. Shan, C. Junshi and C. Jiapin, "Design of Central Pattern Generator for humanoid robot walking based on multi-objective GA," in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000), 2000. [19] D. Katic and M. Vukobratovic, "Intelligent Control Techniques for humanoid robots," in European Control Conference (ECC), 2003. [20] A. Yasin, Q. Huang, Q. Xu and M. S. Sultan, "Humanoid robot push recovery through foot placement," in IEEE International Conference on Mechatronics and Automation, 2012. [21] H. Song, W. Z. Peng and . J. H. Kim, "Partition-aware Stability Control for humanoid robot push recovery with whole- body capturability," Journal of Mechanisms and Robotics, vol. 16, p. 1, 2023. [22] W. Z. Peng, C. Mummolo, H. Song and J. H. Kim, "Whole-body balance stability regions for multi-level momentum and stepping strategies," Mechanism and Machine Theory, vol. 174, p. 104880, 2022. [23] H. Ono, T. Sato and K. Ohnishi, "Balance recovery of ankle strategy: Using knee joint for biped robot," in 1st International Symposium on Access Spaces (ISAS), 2011. [24] D. N. Nenchev and A. Nishio, "Ankle and hip strategies for balance recovery of a biped subjected to an impact," Robotica, vol. 26, p. 643–653, 2008. [25] D. N. Nenchev and A. Nishio, "Experimental validation of ankle and hip strategies for balance recovery with a biped subjected to an impact," in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2007. [26] Y. Liu, J. Shen, J. Zhang, X. Zhang, T. Zhu and D. Hong, "Design and control of a miniature bipedal robot with proprioceptive actuation for dynamic behaviors," in International Conference on Robotics and Automation (ICRA), 2022. [27] B. Stephens, "Integral control of humanoid balance," in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2007. [28] S. Yi, B. Zhang, D. Hong and D. D. Lee, "Learning full body push recovery control for small humanoid robots," in IEEE International Conference on Robotics and Automation, 2011. [29] M. Missura and S. Behnke, "Gradient-driven online learning of Bipedal push recovery," in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2015. [30] J. Pratt, J. Carff, S. Drakunov and A. Goswami, "Capture point: A step toward humanoid push recovery," in 6th IEEE-RAS International Conference on Humanoid Robots, 2006. [31] C. G. Atkeson and B. J. Stephens, "Dynamic Balance Force Control for compliant humanoid robots," in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2010. [32] M. Ashtiani, A. Koma, M. Panahi and M. Khadiv, "Push recovery of a humanoid robot based on model predictive control and Capture Point," in 4th International Conference on Robotics and Mechatronics (ICROM), 2016. [33] C. Ott, M. A. Roa and G. Hirzinger, "Posture and balance control for biped robots based on contact force optimization," in 11th IEEE- RAS International Conference on Humanoid Robots, 2011. Ibrahim Altameemi Al-Khwarizmi Engineering Journal, Vol. 20, No.3, P.P. 20- 35 (2024) 34 [34] R. Schuller, G. Mesesan, J. Englsberger, J. Lee and C. Ott, "Online centroidal angular momentum reference generation and motion optimization for Humanoid push recovery," IEEE Robotics and Automation Letters, vol. 6, no. 3, pp. 5689-5696, 2021. [35] G. Mesesan, J. Englsberger, G. Garofalo, C. Ott and A. Albu-Schaffer, "Dynamic walking on compliant and uneven terrain using DCM and passivity-based whole-body control," in Dynamic walking on compliant and uneven terrain using DCM IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids), 2019. [36] J. Zhao, S. Schütz and K. Berns, "Biologically motivated push recovery strategies for a 3D bipedal robot walking in complex environments," in IEEE International Conference on Robotics and Biomimetics (ROBIO), 2013. [37] R. C. Luo, C. Huang and W. Hung, "Bipedal robot push recovery control mimicking human reaction," in IEEE 14th International Workshop on Advanced Motion Control (AMC), 2016. [38] A. Adiwahono, C. Chew, W. Huang and V. Dau, "Humanoid robot push recovery through walking phase modification," in IEEE Conference on Robotics, Automation and Mechatronics, 2010. [39] J. Shen, J. Zhang, Y. Liu and D. Hong, "Implementation of a robust dynamic walking controller on a miniature bipedal robot with proprioceptive actuation," in IEEE-RAS 21st International Conference on Humanoid Robots (Humanoids), 2022. [40] A. Parashar, A. Parashar and S. Goyal, "Push recovery for humanoid robot in dynamic environment and classifying the data using K-mean," International Journal of Interactive Multimedia and Artificial Intelligence, vol. 4, p. 29, 2016. [41] K. Guan, K. Yamamoto and Y. Nakamura, "Push recovery by angular momentum control during 3D bipedal walking based on virtual-mass-ellipsoid inverted pendulum model," in IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids), 2019. [42] K. Guan, K. Yamamoto and Y. Nakamura, "Virtual-mass-ellipsoid inverted pendulum model and its applications to 3D bipedal locomotion on uneven terrains," in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019. [43] A. Adiwahono, C. Chew, W. Huang and Y. Zheng, "Push recovery controller for bipedal robot walking," in IEEE/ASME International Conference on Advanced Intelligent Mechatronics, 2009. [44] S. Kajita, F. Kanehiro, K. Kaneko, K. Fujiwara, K. Harada, K. Yokoi and H. Hirukawa, "Biped walking pattern generation by using preview control of zero-moment point," in IEEE International Conference on Robotics and Automation, 2003. [45] X. Xiong and A. D. Ames, "Dynamic and versatile humanoid walking via embedding 3D actuated slip model with hybrid lip based stepping," IEEE Robotics and Automation Letters, vol. 5, p. 6286–6293, 2020. [46] X. Xiong, Y. Chen and A. D. Ames, "Robust disturbance rejection for robotic bipedal walking: System-level-synthesis with step-to- step dynamics approximation," in 60th IEEE Conference on Decision and Control (CDC), 2021. [47] X. Xiong and A. Ames, "3-D underactuated bipedal walking via H-lip based gait synthesis and stepping stabilization," IEEE Transactions on Robotics, vol. 38, p. 2405– 2425, 2022. [48] J. Pratt, T. Koolen, T. de Boer, J. Rebula, S. Cotton, J. Carff, M. Johnson and P. Neuhaus, "Capturability-based analysis and control of Legged Locomotion, part 2: Application to M2V2, a lower-body humanoid,” The International Journal of Robotics Research," The International Journal of Robotics Research, vol. 31, p. 1117–1133, 2012. [49] P. Bhounsule, M. Kim and A. Alaeddini, "Approximation of the step-to-step dynamics enables computationally efficient and fast optimal control of Legged Robots," in 44th Mechanisms and Robotics Conference (MR), 2020.. [50] A. Yasin, Q. Huang, Q. Xu and W. Zhang, "Biped robot push detection and recovery," in IEEE International Conference on Information and Automation, 2012. Ibrahim Altameemi Al-Khwarizmi Engineering Journal, Vol. 20, No.3, P.P. 20- 35 (2024) 35 [51] A. Hosseinmemar, J. Baltes, J. Anderson, M. Cheng Lau, C. Fung Lun and Z. Wang, "Closed-loop push recovery for inexpensive humanoid robots," Applied Intelligence, vol. 49, p. 3801–3814, 2019. [52] J. Urata, K. Nshiwaki, Y. Nakanishi, K. Okada, S. Kagami and M. Inaba, "Online walking pattern generation for push recovery and minimum delay to commanded change of direction and speed," in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012. [53] L.-F. Wu and L. Tzuu-Hseng S., "Fuzzy dynamic gait pattern generation for real-time push recovery control of a teen-sized humanoid robot," IEEE Access, vol. 8, pp. 36441-36453, 2020.