Learning-based Energy Control of Underactuated Robots

Underactuated robots, which have fewer actuators than degrees of freedom, are both common in modern robotics and notoriously difficult to control: the missing inputs rule out the standard techniques that work for fully actuated machines, and the system must instead be steered through its own dynamics. This thesis studies the swing-up and stabilization of two canonical underactuated systems, the Acrobot and the Pendubot, through the combination of energy-based control and machine learning. The work develops in three stages. First, starting from the analytical energy-based swing-up controllers of Xin and colleagues, a two-mode EnergyLQR controller is built that pairs an energy-based swing-up with a linear quadratic regulator for stabilization, governed by an explicit switching condition; a systematic search then identifies near-optimal gains for both the Acrobot and the Pendubot. Second, a physics-informed reinforcement-learning controller is designed and trained with Proximal Policy Optimization, in which the policy does not output raw torque but modulates an energy-shaping term, the observation is augmented with kinetic and potential-energy errors, and the reward is built around the total-energy error. Third, to remove the controller’s reliance on an analytically known energy function, an assumption that does not hold on real hardware, a Deep Lagrangian Network is used to estimate the system’s energy from measured motion, so that the same energy-based architecture can operate where no closed-form energy is available. Together, these contributions trace a path from a classical model-based controller, through a learning-augmented version of it, toward a controller whose physical knowledge is itself learned from data.