Springer, 2023. — 131 p. — (Springer Tracts in Advanced Robotics 156). — ISBN: 978-3-031-37831-7.
One important robotics problem is “How can one program a robot to perform a task”? Classical robotics solves this problem by manually engineering modules for state estimation, planning, and control. In contrast, robot learning solely relies on black-box models and data. This book shows that these two approaches of classical engineering and black-box machine learning are not mutually exclusive. To solve tasks with robots, one can transfer insights from classical robotics to deep networks and obtain better learning algorithms for robotics and control. To highlight that incorporating existing knowledge as inductive biases in machine learning algorithms improves performance, this book covers different approaches for learning dynamics models and learning robust control policies. The presented algorithms leverage the knowledge of Newtonian Mechanics, Lagrangian Mechanics as well and the Hamilton-Jacobi-Isaacs differential equation as inductive bias and are evaluated on physical robots.
In this book, I want to show that these two approaches of classical engineering and black-box deep networks are not mutually exclusive. One can transfer insights from classical robotics to the black-box deep networks and obtain better learning algorithms for robotics and control. To show that incorporating existing knowledge as inductive biases in Machine Learning algorithms can improve performance, three different algorithms are presented: (1) The Differentiable Newton-Euler Algorithm (DiffNEA) reinterprets the classical system identification of rigid bodies. By leveraging automatic differentiation, virtual parameters, and gradient-based optimization, this approach guarantees physically consistent parameters and applies to a wider class of dynamical systems. (2) Deep Lagrangian Networks (DeLaN) combine deep networks with Lagrangian mechanics to learn dynamics models that conserve energy. Using two networks to represent the potential and kinetic energy enables the computation of a physically plausible dynamics model using the Euler-Lagrange equation. (3) Robust Fitted Value Iteration (rFVI) leverages the control-affine dynamics of mechanical systems to extend value iteration to the adversarial reinforcement learning with continuous actions. The resulting approach enables the computation of the optimal policy that is robust to changes in the dynamics.
Each of these algorithms is evaluated on physical systems and compared to the classical engineering and Deep Learning baselines. The experiments show that the inductive biases increase performance compared to black-box Deep Learning approaches. DiffNEA solves Ball-in-Cup on the physical Barrett WAM using offline model-based reinforcement learning with only four minutes of data. The deep network models fail on this task despite using more data. DeLaN obtains a model that can be used for energy control of under-actuated systems. Black-box models cannot be applied as these cannot infer the system energy. rFVI learns robust policies that can swing up the Furuta pendulum and cartpole. The rFVI policy is more robust to changes in the pendulum mass compared to deep reinforcement learning with uniform domain randomization.
A Differentiable Newton – Euler Algorithm for Real-World Robotics.
Combining Physics and Deep Learning for Continuous-Time Dynamics Models.
Continuous-Time Fitted Value Iteration for Robust Policies.