Rethinking Optimization with Differentiable Simulation from a Global Perspective
Differentiable simulation is a promising toolkit for fast gradient-based
policy optimization and system identification. However, existing approaches to
differentiable simulation have largely tackled scenarios where obtaining smooth
gradients has been relatively easy, such as systems with mostly smooth
dynamics. In this work, we study the challenges that differentiable simulation
presents when it is not feasible to expect that a single descent reaches a
global optimum, which is often a problem in contact-rich scenarios. We analyze
the optimization landscapes of diverse scenarios that contain both rigid bodies
and deformable objects. In dynamic environments with highly deformable objects
and fluids, differentiable simulators produce rugged landscapes with
nonetheless useful gradients in some parts of the space. We propose a method
that combines Bayesian optimization with semi-local 'leaps' to obtain a global
search method that can use gradients effectively, while also maintaining robust
performance in regions with noisy gradients. We show that our approach
outperforms several gradient-based and gradient-free baselines on an extensive
set of experiments in simulation, and also validate the method using
experiments with a real robot and deformables. Videos and supplementary
materials are available at https://tinyurl.com/globdiff