We study the problem of learning and optimizing through physical simulations via differentiable programming. We present DiffSim, a new differentiable programming language tailored for building high-performance differentiable physical simulations. We demonstrate the performance and productivity of our language in gradient-based learning and optimization tasks on 10 different physical simulators. For example, a differentiable elastic object simulator written in our language is 4.6x faster than the hand-engineered CUDA version yet runs as fast, and is 188x faster than TensorFlow. Using our differentiable programs, neural network controllers are typically optimized within only tens of iterations. Finally, we share the lessons learned from our experience developing these simulators, that is, differentiating physical simulators does not always yield useful gradients of the physical system being simulated. We systematically study the underlying reasons and propose solutions to improve gradient quality. Figure 1: Left: Our language allows us to seamlessly integrate a neural network (NN) controller and a physical simulation module, and update the weights of the controller or the initial state parameterization (blue). Our simulations typically have 512 ∼ 2048 time steps, and each time step has up to one thousand parallel operations. Right: 10 differentiable simulators built with DiffSim. Differentiable physical simulators are effective components in machine learning systems. For example, de Avila Belbute- Peres et al. (2018a) and Hu et al. (2019b) have shown that controller optimization with differentiable simulators converges one to four orders of magnitude faster than model-free reinforcement learning algorithms. The presence of differentiable physical simulators in the inner loop of these applications makes their performance vitally important. Unfortunately, using existing tools it is difficult to implement these simulators with high performance. We present DiffSim, a new differentiable programming language for high performance physical simulations on both CPU and GPU. It is based on the Taichi programming language (Hu et al., 2019a) . The DiffSim automatic differentiation system is designed to suit key language features required by physical simulation, yet often missing in existing differentiable programming tools, as detailed below: Megakernels Our language uses a "megakernel" approach, allowing the programmer to naturally fuse multiple stages of computation into a single kernel, which is later differentiated using source code transformations and just-in-time compilation. Compared to the linear algebra operators in TensorFlow (Abadi et al., 2016) and PyTorch (Paszke et al., 2017) , DiffSim kernels have higher arithmetic intensity and are therefore more efficient for physical simulation tasks. Imperative Parallel Programming In contrast to functional array programming languages that are popular in modern deep learning (Bergstra et al., 2010; Abadi et al., 2016; Li et al., 2018b) , most traditional physical simulation programs are written in imperative languages such as Fortran and C++. DiffSim likewise adopts an imperative approach. The language provides parallel loops and control flows (such as "if" statements), which are widely used constructs in physical simulations: they simplify common tasks such as handling collisions, evaluating boundary conditions, and building iterative solvers. Using an imperative style makes it easier to port existing physical simulation code to DiffSim. Flexible Indexing Existing parallel differentiable programming systems provide element-wise operations on arrays of the same shape, e.g. can only be expressed with unintuitive scatter/gather operations in these existing systems, which are not only inefficient but also hard to develop and maintain. On the other hand, in DiffSim, the programmer directly manipulates array elements via arbitrary indexing, thus allowing partial updates of global arrays and making these common simulation patterns naturally expressible. The explicit indexing syntax also makes it easy for the compiler to perform access optimizations (Hu et al., 2019a) . The three requirements motivated us to design a tailored two-scale automatic differentiation system, which makes DiffSim especially suitable for developing complex and high-performance differentiable physical simulators, possibly with neural network controllers ( Fig. 1, left) . Using our language, we are able to quickly implement and automatically differentiate 10 physical simulators 1 , covering rigid bodies, deformable objects, and fluids ( Fig. 1, right) . A comprehensive comparison between DiffSim and other differentiable programming tools is in Appendix A. We have presented DiffSim, a new differentiable programming language designed specifically for building high-performance differentiable physical simulators. Motivated by the need for supporting megakernels, imperative programming, and flexible indexing, we developed a tailored two-scale automatic differentiation system. We used DiffSim to build 10 simulators and integrated them into deep neural networks, which proved the performance and productivity of DiffSim over existing systems. We hope our programming language can greatly lower the barrier of future research on differentiable physical simulation in the machine learning and robotics communities.
We study the problem of learning and optimizing through physical simulations via differentiable programming, using our proposed DiffSim programming language and compiler.
{"src": "scitldr-AIC_train", "paper_id": "B1eB5xSFvr"}