PKDGRAV3: Beyond Trillion Particle Cosmological Simulations for the Next
  Era of Galaxy Surveys
  We report on the successful completion of a 2 trillion particle cosmological
simulation to z=0 run on the Piz Daint supercomputer (CSCS, Switzerland), using
4000+ GPU nodes for a little less than 80h of wall-clock time or 350,000 node
hours. Using multiple benchmarks and performance measurements on the US Oak
Ridge National Laboratory Titan supercomputer, we demonstrate that our code
PKDGRAV3, delivers, to our knowledge, the fastest time-to-solution for
large-scale cosmological N-body simulations. This was made possible by using
the Fast Multipole Method in conjunction with individual and adaptive particle
time steps, both deployed efficiently (and for the first time) on
supercomputers with GPU-accelerated nodes. The very low memory footprint of
PKDGRAV3 allowed us to run the first ever benchmark with 8 trillion particles
on Titan, and to achieve perfect scaling up to 18000 nodes and a peak
performance of 10 Pflops.
