Scalable Streaming Tools for Analyzing $N$-body Simulations: Finding
  Halos and Investigating Excursion Sets in One Pass
  Cosmological $N$-body simulations play a vital role in studying models for
the evolution of the Universe. To compare to observations and make a scientific
inference, statistic analysis on large simulation datasets, e.g., finding
halos, obtaining multi-point correlation functions, is crucial. However,
traditional in-memory methods for these tasks do not scale to the datasets that
are forbiddingly large in modern simulations. Our prior paper proposes
memory-efficient streaming algorithms that can find the largest halos in a
simulation with up to $10^9$ particles on a small server or desktop. However,
this approach fails when directly scaling to larger datasets. This paper
presents a robust streaming tool that leverages state-of-the-art techniques on
GPU boosting, sampling, and parallel I/O, to significantly improve performance
and scalability. Our rigorous analysis of the sketch parameters improves the
previous results from finding the centers of the $10^3$ largest halos to $\sim
10^4-10^5$, and reveals the trade-offs between memory, running time and number
of halos. Our experiments show that our tool can scale to datasets with up to
$\sim 10^{12}$ particles while using less than an hour of running time on a
single GPU Nvidia GTX 1080.
