Data structures that allow efficient distance estimation (distance oracles, distance sketches, etc.) have been extensively studied, and are particularly well studied in centralized models and classical distributed models such as CONGEST. We initiate their study in newer (and arguably more realistic) models of distributed computation such as the Congested Clique model and the Massively Parallel Computation (MPC) model, as well as in related big data models such as streaming and semi-streaming. We provide algorithms for constructing the well-known Thorup-Zwick distance oracle (or distance sketches) in these different computational models and discuss the tradeoffs between various complexity measures such as space, time, stretch, and message complexity. One key component we use to construct distance sketches are the hopsets of Elkin and Neiman 2016. In particular, we show that these hopsets can be constructed efficiently in the MPC model, even in the very low memory setting. This result has additional applications such as the first subpolynomial algorithm for approximate single-source shortest paths for weighted graphs in the low memory MPC setting.
We note that there are no bounds for distace computation for weighted graphs in the MPC model that we are aware of. It is known that many PRAM algorithms can be simulated in the MPC model ( @cite_18 , @cite_7 ), and shortest path computation is well studied in the PRAM model (e.g. @cite_4 , @cite_5 ). However, most of these algorithms use @math number of processors, in which case the simulations of @cite_18 and @cite_7 do not directly apply in MPC model with total memory of @math . Hence, these PRAM algorithm do not apply to the MPC setting directly.
@cite_5 @cite_18 @cite_4 @cite_7
2963618817 2051586153 2085630237 1503698146
A (β, e)-hopset for a weighted undirected n-vertex graph G = (V, E) is a set of edges, whose addition to the graph guarantees that every pair of vertices has a path between them that contains at most β edges, whose length is within 1 + e of the shortest path. In her seminal paper, Cohen [8, JACM 2000] introduced the notion of hopsets in the context of parallel computation of approximate shortest paths, and since then it has found numerous applications in various other settings, such as dynamic graph algorithms, distributed computing, and the streaming model. Cohen [8] devised efficient algorithms for constructing hopsets with polylogarithmic in n number of hops. Her constructions remain the state-of-the–art since the publication of her paper in STOC'94, i.e., for more than two decades. In this paper we exhibit the first construction of sparse hopsets with a constant number of hops. We also find efficient algorithms for hopsets in various computational settings, improving the best known constructions. Generally, our hopsets strictly outperform the hopsets of [8], both in terms of their parameters, and in terms of the resources required to construct them. We demonstrate the applicability of our results for the fundamental problem of computing approximate shortest paths from s sources. Our results improve the running time for this problem in the parallel, distributed and streaming models, for a vast range of s. In recent years the MapReduce framework has emerged as one of the most widely used parallel computing platforms for processing data on terabyte and petabyte scales. Used daily at companies such as Yahoo!, Google, Amazon, and Facebook, and adopted more recently by several universities, it allows for easy parallelization of data intensive computations over many machines. One key feature of MapReduce that differentiates it from previous models of parallel computation is that it interleaves sequential and parallel computation. We propose a model of efficient computation using the MapReduce paradigm. Since MapReduce is designed for computations over massive data sets, our model limits the number of machines and the memory per machine to be substantially sublinear in the size of the input. On the other hand, we place very loose restrictions on the computational power of of any individual machine---our model allows each machine to perform sequential computations in time polynomial in the size of the original input. We compare MapReduce to the PRAM model of computation. We prove a simulation lemma showing that a large class of PRAM algorithms can be efficiently simulated via MapReduce. The strength of MapReduce, however, lies in the fact that it uses both sequential and parallel computation. We demonstrate how algorithms can take advantage of this fact to compute an MST of a dense graph in only two rounds, as opposed to Ω(log(n)) rounds needed in the standard PRAM model. We show how to evaluate a wide class of functions using the MapReduce framework. We conclude by applying this result to show how to compute some basic algorithmic problems such as undirected s-t connectivity in the MapReduce framework. Shortest paths computations constitute one of the most fundamental network problems. Nonetheless, known parallel shortest-paths algorithms are generally inefficient: they perform significantly more work (product of time and processors) than their sequential counterparts. This gap, known in the literature as the “transitive closure bottleneck,” poses a long-standing open problem. Our main result is an O mn e 0 +s m+n1+ e 0 work polylog-time randomized algorithm that computes paths within (1 + O (1 polylog n ) of shortest from s source nodes to all other nodes in weighted undirected networks with n nodes and m edges (for any fixed e 0 >0). This work bound nearly matches the O d sm sequential time. In contrast, previous polylog-time algorithms required nearly min O d n3 , O d m2 work (even when s =1), and previous near-linear work algorithms required near- O ( n ) time. We also present faster sequential algorithms that provide good approximate distances only between “distant” vertices: We obtain an O m+sn n e 0 time algorithm that computes paths of weight (1+ O (1 polylog n ) dist + O ( w max polylog n ), where dist is the corresponding distance and w max is the maximum edge weight. Our chief instrument, which is of independent interest, are efficient constructions of sparse hop sets . A ( d ,e)-hop set of a network G =( V,E ) is a set E * of new weighted edges such that mimimum-weight d -edge paths in V,E∪E* have weight within (1+e) of the respective distances in G . We construct hop sets of size O n1+ e 0 where e= O (1 polylog n ) and d = O (polylog n ). We study the MapReduce framework from an algorithmic standpoint, providing a generalization of the previous algorithmic models for MapReduce. We present optimal solutions for the fundamental problems of all-prefix-sums, sorting and multi-searching. Additionally, we design optimal simulations of the the well-established PRAM and BSP models in MapReduce, immediately resulting in optimal solutions to the problems of computing fixed-dimensional linear programming and 2-D and 3-D convex hulls.