Not only with the large host memory for supporting large scale graph processing, GPU-accelerated heterogeneous architecture can also provide a great potential for high-performance computing. However, few existing heterogeneous systems can exploit both hardware advantages to enable the scale-up performance for graph processing due to the limited CPU-GPU transmission efficiency. In this paper, we investigate the transmission inefficiency problem of heterogeneous graph systems. Our key insight is that the transmission efficiency for heterogeneous graph processing can be greatly improved by simply iterating each subgraph multiple times (rather than only once in prior work) in the GPU, further enabling to obtain the improvable efficiency of heterogeneous graph systems by enhancing GPU processing capability. We therefore present Seraph, with the highlights of pipelined subgraph iterations and predictive vertex updating, to cooperatively maximize the effective computations of GPU on graph processing. Our evaluation on a wide variety of large graph datasets shows that Seraph outperforms state-of-the-art heterogeneous graph systems by 5.42x (vs. Graphie) and 3.05x (vs. Garaph). Further, Seraph can be significantly scaled up over Graphie as fed with more computing power for large-scale graph processing.
Disk-based Graph Systems There also exist many disk-based systems to support large-scale graph processing. GraphChi @cite_19 proposes parallel sliding windows to lead only non-sequential accesses to the disk. GridGraph @cite_14 uses 2-level hierarchical partitioning to reduce the I O amount. TurboGraph @cite_0 presents a pin-and-slide model to fully exploit the multicore and I O parallism. Since the significantly low disk-to-memory bandwidth, disk-based graph systems are orders-of-magnitude slower than heterogeneous solutions.
@cite_0 @cite_19 @cite_14
1987634495 1788180225 1832683484
Graphs are used to model many real objects such as social networks and web graphs. Many real applications in various fields require efficient and effective management of large-scale graph structured data. Although distributed graph engines such as GBase and Pregel handle billion-scale graphs, the user needs to be skilled at managing and tuning a distributed system in a cluster, which is a nontrivial job for the ordinary user. Furthermore, these distributed systems need many machines in a cluster in order to provide reasonable performance. In order to address this problem, a disk-based parallel graph engine called Graph-Chi, has been recently proposed. Although Graph-Chi significantly outperforms all representative (disk-based) distributed graph engines, we observe that Graph-Chi still has serious performance problems for many important types of graph queries due to 1) limited parallelism and 2) separate steps for I O processing and CPU processing. In this paper, we propose a general, disk-based graph engine called TurboGraph to process billion-scale graphs very efficiently by using modern hardware on a single PC. TurboGraph is the first truly parallel graph engine that exploits 1) full parallelism including multi-core parallelism and FlashSSD IO parallelism and 2) full overlap of CPU processing and I O processing as much as possible. Specifically, we propose a novel parallel execution model, called pin-and-slide. TurboGraph also provides engine-level operators such as BFS which are implemented under the pin-and-slide model. Extensive experimental results with large real datasets show that TurboGraph consistently and significantly outperforms Graph-Chi by up to four orders of magnitude! Our implementation of TurboGraph is available at http: wshan.net turbograph " as executable files. Current systems for graph computation require a distributed computing cluster to handle very large real-world problems, such as analysis on social networks or the web graph. While distributed computational resources have become more accessible, developing distributed graph algorithms still remains challenging, especially to non-experts. In this work, we present GraphChi, a disk-based system for computing efficiently on graphs with billions of edges. By using a well-known method to break large graphs into small parts, and a novel parallel sliding windows method, GraphChi is able to execute several advanced data mining, graph mining, and machine learning algorithms on very large graphs, using just a single consumer-level computer. We further extend GraphChi to support graphs that evolve over time, and demonstrate that, on a single computer, GraphChi can process over one hundred thousand graph updates per second, while simultaneously performing computation. We show, through experiments and theoretical analysis, that GraphChi performs well on both SSDs and rotational hard drives. By repeating experiments reported for existing distributed systems, we show that, with only fraction of the resources, GraphChi can solve the same problems in very reasonable time. Our work makes large-scale graph computation available to anyone with a modern PC. In this paper, we present GridGraph, a system for processing large-scale graphs on a single machine. Grid-Graph breaks graphs into 1D-partitioned vertex chunks and 2D-partitioned edge blocks using a first fine-grained level partitioning in preprocessing. A second coarse-grained level partitioning is applied in runtime. Through a novel dual sliding windows method, GridGraph can stream the edges and apply on-the-fly vertex updates, thus reduce the I O amount required for computation. The partitioning of edges also enable selective scheduling so that some of the blocks can be skipped to reduce unnecessary I O. This is very effective when the active vertex set shrinks with convergence. Our evaluation results show that GridGraph scales seamlessly with memory capacity and disk bandwidth, and outperforms state-of-the-art out-of-core systems, including GraphChi and X-Stream. Furthermore, we show that the performance of GridGraph is even competitive with distributed systems, and it also provides significant cost efficiency in cloud environment.