Given an array @math of @math elements and a value @math , a frequent item or @math -majority element is an element occurring in @math more than @math times. The @math -majority problem requires finding all of the @math -majority elements. In this paper we deal with parallel shared-memory algorithms for frequent items; we present a shared-memory version of the Space Saving algorithm and we study its behavior with regard to accuracy and performance on many and multi-core processors, including the Intel Phi accelerator. We also investigate a hybrid MPI OpenMP version against a pure MPI based version. Through extensive experimental results we prove that the MPI OpenMP parallel version of the algorithm significantly enhances the performance of the earlier pure MPI version of the same algorithm. Results also prove that for this algorithm the Intel Phi accelerator does not introduce any improvement with respect to the Xeon octa-core processor.
The problem of merging two data summaries naturally arises in a distributed or parallel setting, in which a data set is partitioned between two or among several data sets. The goal in this context is to merge two data summaries into a single summary which provides candidate frequent items for the union of the input data sets. In particular, in order for the merged summary to be useful, it is required that its size and error bounds are those of the input data summaries. A few years ago we designed an algorithm @cite_10 @cite_7 for merging in parallel counter-based data summaries which are the output of the @cite_18 algorithm.
@cite_18 @cite_10 @cite_7
1612155886 1694308253 2048234997 2102515914
The problem of merging multiple databases of information about common entities is frequently encountered in KDD and decision support applications in large commercial and government organizations. The problem we study is often called the Merge Purge problem and is difficult to solve both in scale and accuracy. Large repositories of data typically have numerous duplicate information entries about the same entities that are difficult to cull together without an intelligent ’’equational theory‘‘ that identifies equivalent items by a complex, domain-dependent matching process. We have developed a system for accomplishing this Data Cleansing task and demonstrate its use for cleansing lists of names of potential customers in a direct marketing-type application. Our results for statistically generated data are shown to be accurate and effective when processing the data multiple times using different keys for sorting on each successive pass. Combing results of individual passes using transitive closure over the independent results, produces far more accurate results at lower cost. The system provides a rule programming module that is easy to program and quite good at finding duplicates especially in an environment with massive amounts of data. This paper details improvements in our system, and reports on the successful implementation for a real-world database that conclusively validates our results previously achieved for statistically generated data. Merge sort is useful in sorting a great number of data progressively, especially when they can be partitioned and easily collected to a few processors. Merge sort can be parallelized, however, conventional algorithms using distributed memory computers have poor performance due to the successive reduction of the number of participating processors by a half, up to one in the last merging stage. This paper presents load-balanced parallel merge sort where all processors do the merging throughout the computation. Data are evenly distributed to all processors, and every processor is forced to work in all merging phases. An analysis shows the upper bound of the speedup of the merge time as (P- 1) log P where P is the number of processors. We have reached a speedup of 8.2 (upper bound is 10.5) on 32-processor Cray T3E in sorting of 4M 32-bit integers. Abstract An efficient multiprocessor algorithm to merge m, m ⩾ 2, sorted lists containing N elements is described. In a hierarchical-memory multiprocessor, the proposed method substantially reduces the data access costs in comparison with traditional schemes that successively merge the lists two at a time. The proposed method uses a novel partitioning algorithm to split up the m lists into range-disjoint parts of equal size. Each of the parts can then be merged concurrently and independently by a different processor using an m-way merge. The time complexity to find a partition is O(m log ( fN m ) , where f, 0 Existing statistical approaches to natural language problems are very coarse approximations to the true complexity of language processing. As such, no single technique will be best for all problem instances. Many researchers are examining ensemble methods that combine the output of successful, separately developed modules to create more accurate solutions. This paper examines three merging rules for combining probability distributions: the well known mixture rule, the logarithmic rule, and a novel product rule. These rules were applied with state-of-the-art results to two problems commonly used to assess human mastery of lexical semantics|synonym questions and analogy questions. All three merging rules result in ensembles that are more accurate than any of their component modules. The dierences among the three rules are not statistically signicant, but it is suggestive that the popular mixture rule is not the best rule for either of the two problems.