This paper focuses on density-based clustering, particularly the Density Peak (DP) algorithm and the one based on density-connectivity DBSCAN; and proposes a new method which takes advantage of the individual strengths of these two methods to yield a density-based hierarchical clustering algorithm. Our investigation begins with formally defining the types of clusters DP and DBSCAN are designed to detect; and then identifies the kinds of distributions that DP and DBSCAN individually fail to detect all clusters in a dataset. These identified weaknesses inspire us to formally define a new kind of clusters and propose a new method called DC-HDP to overcome these weaknesses to identify clusters with arbitrary shapes and varied densities. In addition, the new method produces a richer clustering result in terms of hierarchy or dendrogram for better cluster structures understanding. Our empirical evaluation results show that DC-HDP produces the best clustering results on 14 datasets in comparison with 7 state-of-the-art clustering algorithms.
Many variants of DBSCAN have been attempted to overcome the weakness of detecting clusters with varied densities. OPTICS @cite_21 draws a reachability'' plot based on the @math -nearest neighbour distance. In the @math -axis of the plot, adjacent points follow close to each other such that point @math is the closest to @math in terms of the reachability distance'' The reachability-distance'' of object @math to object @math is the greater one between the core distance'' of @math and the distance between @math and @math . The core distance'' of @math is the minimum @math that makes @math a core'' object (the distance to its @math -nearest neighbour, @math ). . The reachability distance for each point is shown in @math -axis. Since clusters centre normally has a higher density or lower reachability distance than the cluster boundaries, each cluster is visible as a valley'' in this plot. Then a hierarchical method can be used to extract different clusters. The overall clustering performance depends on the hierarchical method employed on the reachability plot.
@cite_21
2160642098
Hierarchical clustering is common method used to determine clusters of similar data points in multi-dimensional spaces. @math algorithms, where @math is the number of points to cluster, have long been known for this problem. This paper discusses parallel algorithms to perform hierarchical clustering using various distance metrics. I describe @math time algorithms for clustering using the single link, average link, complete link, centroid, median, and minimum variance metrics on an @math node CRCW PRAM and @math algorithms for these metrics (except average link and complete link) on @math node butterfly networks or trees. Thus, optimal efficiency is achieved for a significant number of processors using these distance metrics. A general algorithm is given that can be used to perform clustering with the complete link and average link metrics on a butterfly. While this algorithm achieves optimal efficiency for the general class of metrics, it is not optimal for the specific cases of complete link and average link clustering. We consider @math -median clustering in finite metric spaces and @math -means clustering in Euclidean spaces, in the setting where @math is part of the input (not a constant). For the @math -means problem, show that if the optimal @math -means clustering of the input is more expensive than the optimal @math -means clustering by a factor of @math , then one can achieve a @math -approximation to the @math -means optimal in time polynomial in @math and @math by using a variant of Lloyd's algorithm. In this work we substantially improve this approximation guarantee. We show that given only the condition that the @math -means optimal is more expensive than the @math -means optimal by a factor @math for some constant @math , we can obtain a PTAS. In particular, under this assumption, for any @math we achieve a @math -approximation to the @math -means optimal in time polynomial in @math and @math , and exponential in @math and @math . We thus decouple the strength of the assumption from the quality of the approximation ratio. We also give a PTAS for the @math -median problem in finite metrics under the analogous assumption as well. For @math -means, we in addition give a randomized algorithm with improved running time of @math . Our technique also obtains a PTAS under the assumption of that all @math approximations are @math -close to a desired target clustering, in the case that all target clusters have size greater than @math and @math is constant. Note that the motivation of is that for many clustering problems, the objective function is only a proxy for the true goal of getting close to the target. From this perspective, our improvement is that for @math -means in Euclidean spaces we reduce the distance of the clustering found to the target from @math to @math when all target clusters are large, and for @math -median we improve the largeness'' condition needed in the work of to get exactly @math -close from @math to @math . Our results are based on a new notion of clustering stability. Given a metric space @math , @math , @math , and @math , a distribution over mappings @math is called a @math -sensitive hash family if any two points in @math at distance at most @math are mapped by @math to the same value with probability at least @math , and any two points at distance greater than @math are mapped by @math to the same value with probability at most @math . This notion was introduced by Indyk and Motwani in 1998 as the basis for an efficient approximate nearest neighbor search algorithm and has since been used extensively for this purpose. The performance of these algorithms is governed by the parameter @math , and constructing hash families with small @math automatically yields improved nearest neighbor algorithms. Here we show that for @math it is impossible to achieve @math . This almost matches the construction of Indyk and Motwani which achieves @math . Let @math be a tree space (or tree network) represented by a weighted tree with @math vertices, and @math be a set of @math stochastic points in @math , each of which has a fixed location with an independent existence probability. We investigate two fundamental problems under such a stochastic setting, the closest-pair problem and the nearest-neighbor search. For the former, we study the computation of the @math -threshold probability and the expectation of the closest-pair distance of a realization of @math . We propose the first algorithm to compute the @math -threshold probability in @math time for any given threshold @math , which immediately results in an @math -time algorithm for computing the expected closest-pair distance. Based on this, we further show that one can compute a @math -approximation for the expected closest-pair distance in @math time, by arguing that the expected closest-pair distance can be approximated via @math threshold probability queries. For the latter, we study the @math most-likely nearest-neighbor search ( @math -LNN) via a notion called @math most-likely Voronoi Diagram ( @math -LVD). We show that the size of the @math -LVD @math of @math on @math is bounded by @math if the existence probabilities of the points in @math are constant-far from 0. Furthermore, we establish an @math average-case upper bound for the size of @math , by regarding the existence probabilities as i.i.d. random variables drawn from some fixed distribution. Our results imply the existence of an LVD data structure which answers @math -LNN queries in @math time using average-case @math space, and worst-case @math space if the existence probabilities are constant-far from 0. Finally, we also give an @math -time algorithm to construct the LVD data structure. The one-dimensional Schr "odinger equation with a periodic and symmetric potential is considered, under the assumption that the energy bands do not intersect. The Bloch waves, @math , and energy bands, @math , are studied as functions of the complex variable, @math . In the complex plane, they are branches of multivalued analytic and periodic functions, @math , and @math , with branch points, @math , off the real axis. A simple procedure is described for locating the branch points. Application is made to the power series and Fourier series developments of these functions. The analyticity and periodicity of @math has some consequences for the form of the Wannier functions. In particular, it is shown that for each band there exists one and only one Wannier function which is real, symmetric or antisymmetric under an appropriate reflection, and falling off exponentially with distance. The rate of falloff is determined by the distance of the branch points @math from the real axis. We study the aggregate group nearest neighbor searching for the MAX operator in the plane. For a set @math of @math points and a query set @math of @math points, the query asks for a point of @math whose maximum distance to the points in @math is minimized. We present data structures for answering such queries for both @math and @math distance measures. Previously, only heuristic and approximation algorithms were given for both versions. For the @math version, we build a data structure of O(n) size in @math time, such that each query can be answered in @math time. For the @math version, we build a data structure in @math time and @math space, such that each query can be answered in @math time, and alternatively, we build a data structure in @math time and space for any @math , such that each query can be answered in @math time. Further, we extend our result for the @math version to the top- @math queries where each query asks for the @math points of @math whose maximum distances to @math are the smallest for any @math with @math : We build a data structure of O(n) size in @math time, such that each top- @math query can be answered in @math time. Given a set of @math n points, each is painted by one of the @math k given colors, we want to choose @math k points with distinct colors to form a color spanning set. For each color spanning set, we can construct the convex hull and the smallest axis-aligned enclosing rectangle, etc. Assuming that each point is chosen independently and identically from the subset of points of the same color, we propose an @math O ( n 2 ) time algorithm to compute the expected area of convex hulls of the color spanning sets and an @math O ( n 2 ) time algorithm to compute the expected perimeter of convex hulls of the color spanning sets. For the expected perimeter (resp. area) of the smallest perimeter (resp. area) axis-aligned enclosing rectangles of the color spanning sets, we present an @math O ( n log n ) (resp. @math O ( n 2 ) ) time algorithm. We also propose a simple approximation algorithm to compute the expected diameter of the color spanning sets. For the expected distance of the closest pair, we show that it is @math # P-complete to compute and there exists no polynomial time @math 2 n 1 - ? approximation algorithm to compute the probability that the closest pair distance of all color spanning sets equals to a given value @math d unless @math P = N P , even in one dimension and when each color paints two points. We study exact recovery conditions for convex relaxations of point cloud clustering problems, focusing on two of the most common optimization problems for unsupervised clustering: @math -means and @math -median clustering. Motivations for focusing on convex relaxations are: (a) they come with a certificate of optimality, and (b) they are generic tools which are relatively parameter-free, not tailored to specific assumptions over the input. More precisely, we consider the distributional setting where there are @math clusters in @math and data from each cluster consists of @math points sampled from a symmetric distribution within a ball of unit radius. We ask: what is the minimal separation distance between cluster centers needed for convex relaxations to exactly recover these @math clusters as the optimal integral solution? For the @math -median linear programming relaxation we show a tight bound: exact recovery is obtained given arbitrarily small pairwise separation @math between the balls. In other words, the pairwise center separation is @math . Under the same distributional model, the @math -means LP relaxation fails to recover such clusters at separation as large as @math . Yet, if we enforce PSD constraints on the @math -means LP, we get exact cluster recovery at center separation @math . In contrast, common heuristics such as Lloyd's algorithm (a.k.a. the @math -means algorithm) can fail to recover clusters in this setting; even with arbitrarily large cluster separation, k-means++ with overseeding by any constant factor fails with high probability at exact cluster recovery. To complement the theoretical analysis, we provide an experimental study of the recovery guarantees for these various methods, and discuss several open problems which these experiments suggest. As a first step toward a characterization of the limiting extremal process of branching Brownian motion, we proved in a recent work [Comm. Pure Appl. Math. 64 (2011) 1647-1676] that, in the limit of large time @math , extremal particles descend with overwhelming probability from ancestors having split either within a distance of order 1 from time 0, or within a distance of order 1 from time @math . The result suggests that the extremal process of branching Brownian motion is a randomly shifted cluster point process. Here we put part of this picture on rigorous ground: we prove that the point process obtained by retaining only those extremal particles which are also maximal inside the clusters converges in the limit of large @math to a random shift of a Poisson point process with exponential density. The last section discusses the Tidal Wave Conjecture by Lalley and Sellke [Ann. Probab. 15 (1987) 1052-1061] on the full limiting extremal process and its relation to the work of Chauvin and Rouault [Math. Nachr. 149 (1990) 41-59] on branching Brownian motion with atypical displacement. Clustering problems are well-studied in a variety of fields such as data science, operations research, and computer science. Such problems include variants of centre location problems, @math -median, and @math -means to name a few. In some cases, not all data points need to be clustered; some may be discarded for various reasons. We study clustering problems with outliers. More specifically, we look at Uncapacitated Facility Location (UFL), @math -Median, and @math -Means. In UFL with outliers, we have to open some centres, discard up to @math points of @math and assign every other point to the nearest open centre, minimizing the total assignment cost plus centre opening costs. In @math -Median and @math -Means, we have to open up to @math centres but there are no opening costs. In @math -Means, the cost of assigning @math to @math is @math . We present several results. Our main focus is on cases where @math is a doubling metric or is the shortest path metrics of graphs from a minor-closed family of graphs. For uniform-cost UFL with outliers on such metrics we show that a multiswap simple local search heuristic yields a PTAS. With a bit more work, we extend this to bicriteria approximations for the @math -Median and @math -Means problems in the same metrics where, for any constant @math , we can find a solution using @math centres whose cost is at most a @math -factor of the optimum and uses at most @math outliers. We also show that natural local search heuristics that do not violate the number of clusters and outliers for @math -Median (or @math -Means) will have unbounded gap even in Euclidean metrics. Furthermore, we show how our analysis can be extended to general metrics for @math -Means with outliers to obtain a @math bicriteria. Given a set of points @math , the k-means clustering problem is to find a set of k centers @math , such that the objective function ? x?P e(x,C)2, where e(x,C) denotes the Euclidean distance between x and the closest center in C, is minimized. This is one of the most prominent objective functions that has been studied with respect to clustering. D 2-sampling (Arthur and Vassilvitskii, Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA'07, pp. 1027---1035, SIAM, Philadelphia, 2007) is a simple non-uniform sampling technique for choosing points from a set of points. It works as follows: given a set of points @math , the first point is chosen uniformly at random from P. Subsequently, a point from P is chosen as the next sample with probability proportional to the square of the distance of this point to the nearest previously sampled point. D 2-sampling has been shown to have nice properties with respect to the k-means clustering problem. Arthur and Vassilvitskii (Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA'07, pp. 1027---1035, SIAM, Philadelphia, 2007) show that k points chosen as centers from P using D 2-sampling give an O(logk) approximation in expectation. (NIPS, pp. 10---18, 2009) and (Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques, pp. 15---28, Springer, Berlin, 2009) extended results of Arthur and Vassilvitskii (Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA'07, pp. 1027---1035, SIAM, Philadelphia, 2007) to show that O(k) points chosen as centers using D 2-sampling give an O(1) approximation to the k-means objective function with high probability. In this paper, we further demonstrate the power of D 2-sampling by giving a simple randomized (1+∈)-approximation algorithm that uses the D 2-sampling in its core. We consider distance queries in vertex-labeled planar graphs. For any fixed @math we show how to preprocess a directed planar graph with vertex labels and arc lengths into a data structure that answers queries of the following form. Given a vertex @math and a label @math return a @math -approximation of the distance from @math to its closest vertex with label @math . For a directed planar graph with @math vertices, such that the ratio of the largest to smallest arc length is bounded by @math , the preprocessing time is @math , the data structure size is @math , and the query time is @math . We also point out that a vertex label distance oracle for undirected planar graphs suggested in an earlier version of this paper is incorrect. The @math -Means clustering problem on @math points is NP-Hard for any dimension @math , however, for the 1D case there exists exact polynomial time algorithms. Previous literature reported an @math time dynamic programming algorithm that uses @math space. It turns out that the problem has been considered under a different name more than twenty years ago. We present all the existing work that had been overlooked and compare the various solutions theoretically. Moreover, we show how to reduce the space usage for some of them, as well as generalize them to data structures that can quickly report an optimal @math -Means clustering for any @math . Finally we also generalize all the algorithms to work for the absolute distance and to work for any Bregman Divergence. We complement our theoretical contributions by experiments that compare the practical performance of the various algorithms. Aiming to unify known results about clustering mixtures of distributions under separation conditions, Kumar and Kannan[2010] introduced a deterministic condition for clustering datasets. They showed that this single deterministic condition encompasses many previously studied clustering assumptions. More specifically, their proximity condition requires that in the target @math -clustering, the projection of a point @math onto the line joining its cluster center @math and some other center @math , is a large additive factor closer to @math than to @math . This additive factor can be roughly described as @math times the spectral norm of the matrix representing the differences between the given (known) dataset and the means of the (unknown) target clustering. Clearly, the proximity condition implies center separation -- the distance between any two centers must be as large as the above mentioned bound. In this paper we improve upon the work of Kumar and Kannan along several axes. First, we weaken the center separation bound by a factor of @math , and secondly we weaken the proximity condition by a factor of @math . Using these weaker bounds we still achieve the same guarantees when all points satisfy the proximity condition. We also achieve better guarantees when only @math -fraction of the points satisfy the weaker proximity condition. The bulk of our analysis relies only on center separation under which one can produce a clustering which (i) has low error, (ii) has low @math -means cost, and (iii) has centers very close to the target centers. Our improved separation condition allows us to match the results of the Planted Partition Model of McSherry[2001], improve upon the results of [2006], and improve separation results for mixture of Gaussian models in a particular setting. The min-sum k-clustering problem in a metric space is to find a partition of the space into k clusters as to minimize the total sum of distances between pairs of points assigned to the same cluster. We give the first polynomial time non-trivial approximation algorithm for this problem. The algorithm provides an @math approximation to the min-sum k -clustering problem in general metric spaces, with running time @math . The result is based on embedding of metric spaces into hierarchically separated trees. We also provide a bicriteria approximation result that provides a constant approximation factor solution with only a constant factor increase in the number of clusters. This result is obtained by modifying and drawing ideas from recently developed primal dual approximation algorithms for facility location. We address the problem of replicating a Voronoi diagram @math of a planar point set @math by making proximity queries, which are of three possible (in decreasing order of information content): 1. the exact location of the nearest site(s) in @math ; 2. the distance to and label(s) of the nearest site(s) in @math ; 3. a unique label for every nearest site in @math . We provide algorithms showing how queries of Type 1 and Type 2 allow an exact cloning of @math with @math queries and @math processing time. We also prove that queries of Type 3 can never exactly clone @math , but we show that with @math queries we can construct an @math -approximate cloning of @math . In addition to showing the limits of nearest-neighbor database security, our methods also provide one of the first natural algorithmic applications of retroactive data structures. We study the problem of estimating multivariate log-concave probability density functions. We prove the first sample complexity upper bound for learning log-concave densities on @math , for all @math . Prior to our work, no upper bound on the sample complexity of this learning problem was known for the case of @math . In more detail, we give an estimator that, for any @math and @math , draws @math samples from an unknown target log-concave density on @math , and outputs a hypothesis that (with high probability) is @math -close to the target, in total variation distance. Our upper bound on the sample complexity comes close to the known lower bound of @math for this problem. We give processor-allocation algorithms for grid architectures, where the objective is to select processors from a set of available processors to minimize the average number of communication hops. The associated clustering problem is as follows: Given n points in ℜ d, find a size-k subset with minimum average pairwise L 1 distance. We present a natural approximation algorithm and show that it is a @math -approximation for two-dimensional grids; in d dimensions, the approximation guarantee is @math , which is tight. We also give a polynomial-time approximation scheme (PTAS) for constant dimension d, and we report on experimental results. We study the tolerance of random networks to intentional attack, whereby a fraction @math of the most connected sites is removed. We focus on scale-free networks, having connectivity distribution @math , and use percolation theory to study analytically and numerically the critical fraction @math needed for the disintegration of the network, as well as the size of the largest connected cluster. We find that even networks with @math , known to be resilient to random removal of sites, are sensitive to intentional attack. We also argue that, near criticality, the average distance between sites in the spanning (largest) cluster scales with its mass, @math , as @math , rather than as @math , as expected for random networks away from criticality. We present new and improved data structures that answer exact node-to-node distance queries in planar graphs. Such data structures are also known as distance oracles. For any directed planar graph on n nodes with non-negative lengths we obtain the following: * Given a desired space allocation @math , we show how to construct in @math time a data structure of size @math that answers distance queries in @math time per query. As a consequence, we obtain an improvement over the fastest algorithm for k-many distances in planar graphs whenever @math . * We provide a linear-space exact distance oracle for planar graphs with query time @math for any constant eps>0. This is the first such data structure with provable sublinear query time. * For edge lengths at least one, we provide an exact distance oracle of space @math such that for any pair of nodes at distance D the query time is @math . Comparable query performance had been observed experimentally but has never been explained theoretically. Our data structures are based on the following new tool: given a non-self-crossing cycle C with @math nodes, we can preprocess G in @math time to produce a data structure of size @math that can answer the following queries in @math time: for a query node u, output the distance from u to all the nodes of C. This data structure builds on and extends a related data structure of Klein (SODA'05), which reports distances to the boundary of a face, rather than a cycle. The best distance oracles for planar graphs until the current work are due to Cabello (SODA'06), Djidjev (WG'96), and Fakcharoenphol and Rao (FOCS'01). For @math and space @math , we essentially improve the query time from @math to @math .