<commit_before># Fast density clustering
Python code for clustering two dimensional data using kernel density maps to construct a density graph. Examples for gaussian mixtures and some benchmarks are provided. Our algorithm solves multiscale problems (multiple variances/densities and population sizes) and works for non-convex clusters. It uses cross-validation and is regularized by two main global parameters : a neighborhood
size and a noise threshold measure. The later detects spurious cluster centers while the former guarantees that only local information is used to infer cluster centers (we avoid using long distance information). 

The underlying code is based on fast KD-trees for nearest-neighbor searches O(n log n). While the algorithm is well suited for small datasets with meaningful densities, it works quite well on large datasets (c.g. for N=10000, run time is a few seconds).

# Running

Clone or download this repository and run the following command inside the top directory:

```
pip3 install .
```
That's it ! 
# Examples and comparison with other methods
Check out the example for gaussian mixtures (example.py). You should be able to run it directly. It
should produce a plot similar to this: ![alt tag](https://github.com/alexandreday/fast_density_clustering/blob/master/example/result.png)

In another example (example2.py), the algorithm is benchmarked against some sklearn datasets (note that the same parameters are used across all datasets). This is to be compared with other clustering methods easily accesible from [sklearn](http://scikit-learn.org/stable/modules/clustering.html). ![alt tag](https://github.com/alexandreday/fast_density_clustering/blob/master/example/sklearn_datasets.png)
# Citation

If you use this code in a scientific publication, I would appreciate citation/reference to this repository<commit_msg>Detailed readme for installation<commit_after># Fast density clustering
Python code for clustering two dimensional data using kernel density maps to construct a density graph. Examples for gaussian mixtures and some benchmarks are provided. Our algorithm solves multiscale problems (multiple variances/densities and population sizes) and works for non-convex clusters. It uses cross-validation and is regularized by two main global parameters : a neighborhood
size and a noise threshold measure. The later detects spurious cluster centers while the former guarantees that only local information is used to infer cluster centers (we avoid using long distance information). 

The underlying code is based on fast KD-trees for nearest-neighbor searches O(n log n). While the algorithm is well suited for small datasets with meaningful densities, it works quite well on large datasets (c.g. for N=10000, run time is a few seconds).

# Running

Clone or download this repository. Open the file and use the following command:
```
pip3 install .
```
That's it ! You can now import the package ```clustering``` from you python scripts. Check out the examples
in the file ```example``` and see if you can run the scripts provided.
# Examples and comparison with other methods
Check out the example for gaussian mixtures (example.py). You should be able to run it directly. It
should produce a plot similar to this: ![alt tag](https://github.com/alexandreday/fast_density_clustering/blob/master/example/result.png)

In another example (example2.py), the algorithm is benchmarked against some sklearn datasets (note that the same parameters are used across all datasets). This is to be compared with other clustering methods easily accesible from [sklearn](http://scikit-learn.org/stable/modules/clustering.html). ![alt tag](https://github.com/alexandreday/fast_density_clustering/blob/master/example/sklearn_datasets.png)
# Citation

If you use this code in a scientific publication, I would appreciate citation/reference to this repository
{'fraction_non_alphanumeric': 0.057385486361296964, 'fraction_numerical': 0.003602676273803397, 'mean_word_length': 5.130914826498422, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2509207', 'n_tokens_mistral': 965, 'n_tokens_neox': 885, 'n_words': 501}