<jupyter_start><jupyter_text>Python Clustering with Scikit-learn **(C) 2017-2020 by [<NAME>](http://damir.cavar.me/)** **Download:** This and various other Jupyter notebooks are available from my [GitHub repo](https://github.com/dcavar/python-tutorial-notebooks). **License:** [Creative Commons Attribution-ShareAlike 4.0 International License](https://creativecommons.org/licenses/by-sa/4.0/) ([CA BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/)) This is a tutorial related to the discussion of clustering in the textbook [Machine Learning: The Art and Science of Algorithms that Make Sense of Data](https://www.cs.bris.ac.uk/~flach/mlbook/) by [<NAME>](https://www.cs.bris.ac.uk/~flach/), and various other textbooks. This tutorial was developed as part of my course material for the course Machine Learning for Natural Language Processing, Computational Linguistics at [Indiana University at Bloomington](https://www.indiana.edu/).  K-means Clustering We will use the *array* objects from the Python module *numpy*:<jupyter_code>import numpy

X = numpy.array([[1, 2], [4, 4], [1, 4], [1, 0], [4, 2], [4, 0]])<jupyter_output><empty_output><jupyter_text>To use the *K-means* clustering algorithm from [Scikit-learn](http://scikit-learn.org/), we import it and specify the number of clusters (that is the *k*), and the random state to initialize the centroid centers of the clusters. We assume that the data can be grouped into two clusters:<jupyter_code>from sklearn.cluster import KMeans

kmeans = KMeans(n_clusters=2, random_state=0)<jupyter_output><empty_output><jupyter_text>We can now apply the clustering algorithm to the datapoints in $X$:<jupyter_code>kmeans.fit(X)

print(kmeans.labels_)<jupyter_output>[1 0 1 1 0 0]<jupyter_text>The output above shows the assignment of datapoints to clusters. We can use the model now to make predictions about other datapoints:<jupyter_code>print(kmeans.predict([[0, 0], [1, 2]]))<jupyter_output>[1 1]<jupyter_text>We can also output the centroids of the two clusters:<jupyter_code>print(kmeans.cluster_centers_)<jupyter_output>[[ 1.  2.]
 [ 4.  2.]]<jupyter_text>If we want to use K-Nearest Neighbor in Scikit Learn, we need to import the KNeighborsClassifier from the neighbors submodule:<jupyter_code>from sklearn.neighbors import KNeighborsClassifier<jupyter_output><empty_output><jupyter_text>We instantiate a KNN-classifier:<jupyter_code>KNNClassifier = KNeighborsClassifier(n_neighbors = 3)<jupyter_output><empty_output><jupyter_text>We use the following dataset X and class-vector y:<jupyter_code>X = [[0, 1], [1, 1], [2, 4], [3, 4]]
y = [0, 0, 1, 1]<jupyter_output><empty_output><jupyter_text>We train the classifier:<jupyter_code>KNNClassifier.fit(X, y)<jupyter_output><empty_output><jupyter_text>We ask the classifier to suggest a class for an unseen vector:<jupyter_code>print(KNNClassifier.predict([[1.1, 0.9]]))<jupyter_output>[0]<jupyter_text>It can also give us the likelihoods for the probability of a data-point being in any of the classes:<jupyter_code>print(KNNClassifier.predict_proba([[2.9, 3.1]]))<jupyter_output>[[ 0.33333333  0.66666667]]<jupyter_text>We might not have a class assignment for a sample set. If we want to use a sample set to find the closes data-point, we could use the KNeighborsClassifier as well. Here is a sample.<jupyter_code>samples = [[0., 0., 0.], [0., .5, 0.], [1., 1., .5]]<jupyter_output><empty_output><jupyter_text>We can now train for K=1, the nearest neighbor model:<jupyter_code>from sklearn.neighbors import NearestNeighbors

KNNClassifier = NearestNeighbors(n_neighbors=1)
KNNClassifier.fit(samples)<jupyter_output><empty_output><jupyter_text>We could ask for the nearest neighbor of a concrete data-point:<jupyter_code>print(KNNClassifier.kneighbors([[1., 1., 1.]]))<jupyter_output>(array([[ 0.5]]), array([[2]]))<jupyter_text>The returned result [[0.5]] and [[2]] means that nearest neighbor is the third sample in samples and that the distance between the two is 0.5. One can also query for the distance of multiple data-points. In this case the output of the distance is supressed:<jupyter_code>X = [[0., 1., 0.], [1., 0., 1.]]

KNNClassifier.kneighbors(X, return_distance=False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12461574840387799, 'fraction_numerical': 0.026956727358713645, 'mean_word_length': 6.33102253032929, 'pattern_counts': {'":': 0, '<': 57, '<?xml version=': 0, '>': 57, 'https://': 6, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 2, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '141705', 'license': 'MIT', 'n_tokens_mistral': 1554, 'n_tokens_neox': 1441, 'n_words': 429, 'path': 'Data Science and Machine Learning/Thorough Python Data Science Topics/Python Clustering with Scikit-learn.ipynb', 'repo_name': 'okara83/Becoming-a-Data-Scientist', 'size': '4239'}