<jupyter_start><jupyter_text>SEAI 2021 - Python - Lab 1 Intro to Python <NAME> - <NAME>  DBSCANDBSCAN stands for *Density-based spatial clustering of applications with noise* is a data clustering algorithm developed in 1996 by <NAME>, <NAME>, <NAME>, and <NAME>. It is a non-parametric density-based clustering algorithm: given a set of points in some space, it groups together points that are closely packed together (points with many nearby neighbors), identifying outliers as points that lie alone in low-density regions (whose nearest neighbors are too far away).The major benefits with respect to k-mean and clustering and hierarchical clustering:- The user is  not required to set an initial number of clusters (elbow method to find the right number of clusters miniming SSD)- It is very robust to outliers, SVMs largerly suffer from those- It can draw also non-spherical clusters.In order to fit a DBCAN the user is required to set up 2 parameters:1. **Eps**: the maximum distance between two observation belonging to the same cluster2. **minPts**: the minimum number of observation to be called a cluster.In this notebook we are going to cluster with DBSCAN demonstrating better performances against the other two algorithms.At first let's import the required libraries<jupyter_code>import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.datasets import make_moons

plt.rcParams["figure.figsize"] = (14, 10)
sns.set()<jupyter_output><empty_output><jupyter_text>Then remeber the function make_blobs? make_moons randomly generates moons od data instead of blobs<jupyter_code>X, y = make_moons(n_samples=200, noise=0.05, random_state=0)
plt.scatter(X[:,0],X[:,1])
plt.show()<jupyter_output><empty_output><jupyter_text>With this kind of data it is impossibile to reach a decent classification with already presented two algorithms. Let's see this behaviour.  K-means<jupyter_code>from sklearn.cluster import KMeans

km = KMeans(n_clusters=2)
km.fit(X)
y_km = km.predict(X)

plt.scatter(X[:, 0], X[:, 1], c=y_km,cmap='viridis')
plt.show()<jupyter_output><empty_output><jupyter_text>Hierarchical Clustering (aggolomerative)<jupyter_code>from sklearn.cluster import AgglomerativeClustering

ac = AgglomerativeClustering(n_clusters=2, linkage="complete")
y_ac = ac.fit_predict(X)

plt.scatter(X[:, 0], X[:, 1], c=y_ac,cmap='viridis')
plt.show()<jupyter_output><empty_output><jupyter_text>DBSCANNow let's take a DBSCAN as it is already been pointed out **we are not required to set the number of clusters**, instead **EPS** and **minPts** are the two parameters of interest.<jupyter_code>from sklearn.cluster import DBSCAN

dbscan = DBSCAN(eps=0.25, min_samples=5)
y_dbscan = dbscan.fit_predict(X)

plt.scatter(X[:, 0], X[:, 1], c=y_dbscan,cmap='viridis')
#core_pts = dbscan.components_
#plt.scatter(x=core_pts[:, 0], y=core_pts[:, 1], c='red', s=200, alpha=0.5);
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09982847341337907, 'fraction_numerical': 0.01509433962264151, 'mean_word_length': 6.271820448877805, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 27, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '143008', 'license': 'Apache-2.0', 'n_tokens_mistral': 972, 'n_tokens_neox': 915, 'n_words': 331, 'path': 'lab1/extra/seai_lab1_extra5_DBSCAN.ipynb', 'repo_name': 'vincnardelli/seai2021', 'size': '2962'}