<jupyter_start><jupyter_text>Nigerian Music scraped from Spotify - an analysis Start where we finished in the last lesson, with data imported and filtered.<jupyter_code>import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns


df = pd.read_csv("../data/nigerian-songs.csv")
df.head()<jupyter_output><empty_output><jupyter_text>We will focus only on 3 genres. Maybe we can get 3 clusters built!<jupyter_code>df = df[(df['artist_top_genre'] == 'afro dancehall') | (df['artist_top_genre'] == 'afropop') | (df['artist_top_genre'] == 'nigerian pop')]
df = df[(df['popularity'] > 0)]
top = df['artist_top_genre'].value_counts()
plt.figure(figsize=(10,7))
sns.barplot(x=top.index,y=top.values)
plt.xticks(rotation=45)
plt.title('Top genres',color = 'blue')
df.head()<jupyter_output><empty_output><jupyter_text>Exercise - preparationStart by taking another look at the songs data.1. Create a boxplot, calling boxplot() for each column:<jupyter_code>plt.figure(figsize=(20,20), dpi=200)

plt.subplot(4,3,1)
sns.boxplot(x = 'popularity', data = df)

plt.subplot(4,3,2)
sns.boxplot(x = 'acousticness', data = df)

plt.subplot(4,3,3)
sns.boxplot(x = 'energy', data = df)

plt.subplot(4,3,4)
sns.boxplot(x = 'instrumentalness', data = df)

plt.subplot(4,3,5)
sns.boxplot(x = 'liveness', data = df)

plt.subplot(4,3,6)
sns.boxplot(x = 'loudness', data = df)

plt.subplot(4,3,7)
sns.boxplot(x = 'speechiness', data = df)

plt.subplot(4,3,8)
sns.boxplot(x = 'tempo', data = df)

plt.subplot(4,3,9)
sns.boxplot(x = 'time_signature', data = df)

plt.subplot(4,3,10)
sns.boxplot(x = 'danceability', data = df)

plt.subplot(4,3,11)
sns.boxplot(x = 'length', data = df)

plt.subplot(4,3,12)
sns.boxplot(x = 'release_date', data = df)<jupyter_output><empty_output><jupyter_text>You could go through the dataset and remove these outliers, but that would make the data pretty minimal.1. For now, choose which columns you will use for your clustering exercise. Pick ones with similar ranges and encode the artist_top_genre column as numeric data:<jupyter_code>from sklearn.preprocessing import LabelEncoder
le = LabelEncoder()

X = df.loc[:, ('artist_top_genre','popularity','danceability','acousticness','loudness','energy')]

y = df['artist_top_genre']

X['artist_top_genre'] = le.fit_transform(X['artist_top_genre'])

y = le.transform(y)<jupyter_output><empty_output><jupyter_text>2. Now you need to pick how many clusters to target. You know there are 3 song genres that we carved out of the dataset, so let's try 3:<jupyter_code>from sklearn.cluster import KMeans

nclusters = 3 
seed = 0

km = KMeans(n_clusters=nclusters, random_state=seed)
km.fit(X)

# Predict the cluster for each data point

y_cluster_kmeans = km.predict(X)
y_cluster_kmeans<jupyter_output><empty_output><jupyter_text>1. Use this array to calculate a 'silhouette score':<jupyter_code>from sklearn import metrics
score = metrics.silhouette_score(X, y_cluster_kmeans)
score<jupyter_output><empty_output><jupyter_text>Exercise - build a model1. Import KMeans and start the clustering process<jupyter_code>from sklearn.cluster import KMeans
wcss = []

for i in range(1, 11):
    kmeans = KMeans(n_clusters = i, init = 'k-means++', random_state = 42)
    kmeans.fit(X)
    wcss.append(kmeans.inertia_)<jupyter_output>C:\Users\Asus\anaconda3\lib\site-packages\sklearn\cluster\_kmeans.py:1039: UserWarning: KMeans is known to have a memory leak on Windows with MKL, when there are less chunks than available threads. You can avoid it by setting the environment variable OMP_NUM_THREADS=2.
  warnings.warn(<jupyter_text>Elbow methodPreviously, you surmised that, because you have targeted 3 song genres, you should choose 3 clusters. But is that the case?1. Use the 'elbow method' to make sure.<jupyter_code>plt.figure(figsize=(10,5))
sns.lineplot(range(1, 11), wcss,marker='o',color='red')
plt.title('Elbow')
plt.xlabel('Number of clusters')
plt.ylabel('WCSS')
plt.show()<jupyter_output>C:\Users\Asus\anaconda3\lib\site-packages\seaborn\_decorators.py:36: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  warnings.warn(<jupyter_text>Exercise - display the clusters1. Try the process again, this time setting three clusters, and display the clusters as a scatterplot:<jupyter_code>from sklearn.cluster import KMeans
kmeans = KMeans(n_clusters = 3)
kmeans.fit(X)
labels = kmeans.predict(X)
plt.scatter(df['popularity'],df['danceability'],c = labels)
plt.xlabel('popularity')
plt.ylabel('danceability')
plt.show()<jupyter_output><empty_output><jupyter_text>2. Check the model's accuracy:<jupyter_code>labels = kmeans.labels_

correct_labels = sum(y == labels)

print("Result: %d out of %d samples were correctly labeled." % (correct_labels, y.size))

print('Accuracy score: {0:0.2f}'. format(correct_labels/float(y.size)))<jupyter_output>Result: 69 out of 286 samples were correctly labeled.
Accuracy score: 0.24
{'fraction_non_alphanumeric': 0.1283570300157978, 'fraction_numerical': 0.020339652448657188, 'mean_word_length': 6.277298850574713, 'pattern_counts': {'":': 0, '<': 38, '<?xml version=': 0, '>': 39, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '265205', 'license': 'MIT', 'n_tokens_mistral': 1927, 'n_tokens_neox': 1824, 'n_words': 466, 'path': '5-Clustering/2-K-Means/notebook.ipynb', 'repo_name': 'turhancan97/ML-For-Beginners', 'size': '5064'}