<jupyter_start><jupyter_text>Customer Segmentation in Python This notebook explains how to perform Association Analysis from customer purchase history data. We are using [pandas](https://pandas.pydata.org) (for data manipulation) and [mlxtend](https://github.com/rasbt/mlxtend) (for apriori and association rules algorithnms). The data we're using comes from <NAME>'s book [Data Smart](http://www.john-foreman.com/data-smart-book.html).  Understanding the dataThe dataset contains both information on marketing newsletters/e-mail campaigns (e-mail offers sent)...<jupyter_code>import pandas as pd

df_offers = pd.read_excel("data/WineKMC.xlsx", sheet_name=0)
df_offers.columns = ["offer_id", "campaign", "varietal", "min_qty", "discount", "origin", "past_peak"]
df_offers.head()<jupyter_output><empty_output><jupyter_text>and transaction level data from customers (which offer customers responded to and what they bought).<jupyter_code>df_transactions = pd.read_excel("data/WineKMC.xlsx", sheet_name=1)
df_transactions.columns = ["customer_name", "offer_id"]
df_transactions['n'] = 1
df_transactions.head()<jupyter_output><empty_output><jupyter_text>Clustering the dataFirst we are going to combine both data sets<jupyter_code># join the offers and transactions table
df = pd.merge(df_offers, df_transactions)
# create a "pivot table" which will give us the number of times each customer responded to a given offer
matrix = df.pivot_table(index=['customer_name'], columns=['offer_id'], values='n')
# a little tidying up. fill NA values with 0 and make the index into a column
matrix = matrix.fillna(0).reset_index()
# save a list of the 0/1 columns. we'll use these a bit later
x_cols = matrix.columns[1:]<jupyter_output><empty_output><jupyter_text>We need to propose a proper number of clusters. Let's use the silhouette score<jupyter_code>from sklearn.metrics import silhouette_score

range_n_clusters = [2, 3, 4, 5, 6, 7, 8, 9, 10]            # clusters range you want to select
dataToFit = matrix[matrix.columns[2:]] 
best_clusters = 0                       # best cluster number which you will get
previous_silh_avg = 0.0

for n_clusters in range_n_clusters:
    clusterer = KMeans(n_clusters=n_clusters)
    cluster_labels = clusterer.fit_predict(dataToFit)
    silhouette_avg = silhouette_score(dataToFit, cluster_labels)
    print("For n_clusters =", n_clusters,
          "The average silhouette_score is :", silhouette_avg)
    if silhouette_avg > previous_silh_avg:
        previous_silh_avg = silhouette_avg
        best_clusters = n_clusters

# Final Kmeans for best_clusters
kmeans = KMeans(n_clusters=best_clusters, random_state=0).fit(dataToFit)<jupyter_output>For n_clusters = 2 The average silhouette_score is : 0.09467039888175721
For n_clusters = 3 The average silhouette_score is : 0.12478535792491957
For n_clusters = 4 The average silhouette_score is : 0.13592449651185978
For n_clusters = 5 The average silhouette_score is : 0.13655997960045999
For n_clusters = 6 The average silhouette_score is : 0.11271906335295924
For n_clusters = 7 The average silhouette_score is : 0.13264633025262193
For n_clusters = 8 The average silhouette_score is : 0.11229778764053028
For n_clusters = 9 The average silhouette_score is : 0.13447090205346915
For n_clusters = 10 The average silhouette_score is : 0.11276541602269813<jupyter_text>According to this analysis 5 is a good candidate. Let's determine the number of customers per cluster.<jupyter_code>cluster = KMeans(n_clusters=5)
# slice matrix so we only include the 0/1 indicator columns in the clustering
matrix['cluster'] = cluster.fit_predict(matrix[x_cols])
matrix.cluster.value_counts()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08649674620390456, 'fraction_numerical': 0.0539587852494577, 'mean_word_length': 5.695099818511797, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 22, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '19680', 'license': 'MIT', 'n_tokens_mistral': 1353, 'n_tokens_neox': 1187, 'n_words': 350, 'path': 'Session 08/05-python/customer-segmentation-example.ipynb', 'repo_name': 'dsolito/ca', 'size': '3694'}