<jupyter_start><jupyter_text>Consumer segmentation - choosing optimal cluster numbers for mixed variables In this exercise, you will - use elbow chart to determine how many consumer segments to generate- segment consumers using the optimal number of segments- because the data have mixed data types, k-prototypes algorithm should be used We will use "MallCustomersAllVariables.csv" for analysis. Github link can be found [here](https://github.com/zoutianxin1992/MarketingAnalyticsPython/blob/main/Marketing%20Analytics%20in%20Python/Segmentation/Datasets/MallCustomersTwoVariables.csv).  Load and preprocess data<jupyter_code># importing packages
import numpy as np
import pandas as pd
####################################
from kmodes.kprototypes import KPrototypes   # We will use the k-prototypes algorithm
####################################
from matplotlib import pyplot as plt

# change variable names
url = "https://raw.githubusercontent.com/zoutianxin1992/MarketingAnalyticsPython/main/Marketing%20Analytics%20in%20Python/Segmentation/Datasets/MallCustomersAllVariables.csv"
df = pd.read_csv(url,index_col=0) # use the first column (customer id) as index
df = df.rename(columns = {"Gender":"gender",
                         "Age":"age",
                         "Annual Income (k$)":"annual_income",
                         "Spending Score (1-100)":"spending_score"})

# normalizing the data for k-mean algorithm
df_normalized = df.copy() # create a copy of the orignial dataset



df_normalized[['age','annual_income','spending_score']] = \
    (df[['age','annual_income','spending_score']]-df[['age','annual_income','spending_score']].min()) \
    /(df[['age','annual_income','spending_score']].max()-df[['age','annual_income','spending_score']].min())

# "\" breaks a line into multiple lines. There shouldn't be any spaces after "\"
df_normalized.head()<jupyter_output><empty_output><jupyter_text>Determine the optimal number of segments<jupyter_code># calculate clustering cost (comparable to SSE) for different k's, and store them in a vector

K = 10  # K is the maximum number of clusters we will check
store_cost = np.zeros(K-1)    # create a vector to store the clustering "cost", which is comparable to SSE in k-means. k is between 2 and K. 
####################################
for k in range(2, K+1):                                   # try k from 2 to K  
    kprotoSpec = KPrototypes(n_clusters = k, n_init = 20)                        # set up k-mean model with k clusters
#################################### 
    kproto_result = kprotoSpec.fit(df_normalized, categorical = [0])               # run k-mean on normalized data
    kproto_prediction = kproto_result.labels_                 # the predicted segments for consumers  
################################
    store_cost[k-2] = kproto_result.cost_         # store the clustering cost 
####################################
store_cost
# plot elbow chart

%matplotlib inline
plt.rcParams['figure.figsize'] = [12,8]  # set figure size to be 12*8 inch
plt.plot(range(2, K+1), store_cost)        
plt.xticks(range(2, K+1), fontsize = 18)
plt.yticks(fontsize = 18)
plt.ylabel("Clustering cost",fontsize = 18)
plt.xlabel("number of clusters", fontsize = 18)<jupyter_output><empty_output><jupyter_text>Segment consumers using the optimal number of segments<jupyter_code>kprotoSpec = KPrototypes(n_clusters = 6, n_init = 20)  # setup the k-mean model specs
kproto_result = kprotoSpec.fit(df_normalized,categorical = [0])    # apply the above model to the normalized data and store the result<jupyter_output><empty_output><jupyter_text>Post-segmentation analysis  Which segment does each consumer belong to? Create a new column in the **original** dataframe for which segment a consumer belongs to.<jupyter_code>df["segment"] = kproto_result.labels_ 
df.head()<jupyter_output><empty_output><jupyter_text>For each segment, summarize the mean/min/max's of all the numerical variable, and the proportion of male consumers in each segment.Also count the number of consumers in each segment.<jupyter_code># summarize  variables
   
summary_table = df.groupby("segment").aggregate({
####################################
    "gender": lambda x: x[x == "M"].count()/x.count(),   # lambda allows us to define an inline function
####################################
    "age":["mean","min","max"],
    "annual_income": ["mean","min","max"],             
    "spending_score": ["mean","min","max"],            
    "segment": "count"                                   
}
)
summary_table<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.16262295081967212, 'fraction_numerical': 0.0126775956284153, 'mean_word_length': 4.402597402597403, 'pattern_counts': {'":': 9, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '369543', 'license': 'MIT', 'n_tokens_mistral': 1383, 'n_tokens_neox': 1282, 'n_words': 383, 'path': 'Marketing Analytics in Python/Segmentation/Notebooks/sgmt_mixvar_ElbowChart.ipynb', 'repo_name': 'zoutianxin1992/MarketingAnalyticsPython', 'size': '4575'}