<jupyter_start><jupyter_text>Ranking car preferences ***This is an initial attempt to explore concepts of preference learning. We attempt to create a prediction model that takes user attributes and a pairwise ranking between two cars as inputs, and outputs whether the user prefers the first or the second car.<jupyter_code>#importing libraries
import pandas as pd #df
import numpy as np #arrays and matrices
from sklearn import model_selection
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import OneHotEncoder

#viz
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import seaborn as sns
sns.set(style="white",
        palette="pastel",
        context="notebook")


#other
import urllib.request #download files
import zipfile #unzip
import os 

np.random.seed(42) #random seed for reproducibility<jupyter_output><empty_output><jupyter_text>Getting the dataWe use the readily available car preference [dataset](http://users.cecs.anu.edu.au/~u4940058/CarPreferences.html), with the following description from the authors:****We set up an experiment in Amazon Mechanical Turk to collect real pair-wise preferences over users. In this experiment users are presented with a choice to prefer a car over another based on their attributes. We have run two experiments with 10 and 20 cars respectively.**The preference questionnaire contains control questions that were randomly selected from the preferences with reversed order. These control questions were included to measure the consistency of the answers provided. In the dataset, we include all the information collected from the experiments. The reader may choose to exclude information based on the control questions which are indicators for random responses from that user.****<jupyter_code># download zip file, unzip and delete zipfile
# skip if already done once, TO DO: automate skip if files exist
url = "http://users.cecs.anu.edu.au/~u4940058/data/exp1-prefs.zip"
filename = "cars_data/data.zip"
path= "cars_data"
urllib.request.urlretrieve(url, filename)
z = zipfile.ZipFile(filename,'r')
z.extractall(path)
z.close()
os.remove(filename)
os.listdir()<jupyter_output><empty_output><jupyter_text>Data description User attributes:* Education:No response (0), High school (1), Bachelors (2), PhD (3)* Age: No response (0), Below 25 (1), Between 25-30 (2), Between 30-35 (3), Above 40 (4)* Gender: No response (0), Male (1), Female (2)* Region: No response (0), South (1), West (2), North East (3), Mid West (4)Car attributes* Body type: Sedan (1), SUV (2)* Transmission: Manual (1), Automatic (2)* Engine capacity: 2.5L, 3.5L, 4.5L, 5.5L, 6.2L* Fuel consumed: Hybrid (1), Non-Hybrid (2) Data explorationAs mentioned above, the dataset contains numbers for each category. For visualization purposes we will revert back to the category names for easier interpretation. We will then apply our own transformation to numerical values. Car attributesThe items dataset contains 10 types of cars with unique attributes that describe them, such as body type or type of transmission.  Some insights from data description and visualization:* There is an equal amount of SUVs and Sedans.* Six out of ten vehicles are Automatic transmission.* Six out of ten vehicles are Non-hybrid.* There are no Hybrid cars with manual transmission.* Hybrids have for the most part smaller engines than non-hybrid cars.    * There is one exception, one hybrid car with engine capacity of 5.5L* Manual transmission cars have a wider range of engine capacity.<jupyter_code>#load in data and explore
items = pd.read_csv("cars_data/items1.csv")

#replacing numerical values with text category names according to description
items[" BodyType"] = items[" BodyType"].replace(1,"Sedan")
items[" BodyType"] = items[" BodyType"].replace(2,"SUV")

items[" Transmission"] = items[" Transmission"].replace(1,"Manual")
items[" Transmission"] = items[" Transmission"].replace(2,"Automatic")

items[" Fuel Consumed"] = items[" Fuel Consumed"].replace(1,"Hybrid")
items[" Fuel Consumed"] = items[" Fuel Consumed"].replace(2,"Non-Hybrid")
items.head()
#plot categorical values
sns.catplot(
    y=" Engin Capacity",
    x=" Transmission",
    hue=" Fuel Consumed",
    kind="box",
    data=items,
    #kind="box",
    height=6,
    aspect=1
           )
items.describe(include='object')
items.describe(exclude='object')<jupyter_output><empty_output><jupyter_text>User attributesThe users dataset contains 58 unique responders (after droping two respondents with missing values) with information on their education level, age, gender and region:* The education level amongst respondents is hight, with 38 of them reporting having a PhD.* 30 out of the 58 respondents are female.* The most represented region is Southern U.S. with 18 respondents.* There are no female respondents with high-school education, but instead have a  bachelor's or a PhD degree.<jupyter_code>users = pd.read_csv('cars_data/users1.csv')
# we will replace 0 values (no response) to NaN and drop them 
users = users.replace(0,np.nan)
users = users.dropna()

#replacing numerical values with text category names
users[" Education"] = users[" Education"].replace(1,"High School")
users[" Education"] = users[" Education"].replace(2,"Bachelors")
users[" Education"] = users[" Education"].replace(3,"PhD")

users[" Gender"] = users[" Gender"].replace(1,"Male")
users[" Gender"] = users[" Gender"].replace(2,"Female")

users[" Region"] = users[" Region"].replace(1,"South")
users[" Region"] = users[" Region"].replace(2,"West")
users[" Region"] = users[" Region"].replace(3,"North East")
users[" Region"] = users[" Region"].replace(4,"Mid West")

users.head()
sns.catplot(
    x=" Education",
    y=" Age",
    order=["High School","Bachelors","PhD"],
    hue=" Gender",
    data=users,
    kind="box",
    height=5,
    aspect=2
           )
users.describe(include='object')
users.describe(exclude='object')<jupyter_output><empty_output><jupyter_text>PreferencesThe preferences dataset contains the user ID and the pair-wise preference of one car over another. As explained by the dataset description and seen in the data, there are users with incomplete rankings. We can give one example based on the dataset. * User ID 1 is a woman from Southern U.S., holds a PhD degree and is in the age group of above 40 years of age.* She prefers item 9 over item 3, but prefers item 7 over item 9. Or 7>9>3.* From this sample, she prefers a hybrid, small (2.5l) and automatic transmission SUV, item ID 7.   Pairwise preferencesThe dataset contains only one side of the preference, i.e. (xi-xj). We can exploit this characteristic to easily add a column with y_1 = 1. Then we can construct the inverse (xj-xi) and add a column with y_2 = -1.<jupyter_code>prefs = pd.read_csv("cars_data/prefs1.csv")
prefs.head()<jupyter_output><empty_output><jupyter_text>Pairwise preferencesThe dataset contains only one side of the preference, i.e. (xi-xj). We can exploit this characteristic to easily add a column with y_1 = 1. Then we can construct the inverse (xj-xi) and add a column with y_2 = -1.<jupyter_code>#setting initial value for xi-xj=1
prefs["y"]= 1

#creating a shifted df 
#setting value for xj-xi=-1
prefs_tmp = pd.read_csv("cars_data/prefs1.csv")
prefs_tmp = prefs[["User ID", " Item2 ID", " Item1 ID", " Is Control"]]
prefs_tmp["y"]=-1
prefs_tmp = prefs_tmp.rename(index=str, columns={" Item2 ID":" Item1 ID",
                                    " Item1 ID":" Item2 ID"})

#combining the datasets
prefs_pw = prefs.append(prefs_tmp, ignore_index=True)

#sanity checks to confirm correct results
#prefs_pw = prefs_pw.sort_values(by="User ID")
#prefs_pw.where(prefs_pw["User ID"]==1).head(60)
#prefs_pw.loc[prefs_pw['User ID'] == 1] 
print("Size of original dataset:",prefs.shape)
print("Size of constructed dataset:",prefs_pw.shape)<jupyter_output>Size of original dataset: (2973, 5)
Size of constructed dataset: (5946, 5)<jupyter_text>Preparing dataset for modelWe will join the user dataset to the preferences dataset to hopefully capture some information about users in their preferences.<jupyter_code># recall previous dataset
prefs_pw.head()
#we recall our users dataset
users.head()
#merge datasets
users_prefs_pw = pd.merge(users,
                         prefs_pw[["User ID"," Item1 ID", " Item2 ID", "y"]],
                         on="User ID")
users_prefs_pw.head()<jupyter_output><empty_output><jupyter_text>Preprocessing dataWe will get the categorical values of the data and one hot encode them in order to avoid ranking, for example, Gender or Region.<jupyter_code>X = users_prefs_pw.drop(["User ID", " # Correct Controls", "y"], axis=1)
y = users_prefs_pw["y"]
#we still need to get dummies and one hot encode the data
X.head()
X = pd.get_dummies(X)
X.head()
#one hot encode
X = OneHotEncoder(sparse=False,categories="auto").fit_transform(X)<jupyter_output><empty_output><jupyter_text>Support vector machines Model SelectionWe will start with comparing linear and rbf kernels for SVMs, tuning for the regularization term C. (Following experiments will include tuning of sigma for the rbf kernel)  Train test split The first attempt is a simple train test split, 70-30, of the data, which is a good compromise when the dataset is big (which is not the case here).<jupyter_code>X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3,
                                                    random_state=42)
print("Train input shape:",X_train.shape)
print("Train output shape:",y_train.shape)
print("****")
print("Test input shape:",X_test.shape)
print("Test output shape:",y_test.shape)
import sklearn.svm

def rank_svm(X_pairs, y_pairs, kernel, C,):
    rank_model = sklearn.svm.SVC(kernel=kernel, C=C, gamma="auto")
    rank_model.fit(X_pairs, y_pairs)

    return rank_model
list = [2, 1.5, 1, 0.5, 0.01, 0.05, 0.001, 0.005, 0.0001, 0.0005]
print("SVC Linear Kernel")
for i in (list):
    rank_model = rank_svm(X_train, y_train,"linear",i)
    print("C = {} ->".format(i), "Accuracy: {0:.2f}% ".format((rank_model.score(X_test, y_test)*100)))
print("SVC with RBF Kernel")
for i in (list):
    rank_model = rank_svm(X_train, y_train,"rbf",i)
    print("C = {} ->".format(i), "Accuracy: {0:.2f}% ".format((rank_model.score(X_test, y_test)*100)))<jupyter_output>SVC with RBF Kernel
C = 2 -> Accuracy: 62.91% 
C = 1.5 -> Accuracy: 62.62% 
C = 1 -> Accuracy: 62.33% 
C = 0.5 -> Accuracy: 62.21% 
C = 0.01 -> Accuracy: 49.07% 
C = 0.05 -> Accuracy: 49.07% 
C = 0.001 -> Accuracy: 49.07% 
C = 0.005 -> Accuracy: 49.07% 
C = 0.0001 -> Accuracy: 49.07% 
C = 0.0005 -> Accuracy: 49.07%<jupyter_text>K-fold cross validationWith this approach, we can prevent overfitting by minimizing the variance that can be introduced when using a single train test split by folding the dataset k-times and cross-validating our results.<jupyter_code>#define the folds
kfold = model_selection.KFold(n_splits=10, random_state=42)

print("SVC Linear Kernel")
#iterate through regularization values
for i in (list):
    model = sklearn.svm.SVC(kernel="linear", C=i, gamma="auto")
    results = model_selection.cross_val_score(model,
                                              X,
                                              y,
                                              cv=kfold)
    print("C = {} ->".format(i), "Accuracy: %.3f%% (%.3f%%)" % (results.mean()*100.0, results.std()*100.0))

print("SVC RBF Kernel")
#iterate through regularization values
for i in (list):
    model = sklearn.svm.SVC(kernel="rbf", C=i, gamma="auto")
    results = model_selection.cross_val_score(model,
                                              X,
                                              y,
                                              cv=kfold)
    print("C = {} ->".format(i), "Accuracy: %.3f%% (%.3f%%)" % (results.mean()*100.0, results.std()*100.0))<jupyter_output>SVC RBF Kernel
C = 2 -> Accuracy: 62.292% (6.977%)
C = 1.5 -> Accuracy: 62.361% (6.973%)
C = 1 -> Accuracy: 62.031% (6.833%)
C = 0.5 -> Accuracy: 61.979% (6.908%)
C = 0.01 -> Accuracy: 48.507% (0.545%)
C = 0.05 -> Accuracy: 56.944% (4.873%)
C = 0.001 -> Accuracy: 48.507% (0.545%)
C = 0.005 -> Accuracy: 48.507% (0.545%)
C = 0.0001 -> Accuracy: 48.507% (0.545%)
C = 0.0005 -> Accuracy: 48.507% (0.545%)
{'fraction_non_alphanumeric': 0.12103465104929234, 'fraction_numerical': 0.03391898487066862, 'mean_word_length': 4.196534234995774, 'pattern_counts': {'":': 2, '<': 38, '<?xml version=': 0, '>': 64, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 17, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '556402', 'license': 'MIT', 'n_tokens_mistral': 4084, 'n_tokens_neox': 3704, 'n_words': 1360, 'path': 'portfolio/car_rankings.ipynb', 'repo_name': 'krist0fer/krist0fer.github.io', 'size': '12294'}