<jupyter_start><jupyter_text>This is a large data set with near 10 million observations.<jupyter_code>import matplotlib.pyplot as plt
n, bins, patches = plt.hist(df.prop_country_id, 100, density = 1, facecolor='blue', alpha=0.75)
plt.xlabel('Property country Id')
plt.title('Histogram of prop_country_id')
plt.show();
df.groupby('prop_country_id').size().nlargest(5)
n, bins, patches = plt.hist(df.visitor_location_country_id, 100, density = 1, facecolor='blue', alpha=0.75)
plt.xlabel('Visitor location country Id')
plt.title('Histogram of visitor_location_country_id')
plt.show();
df.groupby('visitor_location_country_id').size().nlargest(5)<jupyter_output><empty_output><jupyter_text>The data is anonymized, so determining the exact country or city to which a consumer plans to travel to is not possible. However, it is evident that the largest country (labeled 219) is the United States. The largest country has 61% of all observations. Out of those, 58% of searches are made by consumers also located in this country, suggesting that the country has a large territory with a large fraction of domestic travel. The price currency also suggested that the largest country being the United States.Therefore, to improve the computational efficiency, we are going to predict Click-Through for US visitors.<jupyter_code>df.columns
us = df.loc[df['visitor_location_country_id'] == 219]
us = us.sample(frac=0.6, random_state=99)
del us['visitor_location_country_id']
us.shape
us.isnull().sum()<jupyter_output><empty_output><jupyter_text>As you can see, we have a lot of missing data in many features. We are going to drop features that have more than 90% of NaN, plus date_time, srch_id, prop_id, and impute the three features, they are prop_review_score, prop_location_score2, orig_destination_distance.<jupyter_code>cols_to_drop = ['date_time', 'visitor_hist_starrating', 'visitor_hist_adr_usd', 'srch_query_affinity_score', 'comp1_rate', 'comp1_inv', 'comp1_rate_percent_diff', 'comp2_rate_percent_diff', 'comp3_rate_percent_diff', 'comp4_rate_percent_diff', 'comp5_rate_percent_diff', 'comp6_rate_percent_diff', 'comp7_rate_percent_diff', 'comp8_rate_percent_diff', 'comp2_rate', 'comp3_rate', 'comp4_rate', 'comp5_rate', 'comp6_rate', 'comp7_rate', 'comp8_rate', 'comp2_inv', 'comp3_inv', 'comp4_inv', 'comp5_inv', 'comp6_inv', 'comp7_inv', 'comp8_inv', 'gross_bookings_usd', 'srch_id', 'prop_id']
us.drop(cols_to_drop, axis=1, inplace=True)
us.isnull().sum()<jupyter_output><empty_output><jupyter_text>prop_review_score, Random imputation<jupyter_code>import random
def randomiseMissingData(df2):
    "randomise missing data for DataFrame (within a column)"
    df = df2.copy()
    for col in df.columns:
        data = df['prop_review_score']
        mask = data.isnull()
        samples = random.choices( data[~mask].values , k = mask.sum() )
        data[mask] = samples
    return df
us = randomiseMissingData(us)
us['prop_review_score'].mean()
def randomiseMissingData(df2):
    "randomise missing data for DataFrame (within a column)"
    df = df2.copy()
    for col in df.columns:
        data = df['prop_location_score2']
        mask = data.isnull()
        samples = random.choices( data[~mask].values , k = mask.sum() )
        data[mask] = samples
    return df
us = randomiseMissingData(us)<jupyter_output>C:\Users\SusanLi\AppData\Local\Continuum\anaconda3\lib\site-packages\ipykernel_launcher.py:8: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy<jupyter_text>prop_location_score2, mean imputation<jupyter_code>us['prop_location_score2'].mean()
us['prop_location_score2'].fillna((us['prop_location_score2'].mean()), inplace=True)<jupyter_output><empty_output><jupyter_text>orig_destination_distance, Impute with median<jupyter_code>us['orig_destination_distance'].fillna((us['orig_destination_distance'].median()), inplace=True)
us.isnull().sum()<jupyter_output><empty_output><jupyter_text>EDA<jupyter_code>us.shape<jupyter_output><empty_output><jupyter_text>After basic data cleaning, our USA data set contains over 3.4 million observations and 22 features.<jupyter_code>import matplotlib.pyplot as plt
import seaborn as sns
sns.countplot(x='booking_bool',data=us, palette='hls')
plt.show();
us['booking_bool'].value_counts()
sns.countplot(x='click_bool',data=us, palette='hls')
plt.show();
us['click_bool'].value_counts()<jupyter_output><empty_output><jupyter_text>Due to the nature of online travel business, both booking rate(2.8%) and click through rate (4.3%) are extremely low. So, the class are very imbalanced.  Let's explore other features that presented in the data.search length of stay<jupyter_code>n, bins, patches = plt.hist(us.srch_length_of_stay, 50, density = 1, facecolor='blue', alpha=0.75)
plt.xlabel('Search length of stay')
plt.title('Histogram of search_length_of_stay')
plt.axis([0, 30, 0, 0.65])
plt.show();
us.groupby('srch_length_of_stay').size().nlargest(5)<jupyter_output><empty_output><jupyter_text>Nothing outlier. search aults counts<jupyter_code>n, bins, patches = plt.hist(us.srch_adults_count, 20, density = 1, facecolor='blue', alpha=0.75)
plt.xlabel('Search adults count')
plt.title('Histogram of search_adults_count')
plt.show();
df.groupby('srch_adults_count').size().nlargest(5)<jupyter_output><empty_output><jupyter_text>The most common search adults count is 2-adults, makes sense. property star rating<jupyter_code>n, bins, patches = plt.hist(us.prop_starrating, 20, density = 1, facecolor='blue', alpha=0.75)
plt.xlabel('Property star rating')
plt.title('Histogram of prop_star_rating')
plt.show();<jupyter_output><empty_output><jupyter_text>The most common property star rating is 3 stars. Good to know, I would have thought higher.<jupyter_code>us.groupby('prop_brand_bool').size()<jupyter_output><empty_output><jupyter_text>More than 73% of the properties are brand properties. It does make sense since we are talking about US hotels and US travelers.<jupyter_code>us.groupby('srch_saturday_night_bool').size()<jupyter_output><empty_output><jupyter_text>Over 54% of searches contain staying at Saturday. Price USD<jupyter_code>sns.set(style="ticks", palette="pastel")

ax = sns.boxplot(x="click_bool", y="price_usd", hue="click_bool", data=us)
ax.set_ylim([0, 200]);
us.groupby('click_bool')['price_usd'].describe()
ax = sns.boxplot(x="booking_bool", y="price_usd", hue="booking_bool", data=us)
ax.set_ylim([0, 200]);
us.groupby('booking_bool')['price_usd'].describe()<jupyter_output><empty_output><jupyter_text>On average, the price_usd that received a click or booking is always lower than those of did not get a click or booking.<jupyter_code>correlation = us.corr()
plt.figure(figsize=(18, 18))
sns.heatmap(correlation, vmax=1, square=True,annot=True,cmap='viridis')
plt.title('Correlation between different fearures');<jupyter_output><empty_output><jupyter_text>Balance  the dataFor fast learning, our balancing strategy is down sampling negative instances.<jupyter_code>import numpy as np

click_indices = us[us.click_bool == 1].index
random_indices = np.random.choice(click_indices, len(us.loc[us.click_bool == 1]), replace=False)
click_sample = us.loc[random_indices]

not_click = us[us.click_bool == 0].index
random_indices = np.random.choice(not_click, sum(us['click_bool']), replace=False)
not_click_sample = us.loc[random_indices]

us_new = pd.concat([not_click_sample, click_sample], axis=0)

print("Percentage of not click impressions: ", len(us_new[us_new.click_bool == 0])/len(us_new))
print("Percentage of click impression: ", len(us_new[us_new.click_bool == 1])/len(us_new))
print("Total number of records in resampled data: ", len(us_new))
from sklearn.ensemble import RandomForestClassifier
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.ensemble import BaggingClassifier
from sklearn.ensemble import AdaBoostClassifier
from sklearn.ensemble import ExtraTreesClassifier
from sklearn.neural_network import MLPClassifier
from sklearn.naive_bayes import GaussianNB
from sklearn.naive_bayes import BernoulliNB
from sklearn.naive_bayes import MultinomialNB
from sklearn import metrics
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler
mms = MinMaxScaler()
us_new[['price_usd','orig_destination_distance']] = mms.fit_transform(us_new[['price_usd','orig_destination_distance']])
# MinMax=MinMaxScaler()
# df['price_usd']=MinMax.fit_transform(df['price_usd'].values.reshape(-1,1))
Y1=us_new['click_bool'].as_matrix()
Y2=us_new['booking_bool'].as_matrix()
us_new=us_new.drop(['click_bool','booking_bool'], 1)
X=us_new.as_matrix()
np.isnan(np.sum(X))
X_train,X_test,y_train,y_test = train_test_split(X,Y1,test_size=0.3,random_state=1)
rf =RandomForestClassifier(n_estimators=51,min_samples_leaf=5,min_samples_split=3)
    bagg = BaggingClassifier(n_estimators=71,random_state=42)
    extra = ExtraTreesClassifier(n_estimators=57,random_state=42)
    ada = AdaBoostClassifier(n_estimators=51,random_state=42)
    grad = GradientBoostingClassifier(n_estimators=101,random_state=42)
    classifier_list = [rf,bagg,extra,ada,grad]
    classifier_name_list = ['Random Forests','Bagging','Extra Trees','AdaBoost','Gradient Boost']
    return classifier_list,classifier_name_list<jupyter_output><empty_output><jupyter_text>Click-through prediction with ensemble models<jupyter_code>def get_ensemble_models():
    rf =RandomForestClassifier(n_estimators=51,min_samples_leaf=5,min_samples_split=3)
    bagg = BaggingClassifier(n_estimators=51,random_state=42)
    extra = ExtraTreesClassifier(n_estimators=51,random_state=42)
    ada = AdaBoostClassifier(n_estimators=51,random_state=42)
    grad = GradientBoostingClassifier(n_estimators=51,random_state=42)
    classifier_list = [rf,bagg,extra,ada,grad]
    classifier_name_list = ['Random Forests','Bagging','Extra Trees','AdaBoost','Gradient Boost']
    return classifier_list,classifier_name_list
def print_evaluation_metrics(trained_model,trained_model_name,X_test,y_test):
    print('--------- Model : ', trained_model_name, ' ---------------\n')
    predicted_values = trained_model.predict(X_test)
    print(metrics.classification_report(y_test,predicted_values))
    print("Accuracy Score : ",metrics.accuracy_score(y_test,predicted_values))
    print("---------------------------------------\n")
classifier_list, classifier_name_list = get_ensemble_models()
for classifier,classifier_name in zip(classifier_list,classifier_name_list):
    classifier.fit(X_train,y_train)
    print_evaluation_metrics(classifier,classifier_name,X_test,y_test)<jupyter_output>--------- Model :  Random Forests  ---------------

             precision    recall  f1-score   support

          0       0.70      0.70      0.70     45275
          1       0.70      0.70      0.70     44893

avg / total       0.70      0.70      0.70     90168

Accuracy Score :  0.702821400053234
---------------------------------------

--------- Model :  Bagging  ---------------

             precision    recall  f1-score   support

          0       0.69      0.69      0.69     45275
          1       0.69      0.69      0.69     44893

avg / total       0.69      0.69      0.69     90168

Accuracy Score :  0.690732854227664
---------------------------------------

--------- Model :  Extra Trees  ---------------

             precision    recall  f1-score   support

          0       0.69      0.69      0.69     45275
          1       0.69      0.69      0.69     44893

avg / total       0.69      0.69      0.69     90168

Accuracy Score :  0.689368733918907
-------------------[...]<jupyter_text>Click-through prediction with Naive bayes models<jupyter_code>def get_naive_bayes_models():
    gnb = GaussianNB()
    mnb = MultinomialNB()
    bnb = BernoulliNB()
    classifier_list = [gnb,mnb,bnb]
    classifier_name_list = ['Gaussian NB','Multinomial NB','Bernoulli NB']
    return classifier_list,classifier_name_list
classifier_list, classifier_name_list = get_naive_bayes_models()
for classifier,classifier_name in zip(classifier_list,classifier_name_list):
    classifier.fit(X_train,y_train)
    print_evaluation_metrics(classifier,classifier_name,X_test,y_test)<jupyter_output>--------- Model :  Gaussian NB  ---------------

             precision    recall  f1-score   support

          0       0.70      0.61      0.65     45275
          1       0.65      0.74      0.69     44893

avg / total       0.68      0.67      0.67     90168

Accuracy Score :  0.6745297666577943
---------------------------------------

--------- Model :  Multinomial NB  ---------------

             precision    recall  f1-score   support

          0       0.66      0.63      0.65     45275
          1       0.64      0.68      0.66     44893

avg / total       0.65      0.65      0.65     90168

Accuracy Score :  0.6530698252151539
---------------------------------------

--------- Model :  Bernoulli NB  ---------------

             precision    recall  f1-score   support

          0       0.53      0.77      0.63     45275
          1       0.57      0.31      0.41     44893

avg / total       0.55      0.54      0.52     90168

Accuracy Score :  0.5425871706148523
-----------[...]<jupyter_text>Click-through prediction with Neural network<jupyter_code>def get_neural_network(hidden_layer_size=50):
    mlp = MLPClassifier(hidden_layer_sizes=hidden_layer_size)
    return [mlp], ['MultiLayer Perceptron']
classifier_list, classifier_name_list = get_neural_network()
for classifier,classifier_name in zip(classifier_list,classifier_name_list):
    classifier.fit(X_train,y_train)
    print_evaluation_metrics(classifier,classifier_name,X_test,y_test)
rf = RandomForestClassifier(n_estimators=51,min_samples_leaf=5,min_samples_split=3)
rf.fit(X_train, y_train)
(pd.Series(rf.feature_importances_, index=us_new.columns).plot(kind='barh'))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13462224441749304, 'fraction_numerical': 0.03966611971177855, 'mean_word_length': 4.3219438116932425, 'pattern_counts': {'":': 0, '<': 74, '<?xml version=': 0, '>': 74, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 15, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '19', 'dir': 'jupyter-structured-clean-dedup', 'id': '622298', 'license': 'MIT', 'n_tokens_mistral': 5047, 'n_tokens_neox': 4668, 'n_words': 962, 'path': 'Hotel Optimize Click-through.ipynb', 'repo_name': 'BeeChainFoundation/Brainstorming', 'size': '14017'}