<jupyter_start><jupyter_text>**Black Friday Sales Prediction** A companhia *ABC Private Limited* gostaria de entender o comportamento de seus clientes (especificamente, a quantidade de compras) em relação a vários produtos de diferentes categorias. Eles compartilharam um resumo de compras de vários clientes.O conjunto de dados também contém informações demográficas (idade, gênero, status matrimonial, cidade, tempo de moradia na cidade), detalhes dos produtos (identificação e categoria) e o total de compras do último mês.Agora eles querem construir um modelo que preveja o volume de vendas dos clientes em relação a vários produtos que ajudarão a criar uma oferta personalizada para os consumidores de diferentes produtos.**Dicionário de Dados**``User_ID``: User ID``Product_ID``: Product ID``Gender``: Sex of User``Age``: Age in bins``Occupation``: Occupation (Masked)``City_Category``: Category of the City (A,B,C)``Stay_In_Current_City_Years``: Number of years stay in current city``Marital_Status``: Marital Status``Product_Category_1``: Product Category (Masked)``Product_Category_2``: Product may belongs to other category also (Masked)``Product_Category_3``:Product may belongs to other category also (Masked)``Purchase``: Purchase Amount (Target Variable)<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
df_test = pd.read_csv('data/test.csv')
df_train = pd.read_csv('data/train.csv')
df_train.head()
df_test.head()
df_train.Purchase.hist();
df_test['Purchase'] = -1
df = pd.concat([df_train,df_test])
df
df.isna().mean().nlargest(df.shape[0]) * 100
df.info()
df2 = df.fillna(0)
df2.describe()
df2.columns
sns.catplot(data=df2, y='Purchase', x= 'City_Category', hue='Gender', kind='swarm' );
df2[['Product_Category_1','Product_Category_2','Product_Category_3']].hist(figsize=(10,8), bins=15);
df2.head()
df2.City_Category.unique()
df2.Age.value_counts(dropna = False)
age_dict = {'0-17': 1, '18-25': 2, '26-35': 2, '36-45': 3, '46-50': 4, '51-55': 5, '55+': 6}
gender_dict = {'F': 0, 'M': 1}
city_cat_dict = {'A':0, 'B': 1, 'C': 2}
stay_city_dict = {'0':0, '1':1, '2':2, '3':3, '4+':4}

df3 = df2

df3['Age'] = df3['Age'].map(age_dict)
df3['City_Category'] = df3['City_Category'].map(city_cat_dict)
df3['Gender'] = df3['Gender'].map(gender_dict)
df3['Stay_In_Current_City_Years'] = df3['Stay_In_Current_City_Years'].map(stay_city_dict)
df3.head()
def normalize(dframe):
    dframe = (dframe - dframe.mean()) / dframe.std()
    return dframe
df4 = df3[df3.Purchase != -1]
test = df3[df3.Purchase == -1]
test_ids = test[['User_ID', 'Product_ID']]
X_test = test.drop(columns=['User_ID', 'Product_ID', 'Purchase'])
X_test.head()
from sklearn.model_selection import train_test_split
df5 = df4.drop(['User_ID', 'Product_ID'], axis=1)
df5.head()
X = df5.drop('Purchase', axis=1)
y = df5.Purchase
X_train, X_validate, y_train, y_validate = train_test_split(X, y, test_size=0.33, random_state=24)
X_train.shape
X_validate.shape
from sklearn.ensemble import RandomForestRegressor
X_train = normalize(X_train)
X_validate = normalize(X_validate)
rf = RandomForestRegressor()
rf.fit(X_train, y_train)
rf_pred = rf.predict(X_validate)
from sklearn.metrics import mean_squared_error
mean_squared_error(y_validate, rf_pred)
from xgboost import XGBRegressor
xgb = XGBRegressor(colsample_bytree=0.4,
                 gamma=0,                 
                 learning_rate=0.07,
                 max_depth=3,
                 min_child_weight=1.5,
                                                                                 
                 reg_alpha=0.75,
                 reg_lambda=0.45,
                 subsample=0.6,
                 seed=42)
xgb.fit(X_train, y_train)
xgb_pred = xgb.predict(X_validate)
mean_squared_error(y_validate, xgb_pred)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1359729096118781, 'fraction_numerical': 0.03151862464183381, 'mean_word_length': 4.647058823529412, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '448929', 'license': 'MIT', 'n_tokens_mistral': 1442, 'n_tokens_neox': 1375, 'n_words': 326, 'path': 'regression/black_friday_sales_prediction/black_friday_sales_prediction.ipynb', 'repo_name': 'dspauloplima/data_science_projects', 'size': '3839'}