<jupyter_start><jupyter_text>No strange dtype in the dataset<jupyter_code>df.columns
numerical=['Avg. Session Length', 'Time on App','Time on Website', 'Length of Membership', 'Yearly Amount Spent']
categorical=['Email', 'Address', 'Avatar']
df.describe()<jupyter_output><empty_output><jupyter_text>Numerical features have no strange data<jupyter_code>for col in categorical:
    print(col,df[col].nunique(),sep='  -->  ')
df['Avatar'].unique()<jupyter_output><empty_output><jupyter_text>Categorical features have no strange data but we won't use them<jupyter_code>df.drop(categorical,axis=1,inplace=True)
df.head()
df.corr()['Yearly Amount Spent']<jupyter_output><empty_output><jupyter_text>'Time on Website' has no correlation with 'Yearly Amount Spent'<jupyter_code>df.drop('Time on Website',axis=1,inplace=True)
numerical.remove('Time on Website')
df.head()
fig,axes=plt.subplots(2,2,figsize=(12,12))
sns.boxplot(data=df,y=numerical[0],ax=axes[0,0])
sns.boxplot(data=df,y=numerical[1],ax=axes[0,1])
sns.boxplot(data=df,y=numerical[2],ax=axes[1,0])
sns.boxplot(data=df,y=numerical[3],ax=axes[1,1])
for col in numerical:
    print(col)
    outliers=df.loc[detect_outliers(df,0,[col]),col]
    print(outliers)
    print()<jupyter_output>Avg. Session Length
12     29.532429
154    36.139662
390    35.860237
Name: Avg. Session Length, dtype: float64

Time on App
65     15.126994
309     8.508152
362     9.316289
442     8.668350
Name: Time on App, dtype: float64

Length of Membership
50     5.976768
151    6.401229
153    6.076654
157    6.922689
161    0.968622
173    0.801516
225    0.789520
303    1.084585
379    1.130477
396    6.115199
434    0.936498
447    0.269901
Name: Length of Membership, dtype: float64

Yearly Amount Spent
50     725.584814
65     765.518462
157    744.221867
205    712.396327
225    256.670582
299    282.471246
303    266.086341
368    708.935185
415    275.918421
Name: Yearly Amount Spent, dtype: float64<jupyter_text>outliers are not bad so we won't deal with them<jupyter_code>df.isna().sum()
df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 500 entries, 0 to 499
Data columns (total 4 columns):
 #   Column                Non-Null Count  Dtype  
---  ------                --------------  -----  
 0   Avg. Session Length   500 non-null    float64
 1   Time on App           500 non-null    float64
 2   Length of Membership  500 non-null    float64
 3   Yearly Amount Spent   500 non-null    float64
dtypes: float64(4)
memory usage: 15.8 KB<jupyter_text>Splitting the dataset<jupyter_code>x=df.drop('Yearly Amount Spent',axis=1)
y=df['Yearly Amount Spent']
from sklearn.model_selection import train_test_split
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2)
print(f"{x_train.shape} ==> {y_train.shape}")
print(f"{x_test.shape} ==> {y_test.shape}")<jupyter_output>(400, 3) ==> (400,)
(100, 3) ==> (100,)<jupyter_text>feature scaling<jupyter_code>scaler = StandardScaler()
scaler.fit(x_train)
x_train = scaler.transform(x_train)
x_test = scaler.transform(x_test)<jupyter_output><empty_output><jupyter_text>try different regression models<jupyter_code>from sklearn.linear_model import LinearRegression
from sklearn.neighbors import KNeighborsRegressor
from sklearn.svm import SVR
from sklearn.tree import DecisionTreeRegressor
from sklearn.ensemble import RandomForestRegressor
from xgboost import XGBRegressor

from sklearn.metrics import mean_squared_error, r2_score
models = {
    "LinearRegression": LinearRegression(),
    "KNeighborsRegressor" : KNeighborsRegressor(), 
    "SVR": SVR(),
    "DecisionTreeRegressor": DecisionTreeRegressor(),
    "RandomForestRegressor": RandomForestRegressor(),
    "XGBRegressor": XGBRegressor()
}
for name, model in models.items():
    print(f'Using model: {name}')
    model.fit(x_train, y_train)
    y_pred = model.predict(x_test)
    print(f'RMSE: {np.sqrt(mean_squared_error(y_test, y_pred))}')
    print(f'R2 Score: {r2_score(y_test, y_pred)}')
    print('-----------------------------------------------------------')<jupyter_output>Using model: LinearRegression
RMSE: 9.310502965083604
R2 Score: 0.9880168043003342
-----------------------------------------------------------
Using model: KNeighborsRegressor
RMSE: 21.666674456440788
R2 Score: 0.9351050529304522
-----------------------------------------------------------
Using model: SVR
RMSE: 61.13350976463295
R2 Score: 0.4833635703302941
-----------------------------------------------------------
Using model: DecisionTreeRegressor
RMSE: 25.89137521880253
R2 Score: 0.9073305461376577
-----------------------------------------------------------
Using model: RandomForestRegressor
RMSE: 18.110944761878645
R2 Score: 0.954657174598854
-----------------------------------------------------------
Using model: XGBRegressor
RMSE: 15.94553042748174
R2 Score: 0.9648517013700947
-----------------------------------------------------------<jupyter_text>LinearRegression model gives the best results<jupyter_code>model = LinearRegression()
model.fit(x_train, y_train)
y_predict = model.predict(x_test)
y_test
y_predict
import joblib
joblib.dump(model, 'model.h5')
joblib.dump(scaler, 'scaler.h5')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1894534995206136, 'fraction_numerical': 0.11332694151486097, 'mean_word_length': 3.328630705394191, 'pattern_counts': {'":': 5, '<': 34, '<?xml version=': 0, '>': 39, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '243385', 'license': 'MIT', 'n_tokens_mistral': 2217, 'n_tokens_neox': 1831, 'n_words': 391, 'path': 'Ecommerce_Regression.ipynb', 'repo_name': 'KarimSakr99/Assignment_11', 'size': '5215'}