<jupyter_start><jupyter_text>IntroductionThe aim of this project is to build a regression model to predict **AmountPaid360**.The input variables in play are described below:   * Product : The product type purchased by the customer   * CustomerGender : Self-explanatory   * Location : The location where the loan was issued. (Also includes a long-tail of legacy locations)   * Region : The sales region where the device was purchased. Includes many regions which have since been changed. The current               set of main sales regions are Regions 1-7.   * TotalPrice : The total amount to-be-paid by the customer over the course of the loan (including deposit and daily payments)   * StartDate : The day on which the customer's loan became active   * Deposit : The initial deposit required from the customer prior to loan activation   * DailyRate : The amount that the customer must pay daily in order to continue to use their device   * TotalDays : The number of days for which the customer is expected to continue paying the daily rate   * AmountPaid30 : The amount that the customer had repaid as of the 30th day of their loan   * AmountPaid60 : The amount that the customer had repaid as of the 60th day of their loanThe outcome variables are as follows:   * AmountPaid360 : The amount that the customer had repaid as of the 360th day of their loan   * LoanStatus360 : The status of the loan as of day 360:   * Active : The customer is still actively repaying their loan.   * Blocked : The customer has had their account blocked after more than 30 days without a payment.   * Finished Payment : The customer has successfully repaid their loan.  Importing the necessary libraries<jupyter_code>#Library imports
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns


# magic function to allow matplotlib plots to show
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Data Importation and Exploration<jupyter_code>#ensuring all columns are displayed
pd.pandas.set_option('display.max_columns', None)
#loading the dataframe as df and setting the variable 'StartDate' as  datetime 

df = pd.read_csv('Data Science assessment.csv', parse_dates = ['StartDate'])
#Look at the first five rows
df.head()
#Summarized information on the dataset
df.info()
#checking for null values

df.isnull().sum()
#Summary of the characteristics of the numerical columns
df.describe()
#Dropping any duplicates present
df.drop_duplicates()
#Checking whether the duplicates were deleted 
df.duplicated().sum()<jupyter_output><empty_output><jupyter_text>Data VisualisationVisualize the data to gain insights<jupyter_code>#lineplot. TO DO. Plot line plot and play with markers and line width

X = np.linspace(1, 20, 5)

y = np.linspace(10,100,5)

z = [100, 300,500,700, 1100]

#plt.plot(X,y)
#plt.plot(X,z)

#plt.show()
plt.plot(X, y, color='green', linewidth=4.0, linestyle='--', label='X and Y')

plt.plot(X, z, label='X and Z')

plt.xlabel('Values of X')
plt.ylabel('Values of Y and Z')
plt.title('X,Y and Z')
plt.legend()
#scatterplot

plt.scatter(df['AmountPaid60'], df['AmountPaid360'], s= df['DailyRate'],
            c=df['Deposit'], cmap='RdYlGn', edgecolors = 'black')
cbar = plt.colorbar()
cbar.set_label('Deposit')
##distribution plot

sns.distplot(df['AmountPaid60'])
#Pairplot

sns.pairplot(df)
#Countplot
sns.countplot('CustomerGender' , hue = 'LoanStatus360', data = df )

sns.countplot('CustomerGender' , hue = 'Product', data = df)<jupyter_output>/usr/local/lib/python3.6/dist-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  FutureWarning<jupyter_text>Product C was most popular amongst those who did not specify their gender<jupyter_code>sns.countplot('LoanStatus360', hue ='Product', data=df)
#Adjusting size
plt.figure(figsize=(12,7))

sns.countplot('Product', hue='LoanStatus360', data = df).set(title= 'Product and Loan Status', ylabel = 'Product count')


sns.countplot('Product' , data = df)
#Product B is the most popular followed by Product A
#Product F is the least popular followed by Product E
 #Barplot
 sns.barplot(x = 'Product', y = 'TotalPrice', data = df).set(xlabel=' Product',
      ylabel='Price',
      title='Price vs Product')<jupyter_output><empty_output><jupyter_text>Product F is the highest priced followed by E which accounts for their low popularityProduct C is the lowest priced Product B is the 2nd cheapest which accounts for its popularityProduct A is moderately priced which accounts for its popularity<jupyter_code>ax = sns.barplot(x = 'Product', y = 'Deposit', data = df, ci = None, estimator = np.median)
ax.set(xlabel=' Product',
       ylabel='Deposit',
       title='Deposit vs Product')
plt.show()
ax = sns.barplot(x = 'Product', y = 'DailyRate', data = df, ci = None, estimator = np.median)
ax.set(xlabel=' Product',
       ylabel='DailyRate',
       title='DailyRate vs Product')
plt.show()<jupyter_output><empty_output><jupyter_text>Products B and C have the lowest daily ratesProducts E and F have the highest daily rates<jupyter_code>#boxplot

sns.boxplot(df['AmountPaid60'])
#Heatmap

corr_matrix = df.corr()
sns.heatmap(corr_matrix, annot=True)<jupyter_output><empty_output><jupyter_text>Model DevelopmentModels will be trained and the best chosen. the models include:  * Linear Regression * Lasso * Ridge * ElasticNet We will go about it in the following steps:  * Ensure that data is model ready: Clean and Numerical  * Separate our data into features and target.  * Split our data into train and test Hold out Validation and Cross Validation)  * Train our models  * Evaluate our models  * Hyperparameter tune the best model  Feature Engineering<jupyter_code>df.head()
#Drop columns that we will not use
df.drop(['TotalPrice','TotalDays', 'StartDate','AmountPaid60', 'Location'], axis = 1, inplace = True)
df.head()
#Change our categorical columns into numerical columns
#One hot encode Customer gender
df = pd.get_dummies(df, columns=['CustomerGender'])
df.head()
#Label encode the rest

#Import Label Encoder from sklearn
from sklearn.preprocessing import LabelEncoder

#Initialise the encoder
le = LabelEncoder()

#Get a list called objects that contains the columns with categorical data
objects= df.select_dtypes(include = 'object').columns


#Label encode these columns using the label encoder
for column in objects:
  df[column]=le.fit_transform(df[column])
#Confirm the changes
df.head()<jupyter_output><empty_output><jupyter_text>Preparation for model development<jupyter_code># Separate our features(X) and target(y)

X = df.drop('AmountPaid360', axis = 1)

y = df['AmountPaid360']
#Split into train and test

#Import train test split from sklearn

from sklearn.model_selection import train_test_split

#Split our data

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, 
                                                    random_state=42)<jupyter_output><empty_output><jupyter_text>Model 1 : Linear regression<jupyter_code># Import models

from sklearn.linear_model import LinearRegression, Lasso, Ridge, ElasticNet

#Initialise the model
lr = LinearRegression()

#train the model
lr.fit(X_train, y_train)

#predictions
lr_predictions = lr.predict(X_test)
#Evaluating the accuracy of the model using MAE,MSE, RMSE and R2 score

#import the metrics
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score

# Evaluation function
def evaluator(y_true, y_pred):
  ''' Prints out the MAE, RMSE and R2 score '''
  
  mae = mean_absolute_error(y_true, y_pred)
  rmse = mean_squared_error(y_true, y_pred, squared=False)
  r2 = r2_score(y_true, y_pred)
  print('The MAE is {:.2f}, the MSE is {:.2f} and the R2 score is {:.2f}'.format(mae, rmse, r2))


evaluator(y_test, lr_predictions)
# Checking the Training accuracy using the score function. It uses the r2 score
lr.score(X_train, y_train)
#Plot the data to see the line of best fit
sns.regplot(x = y_test, y = lr_predictions).set(xlabel = 'Actual Bike Count',
      ylabel = 'Predicted Bike Count')<jupyter_output><empty_output><jupyter_text>Lasso ModelCheck the following links for more information on Lasso and Ridge:* https://www.analyticsvidhya.com/blog/2016/01/ridge-lasso-regression-python-complete-tutorial/* https://www.analyticsvidhya.com/blog/2017/06/a-comprehensive-guide-for-linear-ridge-and-lasso-regression/<jupyter_code>#Call the model
lasso = Lasso(alpha=1)
#Constant that multiplies the L1 term. Defaults to 1.0.
#alpha = 0 is equivalent to the Linear Regression model. For numerical

#train
lasso.fit(X_train, y_train)

#predict
lasso_predictions = lasso.predict(X_test)


#Evaluate model
evaluator(y_test, lasso_predictions)<jupyter_output>The MAE is 3398.20, the MSE is 4323.42 and the R2 score is 0.63<jupyter_text>There is a slight increase in the MAE. This is because of the constrained weights.   Ridge<jupyter_code>#Call it
ridge = Ridge(alpha=1)

#train

ridge.fit(X_train, y_train)

#Predict
ridge_predictions = ridge.predict(X_test)
#Evaluate model
evaluator(y_test, ridge_predictions)<jupyter_output>The MAE is 3397.98, the MSE is 4323.27 and the R2 score is 0.63<jupyter_text>ElasticNet<jupyter_code>#Call it
enet = ElasticNet(alpha = 0.8, l1_ratio= 0.5)

#train
enet.fit(X_train, y_train)

#predict
enet_predictions = enet.predict(X_test)


#Evaluate model
evaluator(y_test, enet_predictions)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09342379958246347, 'fraction_numerical': 0.01722338204592902, 'mean_word_length': 5.141666666666667, 'pattern_counts': {'":': 0, '<': 46, '<?xml version=': 0, '>': 46, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '200829', 'license': 'MIT', 'n_tokens_mistral': 3188, 'n_tokens_neox': 2977, 'n_words': 1052, 'path': 'Kopa_Project.ipynb', 'repo_name': 'mDianga/Kopa_Project', 'size': '9580'}