<jupyter_start><jupyter_text>Car Price PredictionIn this session we will be using the used car data set to explore the various features, do some feature enginering, tune some parameters, create a ML model and store it to use whenever requiredThe columns in the given dataset are as follows:Car_Name - Name of the carYear - Year of PurchaseSelling_Price - Selling Price of the used car in lacsPresent_Price - Selling Price of the new car in lacsKms_driven - Total kilometers drivenFuel - Type of fuels requiredSeller_Type -  Type of the SellerTransmission - Type of transmissionOwner - Number of owners the car had  Importing the libraries<jupyter_code>import os
import pandas as pd
import seaborn as sns
import numpy as np
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression, Lasso
from sklearn.metrics import mean_squared_error
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.ensemble import RandomForestRegressor
import pickle

pd.set_option('display.max_rows', 500)
pd.set_option('display.max_columns', 500)
pd.set_option('display.width', 1000)<jupyter_output><empty_output><jupyter_text>Read the data<jupyter_code>df = pd.read_csv('../data/car data.csv')
df.head()<jupyter_output><empty_output><jupyter_text>Basic Information<jupyter_code>df.info()
# Check for null values
df.isnull().sum()
# Descriptive stats for numeric data type

df.describe()
# Descriptive stats for categorical data type

df.describe(exclude=np.number)<jupyter_output><empty_output><jupyter_text>***Observations:*** - Year is in integer but its true nature is categorical - 32k for a new car price looks odd - Owners seem to have a few outliers and it has low variance (because approx. 75% of the values have 0 (meaning 1st owner cars))  EDA  Univariate analysis - Numeric features<jupyter_code># Number of cars for a given year

df['Year'].value_counts().plot.bar(figsize=(15, 5), rot=45)
plt.title('Car count by year of the original purchase')
plt.xlabel('Year')
plt.ylabel('Count')
plt.show()
# Distribution of selling price and present price

fig, ax = plt.subplots(1, 2, figsize=(16, 5))
ax = ax.flatten()

ax[0].hist(df['Selling_Price'], bins=20)
ax[0].set_title('Distribution of selling price')

ax[1].hist(df['Present_Price'], bins=40)
ax[1].set_title('Distribution of present price')

plt.show()
# Categorize car w.r.t. its present price

def car_segment(price):
    if price <= 5:
        return 'economical'
    elif price <= 10:
        return 'normal'
    elif price <= 20:
        return 'high-end'
    elif price > 20:
        return 'premium'
df['Car_Segment'] = df['Present_Price'].map(car_segment)
df['Car_Segment'].value_counts().plot.bar(figsize=(15, 5), rot=0)
plt.title('Car segment by original purchase price')
plt.xlabel('Car Segment')
plt.ylabel('Count')
plt.show()
# Distribution of kms driven

df['Kms_Driven'].hist(bins=50, figsize=(15, 5), grid=False)
plt.title('Distribution of Kms driven')
plt.xlabel('Kms Driven')
plt.ylabel('Count')
plt.show()
# Create segments for kms driven

def driven_segment(kms):
    if kms < 10000:
        return 'below_10k'
    elif kms < 30000:
        return 'below_30k'
    elif kms < 50000:
        return 'below_50k'
    elif kms < 100000:
        return 'below_100k'
    elif kms >= 100000:
        return 'above_100k'
df['Driven_Segment'] = df['Kms_Driven'].map(driven_segment)
df['Driven_Segment'].value_counts().plot.bar(figsize=(15, 5), rot=0)
plt.title('Driven segment by kms driven')
plt.xlabel('Driven Segment')
plt.ylabel('Count')
plt.show()
# Count of cars wrt number of owners

df['Owner'].value_counts().plot.bar(figsize=(10, 5), rot=0)
plt.title('Count of cars by number of owners')
plt.xlabel('Owners')
plt.ylabel('Count')
plt.show()
df['Owner'].value_counts(normalize=True) * 100<jupyter_output><empty_output><jupyter_text>***Observations*** - 96% of the cars have single owner hence this feature does not have significant variance and will not contribute much in predicting selling price.  Univariate analysis - Categorical features<jupyter_code># Plot top 10 cars

df['Car_Name'].value_counts()[:10].plot.bar(figsize=(15, 7), rot=45)
plt.title('Top 10 cars')
plt.xlabel('Car Name')
plt.ylabel('Count')
plt.show()<jupyter_output><empty_output><jupyter_text>***Observations***: - Royal Enfield is the bike which indicates that data contains bikes informaiton as well<jupyter_code>df['Car_Name'].unique()
# Value counts by fuel type

df['Fuel_Type'].value_counts(normalize=True).plot.bar(figsize=(10, 5), rot=0)
plt.title('Count of cars by fuel type')
plt.xlabel('Fuel Type')
plt.ylabel('Count')
plt.show()
# Value counts by seller type

df['Seller_Type'].value_counts(normalize=True).plot.bar(figsize=(10, 5), rot=0)
plt.title('Count of cars by seller type')
plt.xlabel('Seller Type')
plt.ylabel('Count')
plt.show()
# Value counts by transmission type

df['Transmission'].value_counts(normalize=True).plot.bar(figsize=(10, 5), rot=0)
plt.title('Count of cars by transmission type')
plt.xlabel('Transmission')
plt.ylabel('Count')
plt.show()
bikes_identifier = [
    'Royal Enfield', 'Bajaj', 'Honda CB', 'Hero', 'Yamaha', 'Activa', 'TVS', 'KTM',
    'Hyosung', 'Honda Karizma', 'Access 125', 'Mahindra Mojo', 'Honda Dream Yuga', 'UM Renegade Mojave'
]

def vehicle_type(name):
    if any(i in name for i in bikes_identifier):
        return 'two_whl'
    
    return 'four_whl'
df['Vehicle_Type'] = df['Car_Name'].map(vehicle_type)
# Value counts by vehicle type

df['Vehicle_Type'].value_counts(normalize=True).plot.bar(figsize=(10, 5), rot=0)
plt.title('Count of vehicles by its type')
plt.xlabel('Vehicle')
plt.ylabel('Count')
plt.show()<jupyter_output><empty_output><jupyter_text>***Observations***: - Bikes included - Royal Enfield, Bajaj, Honda CB, Hero, Yamaha, Activa, TVS, KTM, Hyosung, Honda Karizma, Access 125, Mahindra Mojo - Approx. 80% of the vehicles are on petrol  Bivariate analysis<jupyter_code># Plot new purchase price of the bikes and cars

plt.figure(figsize=(15, 7))
plt.plot(np.sort(df[df['Vehicle_Type'] == 'two_whl']['Present_Price']), color='b', label='Two Wheeler')
plt.plot(np.sort(df[df['Vehicle_Type'] == 'four_whl']['Present_Price']), color='r', label='Four Wheeler')
plt.legend()
plt.show()
# Min and max of cars and bikes of present price

df.pivot_table(index='Vehicle_Type', values='Present_Price', aggfunc=['min', 'max'])
# Average of present price of cars

df[df['Vehicle_Type'] == 'four_whl'].pivot_table(index='Vehicle_Type', values='Present_Price', 
                                                 aggfunc=['median', 'mean'], columns='Car_Segment')
# Looking for 4 wheelers under 3 lac

df[(df['Vehicle_Type'] == 'four_whl') & (df['Present_Price'] < 3)]
df['Car_Segment'].unique()
cars = df[df['Vehicle_Type'] == 'four_whl']
# Correlation between selling price and year

fig, ax = plt.subplots(2, 2, figsize=(15, 7))
ax = ax.flatten()

for i, seg in enumerate(cars['Car_Segment'].unique()):
    car_seg = cars[cars['Car_Segment'] == seg]
    
    sns.regplot(x='Year', y='Selling_Price', data=car_seg, ax=ax[i])
    ax[i].set_title(f'Price correlation with year for {seg} segment')
    
    #ax[i].scatter(car_seg['Year'], car_seg['Selling_Price'])
    #ax[i] = plt.gca().invert_xaxis()

plt.tight_layout()
plt.show()
cars['Driven_Segment'].unique()
# Correlation between selling price and year w.r.t. kms driven

fig, ax = plt.subplots(3, 2, figsize=(15, 8))
ax = ax.flatten()
fig.delaxes(ax[-1])   # Deleting 6th axis to accomodate 5 subplots

for i, seg in enumerate(cars['Driven_Segment'].unique()):
    car_kms = cars[cars['Driven_Segment'] == seg]
    
    sns.regplot(x='Year', y='Selling_Price', data=car_seg, ax=ax[i])
    ax[i].set_title(f'Price correlation with year for {seg} driven segment')
    
plt.tight_layout()
plt.show()<jupyter_output><empty_output><jupyter_text>***Observations:*** - All of above two subplots shows upward trend and that means the more recent the cars more the price.  Create a pair plot of the entire data<jupyter_code># Pair plot of the entire data

sns.pairplot(df)
plt.show()
df.corr()<jupyter_output><empty_output><jupyter_text>Create a new feature which tells us how old the car is in terms of years  Feature Engineering<jupyter_code># # Subtracting the year by current year
# cars['no_of_years'] = 2021 - cars['Year']
# cars.head()<jupyter_output><empty_output><jupyter_text>Feature Encoding<jupyter_code># Create features and target (consider only cars data)

X = cars.drop(['Car_Name', 'Car_Segment', 'Driven_Segment', 'Selling_Price', 'Owner'], axis=1)
y = cars['Selling_Price']
# Categorical feature encoding

X_OHE = pd.get_dummies(X, drop_first=True)
X_OHE.head()
# Split the data into train and test

X_train, X_test, y_train, y_test = train_test_split(X_OHE, y, test_size=0.2, random_state=10)<jupyter_output><empty_output><jupyter_text>Model Building and Evaluating<jupyter_code>lr = LinearRegression()
ls = Lasso()
rfr = RandomForestRegressor()
gbr = GradientBoostingRegressor()
# Loop over models and calculate RMSE

models = [('Linear Regression', lr), ('Lasso', ls), ('Random Forest', rfr), ('Gradient Boosting', gbr)]

for m in models:
    model = m[1]
    name = m[0]
    
    model.fit(X_train, y_train)
    y_pred = model.predict(X_test)
    
    print(f'RMSE for {name}: ')
    print(round(np.sqrt(mean_squared_error(y_test, y_pred)), 4))<jupyter_output>RMSE for Linear Regression: 
1.6804
RMSE for Lasso: 
1.918
RMSE for Random Forest: 
1.1326
RMSE for Gradient Boosting: 
0.9866<jupyter_text>***Note:*** - Scaling is required for parametric models (e.g. linear regression, logistic regression --have certain assumption about data), however non-parametric models such as decision trees, random forest, gradient boosting etc. does not require to scale the data.  Randomized Search CV on the GBR model, find out the best parameters and the results as well<jupyter_code>from sklearn.model_selection import RandomizedSearchCV

# Number of trees in random forest
n_estimators = [int(x) for x in np.linspace(start = 100, stop = 1200, num = 12)]

# Number of features to consider at every split
max_features = ['auto', 'sqrt']

# Maximum number of levels in tree
max_depth = [int(x) for x in np.linspace(5, 30, num = 6)]

# max_depth.append(None)
# Minimum number of samples required to split a node
min_samples_split = [2, 5, 10, 15, 100]

# Minimum number of samples required at each leaf node
min_samples_leaf = [1, 2, 5, 10]
# Create the random grid
random_grid = {'n_estimators': n_estimators,
               'max_features': max_features,
               'max_depth': max_depth,
               'min_samples_split': min_samples_split,
               'min_samples_leaf': min_samples_leaf}

print(random_grid)
# Initalizing the RandomSearchCV
gbr_random = RandomizedSearchCV(estimator=gbr, param_distributions=random_grid,
                                scoring='neg_mean_squared_error', n_iter=10, cv=5, 
                                verbose=2, random_state=42, n_jobs=1)
# Fitting the data
gbr_random.fit(X_train,y_train)
gbr_random.best_params_
# Getting the predictions
y_pred = gbr_random.predict(X_test)

print('RMSE for Fine tuned GBR: ')
print(round(np.sqrt(mean_squared_error(y_test, y_pred)), 4))<jupyter_output>RMSE for Fine tuned GBR: 
1.254<jupyter_text>Package your model using pickle and store it<jupyter_code>import pickle

# Open a file, where you ant to store the data
with open('gradient_boost_regression_model.pkl', 'wb') as f:
    # Dump information to that file
    pickle.dump(gbr_random, f)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1281123459981046, 'fraction_numerical': 0.01947100887395537, 'mean_word_length': 5.135306553911205, 'pattern_counts': {'":': 0, '<': 59, '<?xml version=': 0, '>': 53, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '455201', 'license': 'Apache-2.0', 'n_tokens_mistral': 4209, 'n_tokens_neox': 3913, 'n_words': 1107, 'path': '14_Car_price_prediction/notebook/Selling_price_prediction.ipynb', 'repo_name': 'hirenhk15/ga-code-alongs', 'size': '11607'}