<jupyter_start><jupyter_text>[Task] Predict the selling price for the [Vehicle Dataset](https://www.kaggle.com/nehalbirla/vehicle-dataset-from-cardekho)  Load the dataset, perform all necessary data exploration and preprocessing  Perform proper hyperparameter tuning to choose the best base model, depth, max_features, and any other important hyper-parameters<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder
from sklearn import metrics
from sklearn import preprocessing
from sklearn.impute import SimpleImputer
from sklearn.ensemble import RandomForestClassifier
import plotly.express as px
from sklearn.preprocessing import OneHotEncoder
from sklearn.compose import make_column_transformer
from sklearn.tree import DecisionTreeRegressor
from sklearn.model_selection import GridSearchCV, RandomizedSearchCV
from sklearn.ensemble import RandomForestRegressor
from sklearn.model_selection import cross_val_score<jupyter_output><empty_output><jupyter_text>**Loading data**<jupyter_code>df=pd.read_csv('Car details v3.csv')
df.head()
df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 8128 entries, 0 to 8127
Data columns (total 13 columns):
 #   Column         Non-Null Count  Dtype  
---  ------         --------------  -----  
 0   name           8128 non-null   object 
 1   year           8128 non-null   int64  
 2   selling_price  8128 non-null   int64  
 3   km_driven      8128 non-null   int64  
 4   fuel           8128 non-null   object 
 5   seller_type    8128 non-null   object 
 6   transmission   8128 non-null   object 
 7   owner          8128 non-null   object 
 8   mileage        7907 non-null   object 
 9   engine         7907 non-null   object 
 10  max_power      7913 non-null   object 
 11  torque         7906 non-null   object 
 12  seats          7907 non-null   float64
dtypes: float64(1), int64(3), object(9)
memory usage: 825.6+ KB<jupyter_text>**EDA**<jupyter_code>plt.figure(figsize= (20,10))
sns.histplot(x = 'selling_price', kde = True, data = df)
plt.figure(figsize= (20,10))
sns.boxplot(x= 'transmission' , y= 'selling_price', data = df)
plt.figure(figsize= (20,10))
sns.boxplot(x= 'fuel' , y= 'selling_price', data = df)
plt.figure(figsize= (20,10))
sns.boxplot(x= 'owner' , y= 'selling_price', data = df)
sns.boxplot(x= 'seats' , y= 'selling_price', data = df)
sns.scatterplot(x= 'year' , y= 'selling_price', data = df)
sns.scatterplot(x= 'km_driven' , y= 'selling_price', data = df)<jupyter_output><empty_output><jupyter_text>**As expected, the more km_driven, the older the car, the lesser the price is. Also, as the number of owners increase, the price decreases with the test driver car way higher in price. Also, automatic cars are generally more expensive than manual cars. We can also norte that the selling price follows a skewed normal distribution.**  **Data Cleaning and preprocessing**<jupyter_code>df.isnull().sum()
df.duplicated().sum()
df.drop_duplicates(inplace=True)
df.duplicated().sum()<jupyter_output><empty_output><jupyter_text>First, I am going to drop the torque column as it's not straighforward to deal with and I don't know the rules that govern it. I am also going to drop the name as it's irrelevant to the price.<jupyter_code>df.drop('torque',axis=1,inplace=True)
df.drop('name',axis=1,inplace=True)<jupyter_output><empty_output><jupyter_text>Second, I am going to drop CC from the engine column and bhp from the max_power and the milage unit.Note: Even though there are different milage units for different fuel types, the units can be dropped as the fuel type is specified with each car.<jupyter_code>df['engine']=df['engine'].str.extract(r'(^\d*)').astype(float)
df['max_power']=df['max_power'].str.extract(r'(\d+.\d*)').astype(float)
df['mileage']=df['mileage'].str.extract(r'(\d+.\d*)').astype(float)<jupyter_output><empty_output><jupyter_text>**Columns transformers**<jupyter_code>pip install catboost
from sklearn.impute import KNNImputer, SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import make_column_transformer
from imblearn.pipeline import make_pipeline
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_validate, train_test_split
from sklearn.model_selection import RepeatedStratifiedKFold, StratifiedKFold

from sklearn.linear_model import LinearRegression
from sklearn.ensemble import RandomForestRegressor
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.svm import SVC
from sklearn.neighbors import KNeighborsRegressor
from sklearn.tree import DecisionTreeRegressor
from catboost import CatBoostRegressor
from sklearn import metrics
from collections import Counter

#from sklearn.metrics import accuracy_score
from sklearn.metrics import precision_recall_fscore_support, classification_report, confusion_matrix
df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
Int64Index: 6926 entries, 0 to 8125
Data columns (total 11 columns):
 #   Column         Non-Null Count  Dtype  
---  ------         --------------  -----  
 0   year           6926 non-null   int64  
 1   selling_price  6926 non-null   int64  
 2   km_driven      6926 non-null   int64  
 3   fuel           6926 non-null   object 
 4   seller_type    6926 non-null   object 
 5   transmission   6926 non-null   object 
 6   owner          6926 non-null   object 
 7   mileage        6718 non-null   float64
 8   engine         6718 non-null   float64
 9   max_power      6717 non-null   float64
 10  seats          6718 non-null   float64
dtypes: float64(4), int64(3), object(4)
memory usage: 649.3+ KB<jupyter_text>**To deal with missing data, I will use knnImputer.**<jupyter_code>scaler = StandardScaler()
imp_knn = KNNImputer(n_neighbors=6)
imputer = SimpleImputer(missing_values=np.nan, strategy='mean')

ohe = OneHotEncoder()

num_cols = ['year', 'km_driven', 'mileage', 'engine', 'max_power', 'seats']
cat_cols = ['fuel', 'seller_type', 'transmission', 'owner']

preprocessor = make_column_transformer(
    (imp_knn, num_cols),
    (ohe, cat_cols),
    remainder='passthrough')
X = df.drop('selling_price', axis = 1)
y = df['selling_price']

preprocessor.fit_transform(X)
X
transformed_column_names = (num_cols +
                            preprocessor.named_transformers_['onehotencoder'].
                            get_feature_names_out().tolist())

X
X = pd.DataFrame(preprocessor.fit_transform(X),
                 columns=transformed_column_names)
X
X_train, X_test, y_train, y_test = train_test_split(X,
                                                    y,
                                                    test_size=0.2,
                                                    random_state=42)
X.isna().sum()
y_train<jupyter_output><empty_output><jupyter_text>**Modeling**I will try different models and test their accuracy. The models are:1.   Decision Tree2.   Random forest regression3. K neighbor regression4. Cat boost regression5. Linear regression<jupyter_code>Models = {
    'Decison Tree': DecisionTreeRegressor(),
    'RF': RandomForestRegressor(),
    'KNN': KNeighborsRegressor(),
    'CatBoost': CatBoostRegressor(),
    'LinReg': LinearRegression()
}
model_list = []
train_r2_list = []
train_MSE_list = []
counter_list = []
test_r2 = []
test_MSE = []

# Main model loop
for model, clf in Models.items():
    print('Running on '+model)
    print(60*'=')
    clf.fit(X_train, y_train)
    predictions = clf.predict(X_test)
    r2 = metrics.r2_score(y_test, predictions)
    MSE = metrics.mean_squared_error(y_test, predictions, squared=False)
    train_pred = clf.predict(X_train)
    train_r2 = metrics.r2_score(y_train, train_pred)
    train_MSE = metrics.mean_squared_error(y_train, train_pred, squared=False)
    counter = Counter(predictions)
    
  

    model_list.append(model)
    train_r2_list.append(train_r2)
    counter_list.append(counter)
    test_r2.append(r2)
    train_MSE_list.append(train_MSE)
    test_MSE.append(MSE)

results = pd.DataFrame({
    "model": model_list,
    "train_R2_Score": train_r2_list,
    'test_r2': test_r2,
    'train_MSE': train_MSE_list,
    'test_MSE': test_MSE,
    'counter': counter_list
})
results<jupyter_output><empty_output><jupyter_text>The best performing model is the random forest.  **Cross Validation**<jupyter_code>RF=RandomForestRegressor()
p_grid= {'n_estimators' : np.arange(10,100,10), 'max_features' : ['auto', 'sqrt'],
         'max_depth' : np.arange(2,20,2)}
RF_random = RandomizedSearchCV(estimator = RF, param_distributions = p_grid, n_iter = 20, cv = 3, verbose=2, random_state=42, n_jobs = -1)
RF_random.fit(X_train,y_train)

RF_random.best_params_, RF_random.best_score_<jupyter_output>Fitting 3 folds for each of 20 candidates, totalling 60 fits
{'fraction_non_alphanumeric': 0.10135817712425638, 'fraction_numerical': 0.02929621730833988, 'mean_word_length': 4.00561797752809, 'pattern_counts': {'":': 2, '<': 40, '<?xml version=': 0, '>': 40, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '319410', 'license': 'MIT', 'n_tokens_mistral': 3097, 'n_tokens_neox': 2888, 'n_words': 756, 'path': 'Vehicle Dataset Analysis/Copy_of_Task_9 (1).ipynb', 'repo_name': 'ShadyFouadd/Machine_Learning', 'size': '8909'}