<jupyter_start><jupyter_text>**Car dataset price prediction - SELL-CAR-O**<jupyter_code>import pandas as pd#numerical computations library

df = pd.read_csv("car data.csv")#read the data
df.head()#shows first 5 rows of dataset
df.shape#shows shape of the entire dataset<jupyter_output><empty_output><jupyter_text>**In the above dataset observation, features like Fuel_Type	and Seller_Type,Transmission,Owner are categorical**<jupyter_code>print(df['Seller_Type'].unique())#seeing unique values in seller type
print(df['Transmission'].unique())#seeing unique values in transmission
print(df['Owner'].unique())#seeing unique values in owner
print(df['Fuel_Type'].unique())<jupyter_output>['Dealer' 'Individual']
['Manual' 'Automatic']
[0 1 3]
['Petrol' 'Diesel' 'CNG']<jupyter_text>**Missing values findings**<jupyter_code>df.isnull().sum()<jupyter_output><empty_output><jupyter_text>**shows that No missing values in this dataset**<jupyter_code>df.describe()<jupyter_output><empty_output><jupyter_text>**shows the details of count,standard deviation and other properties**<jupyter_code>df.columns<jupyter_output><empty_output><jupyter_text>**Removed car names- not usable**<jupyter_code>mod_dataset= df[[ 'Year', 'Selling_Price', 'Present_Price', 'Kms_Driven',
       'Fuel_Type', 'Seller_Type', 'Transmission', 'Owner']] 
mod_dataset.head()<jupyter_output><empty_output><jupyter_text>**Categorical labelling of column feature**<jupyter_code>mod_dataset['Present_Year'] = 2020
mod_dataset.head()
mod_dataset['no_year']=mod_dataset['Present_Year']- df['Year']
mod_dataset.head()<jupyter_output><empty_output><jupyter_text>**Now "years" and "present year" feature can be used as estimating age of the car in terms of money value so, we are creating new feature "no. of years"**<jupyter_code>mod_dataset.drop(['Year'],axis=1,inplace=True)
mod_dataset.head()
mod_dataset.drop(['Present_Year'],axis=1,inplace=True)
mod_dataset.head()
mod_dataset = pd.get_dummies(mod_dataset,drop_first = True)
mod_dataset.head()
mod_dataset.corr()<jupyter_output><empty_output><jupyter_text>**Data Visualisation**<jupyter_code>import seaborn as sns#data visualisation library
import matplotlib.pyplot as plt#plotting library
%matplotlib inline<jupyter_output><empty_output><jupyter_text>**Heat map**<jupyter_code>corrmat = mod_dataset.corr()
top_corr_features = corrmat.index
plt.figure(figsize=(20,20))
heat = sns.heatmap(mod_dataset[top_corr_features].corr(),annot=True,cmap="RdYlGn")<jupyter_output><empty_output><jupyter_text>**Dark green - highly +vely correlated     Dark red   - highly -vely correlated**<jupyter_code>mod_dataset.head()<jupyter_output><empty_output><jupyter_text>**Here dependent feature is "Selling price" as y and rest are independent features**<jupyter_code>X = mod_dataset.iloc[:,1:]

X.head()
y = mod_dataset.iloc[:,0]

y.head()<jupyter_output><empty_output><jupyter_text>**we are going to see the importance of features**<jupyter_code>from sklearn.ensemble import ExtraTreesRegressor
model = ExtraTreesRegressor()
model.fit(X,y)
print(model.feature_importances_)<jupyter_output>[0.36918615 0.03906453 0.00096716 0.07662817 0.23779576 0.01351081
 0.12545851 0.13738891]<jupyter_text>**From this, Present_price , fuel-type-diesel are highly important, just to see for the comparison** **Let's see as visualisation**<jupyter_code>f_imp = pd.Series(model.feature_importances_, index =X.columns)
f_imp.nlargest(5).plot(kind='barh')
plt.show()<jupyter_output><empty_output><jupyter_text>**Training and test dataset splitting**<jupyter_code>from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.2,random_state=0)
from sklearn.linear_model import LinearRegression
lr = LinearRegression()
lr.fit(X_train,y_train)
y_pred_lr = lr.predict(X_test)
print(y_pred_lr)
from sklearn.ensemble import RandomForestRegressor
rf = RandomForestRegressor()
rf.fit(X_train,y_train)<jupyter_output><empty_output><jupyter_text>**Next by optimising the best values for hyper-parameters by doing Randomised serchCV** **Previously we had trained the model with default paramsbut had to be increased its model performance in terms of hyperparameters****One such is by no. of estimators and goes on**<jupyter_code>import numpy as np
n_estimators = [int(x) for x in np.linspace(start = 100,stop = 1200,num =12)]
print(n_estimators)##tells the no. of decision trees in the random forest
max_features = ['auto','sqrt']#functions to be used at each split happens in the forest
max_depth = [int(x) for x in np.linspace(5,30,num =6)]
##tells the depth of the tree(say like in terms of leafs and nodes of a normal tree)


min_samples_split = [2,5,10,15,100] #minimum number of samples required to split an internal node of a tree
min_samples_leaf = [1, 2, 5, 10] #tells min.no of leaf in nodes of tree
from sklearn.model_selection import RandomizedSearchCV
   
rscv = {'n_estimators': n_estimators,
       'max_features': max_features,
       'max_depth': max_depth,
       'min_samples_split': min_samples_split,
       'min_samples_leaf': min_samples_leaf}

print(rscv)
rf =RandomForestRegressor()
rf_rcv = RandomizedSearchCV(estimator = rf, param_distributions = rscv,scoring='neg_mean_squared_error', n_iter = 10, cv = 5, verbose=2, random_state=42, n_jobs = 1)
#scoring tells based on what the predict evaluations take place
#verbose shows the results
#njobs tells parallel computaion use of the laptop cores
rf_rcv.fit(X_train,y_train)
pred = rf_rcv.predict(X_test)
pred
sns.distplot(y_test - pred)
plt.scatter(y_test,pred)
from sklearn import metrics
print('MAE:', metrics.mean_absolute_error(y_test, pred))
print('MSE:', metrics.mean_squared_error(y_test, pred))
print('RMSE:', np.sqrt(metrics.mean_squared_error(y_test, pred)))
import pickle
file = open('r_forest_regress_model.pkl', 'wb')
pickle.dump(rf_rcv, file)#store info to that file<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12115059221658206, 'fraction_numerical': 0.02131979695431472, 'mean_word_length': 8.2359375, 'pattern_counts': {'":': 0, '<': 63, '<?xml version=': 0, '>': 63, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '429040', 'license': 'MIT', 'n_tokens_mistral': 2185, 'n_tokens_neox': 2057, 'n_words': 465, 'path': 'Car_prices.ipynb', 'repo_name': 'Gnanaprakaasham/SELL_CAR-O-', 'size': '5910'}