<jupyter_start><jupyter_text><jupyter_code># Getting our tools ready
import pandas as pd
import numpy as plt
import numpy as np

## IMporting the preprocessed data uploaded from jupyter 
train_preprocessed=pd.read_csv("/content/drive/My Drive/house_price_preprocessed_data.csv")
train_preprocessed.head(5)<jupyter_output><empty_output><jupyter_text>Evaluating results in different MODELS<jupyter_code>## Splitting the data into train and test
np.random.seed(100)
from sklearn.model_selection import train_test_split
X=train_preprocessed.drop("SalePrice",axis=1)
y=train_preprocessed["SalePrice"]
X_train,X_valid,y_train,y_valid=train_test_split(X,
                                                 y,
                                                 test_size=0.2,
                                                 shuffle=True)
## Import the models
## IMporting the models for regression to experiment on those 
from sklearn.linear_model import LinearRegression,Lasso,Ridge
from sklearn.ensemble import RandomForestRegressor,GradientBoostingRegressor,AdaBoostRegressor
from sklearn.svm import SVR
from sklearn.tree import DecisionTreeRegressor
## Making a dictionary of models
models={"LinearRegression":LinearRegression(),
       "Lasso":Lasso(),
       "Ridge":Ridge(),
       "RandomForestRegression":RandomForestRegressor(),
       "Gradient":GradientBoostingRegressor(),
       "AdaBoostRegressor":AdaBoostRegressor(),
       "SVR":SVR(),
       "DecisionTreeRegressor":DecisionTreeRegressor()}
## Making a function to fit and score on the models
from sklearn.metrics import mean_absolute_error
def fit_and_score(models,X_train,y_train,X_valid,y_valid):
    results={}
    for model_name,model in models.items():
        model.fit(X_train,y_train)
        results[model_name+" accuracy"]=model.score(X_valid,y_valid)
        y_preds=model.predict(X_valid)
        results[model_name+" mean_absolute_error"]=mean_absolute_error(y_valid,y_preds)

    return results
        
## Let's evaluate the function
%%time
result_1=fit_and_score(models,X_train,y_train,X_valid,y_valid)
result_1
result_1<jupyter_output><empty_output><jupyter_text>NOTE: **SVR is discarded due to low metric obtained and is very hard to improve**  Hyperparameter tuning on different models using GridSearchCV  1.RandomForestRegressor<jupyter_code>from sklearn.model_selection import GridSearchCV

## The parameter's values have been set using thorough research
np.random.seed(100)
randomforest_grid={"criterion":["mae"],
           "n_estimators":[100,150,200],
           "min_samples_split":[2,4],
           "max_features":[None,"sqrt"]}
RandomForest_grid=GridSearchCV(RandomForestRegressor(),
                               param_grid=randomforest_grid,
                               cv=5,
                               verbose=True)
## Create a function to check metrics of each model
def check_metrics(model):
    model.fit(X_train,y_train)
    print(model.best_params_)
    print("accuracy",model.score(X_valid,y_valid))
    print("mae",mean_absolute_error(y_valid,model.predict(X_valid)))
## Fitting on the data
check_metrics(RandomForest_grid)<jupyter_output>Fitting 5 folds for each of 12 candidates, totalling 60 fits<jupyter_text>**NOte:It shows that mean_absolute_error has improved from 17042 to 17015 thought the accuracy has decreased.**<jupyter_code><jupyter_output><empty_output><jupyter_text>2.GradientBoostingRegressor<jupyter_code>## Making a gradientboosting hyperparameters grid 
np.random.seed(100)
gradientboost_grid={"criterion":["mae"],
           "n_estimators":[100,150,200],
           "min_samples_split":[2,4],
           "max_features":[None,"sqrt"]}
Gradient_grid=GridSearchCV(RandomForestRegressor(),
                               param_grid=gradientboost_grid,
                               cv=5,
                               verbose=True)
check_metrics(Gradient_grid)
def check_metrics_new(model):
    model.fit(X_train,y_train)
    print("accuracy",model.score(X_valid,y_valid))
    print("mae",mean_absolute_error(y_valid,model.predict(X_valid)))
for i in range(590,650,10):
  print(f"{i} estimators")
  check_metrics_new(GradientBoostingRegressor(n_estimators=i))
  print("\n")<jupyter_output>590 estimators
accuracy 0.8852450493791971
mae 15633.017280666898


600 estimators
accuracy 0.8827160527610369
mae 15800.072721166362


610 estimators
accuracy 0.8864951020326327
mae 15691.116060834514


620 estimators
accuracy 0.8882523848405276
mae 15621.247033954407


630 estimators
accuracy 0.8853231644387081
mae 15701.377759707291


640 estimators
accuracy 0.885152815048356
mae 15679.425202089096<jupyter_text>**NOTE:For gradientboosting regressor ,the model with hypertuning n_estimators is better and 260 found to be the best**  3.DecisionTreeRegressor<jupyter_code>np.random.seed(100)
decision_tree_grid={"criterion":['mae'],
                    "min_samples_split":[2,4,6,8,10],
                    "max_features":["auto","sqrt"],

                    }
Decision_grid=GridSearchCV(DecisionTreeRegressor(),
                                param_grid=decision_tree_grid,
                                cv=5,
                                verbose=5)
check_metrics(Decision_grid)<jupyter_output>Fitting 5 folds for each of 10 candidates, totalling 50 fits
[CV] criterion=mae, max_features=auto, min_samples_split=2 ...........<jupyter_text>**NOTE:For decision tree regressor ,the model without hypertuning was better** **FINAL NOTE:WITH ALL THOSE TUNING WE FOUND OUT THAT GradientBoostingRegressor IS THE BEST FOR THIS PROBLEM.**   Train the model on full data<jupyter_code>##Splitting the data
X=train_preprocessed.drop("SalePrice",axis=1)
y=train_preprocessed["SalePrice"]
model=GradientBoostingRegressor(n_estimators=260).fit(X,y)
## Exporting the model
import pickle
pickle.dump(model,open("house_regression_model.pkl","wb"))<jupyter_output><empty_output><jupyter_text>Since while importing the model in pkl format in jupyter, it showed warning,we predicted on the test data here  by importing preprocessed test data**<jupyter_code>## importing the test data
test=pd.read_csv("/content/drive/My Drive/test_preprocessed_house_regression.csv")
test.head()<jupyter_output><empty_output><jupyter_text>NOTE: TEST DATA PREPROCESSING IS DONE ON ANOTHER NOTEBOOK WHICH YOU CAN SEE IF YOU GO THROUGH THE REPOSITORY  Predicting on the model<jupyter_code>predictions=model.predict(test)
predictions
## Creating data frame 
submit_to_kaggle_colab=pd.DataFrame(columns=['Id','SalePrice'])
submit_to_kaggle_colab
submit_to_kaggle_colab["Id"],submit_to_kaggle_colab["SalePrice"]=test["Id"],predictions
submit_to_kaggle_colab
## Exporting the data to be submitted in the kaggle
submit_to_kaggle_colab.to_csv("submit_to_kaggle_colab.csv",index=False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09453688705639818, 'fraction_numerical': 0.04579590634663525, 'mean_word_length': 4.490703314470493, 'pattern_counts': {'":': 19, '<': 34, '<?xml version=': 0, '>': 34, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '310283', 'license': 'Apache-2.0', 'n_tokens_mistral': 2442, 'n_tokens_neox': 2132, 'n_words': 429, 'path': 'House_Price_Regression_Hypertuning.ipynb', 'repo_name': 'anjanpa/Regression-House-Price-prediction-kaggle-', 'size': '6791'}