<jupyter_start><jupyter_text>Colab preparation Don't use it, if you don't need it on your own pc<jupyter_code>!pip install fastai==0.7.0
!pip install keras
!pip install kaggle
%load_ext autoreload
%autoreload 2

%matplotlib inline
from fastai.imports import *
from fastai.structured import *

from pandas_summary import DataFrameSummary
from sklearn.ensemble import RandomForestRegressor, RandomForestClassifier
from IPython.display import display

from sklearn import metrics
#Importing Libraries for data preparation
import pandas as pd
import numpy as np
from fastai.imports import *
from fastai.structured import *
# Import kaggle.json from google drive
# This snippet will output a link which needs authentication from any google account
from googleapiclient.discovery import build
import io, os
from googleapiclient.http import MediaIoBaseDownload
from google.colab import auth
auth.authenticate_user()
drive_service = build('drive', 'v3')
results = drive_service.files().list(
    q="name = 'kaggle.json'", fields="files(id)").execute()
kaggle_api_key = results.get('files', [])
# print(kaggle_api_key)
filename = "/content/.kaggle/kaggle.json"
os.makedirs(os.path.dirname(filename), exist_ok=True)
request = drive_service.files().get_media(fileId=kaggle_api_key[0]['id'])
fh = io.FileIO(filename, 'wb')
downloader = MediaIoBaseDownload(fh, request)
done = False
while done is False:
    status, done = downloader.next_chunk()
    print("Download %d%%." % int(status.progress() * 100))
os.chmod(filename, 600)

!cp /content/.kaggle/kaggle.json ~/.kaggle/kaggle.json
!kaggle competitions download -c bluebook-for-bulldozers
!ls
!unzip Train.zip
!ls<jupyter_output><empty_output><jupyter_text>Read Data<jupyter_code>df_raw = pd.read_csv('Train.csv', low_memory=False, 
                     parse_dates=["saledate"])<jupyter_output><empty_output><jupyter_text>Look at the data<jupyter_code>def display_all(df):
    with pd.option_context("display.max_rows", 1000, "display.max_columns", 1000): 
        display(df)
display_all(df_raw.tail().T)
from random import sample
?df_raw.sample<jupyter_output><empty_output><jupyter_text>We take a random sample, of 100 000 records, to speed up the training process<jupyter_code>df_raw = df_raw.sample(100000)
np.shape(df)<jupyter_output><empty_output><jupyter_text>Remember from the Kaggle competition that we're interested in the Log of the SalePrice<jupyter_code>df_raw.SalePrice = np.log(df_raw.SalePrice)<jupyter_output><empty_output><jupyter_text>Data Cleaning and preparationAdd extra date attributes to our dataset<jupyter_code>add_datepart(df_raw, 'saledate')<jupyter_output><empty_output><jupyter_text>Change Strings to categories<jupyter_code>train_cats(df_raw)<jupyter_output><empty_output><jupyter_text>Convert categories to their numerical values.Handle missing values: - Take the median for missing continuous variables- Add extra attributes for missing categorical variables<jupyter_code>df, y, nas = proc_df(df_raw, 'SalePrice')
display_all(df.tail().T)<jupyter_output><empty_output><jupyter_text>Creata a validation set<jupyter_code>from sklearn.model_selection import train_test_split
X_train, X_valid, y_train, y_valid = train_test_split(df, y, test_size=0.30, random_state=42)<jupyter_output><empty_output><jupyter_text>Train ModelsFirst, let's train our random forest<jupyter_code>#import necessary libraries to build model
import random
from sklearn.ensemble import RandomForestRegressor
random.seed(42)
rf = RandomForestRegressor(n_estimators=10)
rf.fit(X_train, y_train)

rf.score(df,y)<jupyter_output><empty_output><jupyter_text>Check how many attributes we have, to create our Neural Network<jupyter_code>np.shape(df)<jupyter_output><empty_output><jupyter_text>We have 66 attributes, so 66 is our input dimension<jupyter_code>from keras.models import Sequential
from keras.layers import Dense, Activation

# Define model
model = Sequential()
model.add(Dense(100, input_dim=66, activation= "relu"))
model.add(Dense(50, activation= "relu"))
model.add(Dense(1))
model.summary() #Print model Summary<jupyter_output><empty_output><jupyter_text>We use Mean Squared Error as the loss function, because we're dealing with a regression problem.We use the optimizer Adam, it uses an adaptive learning rate. Check Keras' documentation for details.<jupyter_code># Compile model
model.compile(loss= "mean_squared_error" , optimizer="adam", metrics=["mean_squared_error"])<jupyter_output><empty_output><jupyter_text>Train the Neural Network.The large discrepancy in the loss function can be explained by the adaptive learning rate. At first, the learning rate is too large, resulting in no convergence. At a certain epoch, the learning rate is at a level which is useful to train the Neural Network.Compared to traditional Machine Learning techniques like Random Forests and Boosting, Neural Networks are a black box. We're not really sure why it works the way it does. This makes understanding our training process harder. It's more difficult to use our understanding of the data to optimize a Neural Network.<jupyter_code># Fit Model
model.fit(X_train, y_train, epochs=100)
# Evaluation while fitting the model
model.fit(X_train, y_train, epochs=10, validation_data=(X_valid, y_valid))<jupyter_output><empty_output><jupyter_text>Model comparisonCompare the mean squared error metric for the Random Forest and the Neural Network<jupyter_code>from sklearn.metrics import mean_squared_error
#Prediction using Random Forest 
y_valid_rf = rf.predict(X_valid)
score = np.sqrt(mean_squared_error(y_valid,y_valid_rf))
print (score)
#Prediction using Neural Network
y_valid_nn = model.predict(X_valid)
score = np.sqrt(mean_squared_error(y_valid,y_valid_nn))
print (score)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09181011997913406, 'fraction_numerical': 0.010085202573465485, 'mean_word_length': 7.3001443001443, 'pattern_counts': {'":': 0, '<': 61, '<?xml version=': 0, '>': 61, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '634244', 'license': 'MIT', 'n_tokens_mistral': 1932, 'n_tokens_neox': 1817, 'n_words': 508, 'path': 'PXL_DIGITAL_JAAR_2/AI & Robotics/Week 11/docker_notebooks/keras/jupyter/NeuralNetworks_Bulldozers.ipynb', 'repo_name': 'Limoentaart/PXL_IT_JAAR_1', 'size': '5751'}