<jupyter_start><jupyter_text>- In regression problem, the aim is to predict the output of a continuous value, like price or pobability<jupyter_code>import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns
import tensorflow as tf

from tensorflow import keras
from tensorflow.keras import layers

print(tf.__version__)<jupyter_output>2.7.0<jupyter_text>- import dataset<jupyter_code>url = 'http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data'   #Import data from url
column_names = ['MPG', 'Cylinders', 'Displacement', 'Horsepower', 'Weight',           #For dataframe, give column names 
                'Acceleration', 'Model Year', 'Origin']

raw_dataset = pd.read_csv(url, names=column_names,                   #Convert csv to pd with specific requirements
                          na_values='?', comment='\t',
                          sep=' ', skipinitialspace=True)                        
dataset = raw_dataset.copy()
dataset.head()<jupyter_output><empty_output><jupyter_text>- clean data (Checking for NA)<jupyter_code>dataset.isna().sum() #Check if any na or nan value is present in the df
#Drop rows with na
dataset = dataset.dropna()<jupyter_output><empty_output><jupyter_text>- Origin column is a categorical value, not numeric. One-hot encoding with pd.get_dummies<jupyter_code>dataset['Origin'] = dataset['Origin'].map({1: 'USA', 2: 'Europe', 3: 'Japan'})
dataset.head()<jupyter_output><empty_output><jupyter_text>- Seperating Origin column by value into column for each<jupyter_code>dataset = pd.get_dummies(dataset, columns=['Origin'], prefix='', prefix_sep='')
dataset.head()<jupyter_output><empty_output><jupyter_text>Split data into Train / Test sets<jupyter_code>train_dataset = dataset.sample(frac=0.8, random_state=0)               #Seperate into 80/20
test_dataset = dataset.drop(train_dataset.index)<jupyter_output><empty_output><jupyter_text>- Using seaborn to inspect Train Dataset<jupyter_code>sns.pairplot(train_dataset[['MPG', 'Cylinders', 'Displacement', 'Weight']], diag_kind='kde')<jupyter_output><empty_output><jupyter_text>- Overall statistics of the training Dataset<jupyter_code>train_dataset.describe().transpose()<jupyter_output><empty_output><jupyter_text>- Seperate into Feature and labels<jupyter_code>train_features = train_dataset.copy()
test_features = test_dataset.copy()

train_labels = train_features.pop('MPG')
test_labels = test_features.pop('MPG')<jupyter_output><empty_output><jupyter_text>Normalization- Initially check range of features to check how they vary<jupyter_code>train_dataset.describe().transpose()[['mean','std']]<jupyter_output><empty_output><jupyter_text>- One important reason normalization is important is because the features are multiplied by the model weights. Therefore, the scale of the outputs and scale of gradients is affected by scale of the inputs.- A model 'might' converge without feature normalization, but it makes it more stable.  Normalization layer- A Preprocessing layer which normalizes continuous features- This layer will shift and scale inputs into a distribution centered around 0 with standard deviation 1.- Calculates by precomputing the mean and variance of data, and calling *(input - mean) / sqrt(var)*- The mean and variance values for the layer must be either supplied on construction or learned via **adapt()** adapt()- adapt() will compute the mean and variance of the data and store them as the layer's weights.- Should be called before fit(), evaluate() or predict()- Calling adapt() on a Normalization layer is an alternative to passing in 'mean' and 'variance' arguments during layer construction.- A normalization layer should always be adapted over a dataset or pass 'mean' and 'variance' manually.- During adapt(), the layer will compute 'mean' and 'variance' seperately for each position in each axis specified by 'axis' argument.- if 'axis' == None, a single 'mean' and 'variance' will be calculated.<jupyter_code>#First, create the layer
normalizer = tf.keras.layers.Normalization(axis=-1)

#Then, fit the state of the preprocessing layer to the data by calling
normalizer.adapt(np.array(train_features))
normalizer.mean.numpy()<jupyter_output><empty_output><jupyter_text>- When the layer is called, it returns the input data -> with each feature independently normalized<jupyter_code>#Trial
first = np.array(train_features[:1])

with np.printoptions(precision=2, suppress=True):
    print('First example:', first)
    print()
    print('Normalized', normalizer(first).numpy())<jupyter_output>First example: [[   4.    90.    75.  2125.    14.5   74.     0.     0.     1. ]]

Normalized [[-0.87 -1.01 -0.79 -1.03 -0.38 -0.52 -0.47 -0.5   0.78]]<jupyter_text>Linear Regression  Linear regression with one variable<jupyter_code>#Comparing 'MPG' and 'HorsePower'

#Required two steps: Normalize input feature and apply linear transormation

#Taking all Horsepower features
horsepower = np.array(train_features['Horsepower'])

#Normalize data
horsepower_normalizer = layers.Normalization(input_shape=[1,], axis=None)
horsepower_normalizer.adapt(horsepower)<jupyter_output><empty_output><jupyter_text>- Apply a linear transformation (y=mx+b) to produce 1 output using a linear layer (tf.keras.layers.Dense)<jupyter_code>#Build the keras Sequential model

horsepower_model = tf.keras.Sequential([
    horsepower_normalizer,
    layers.Dense(units=1)
])

horsepower_model.summary()
#Compile Model
horsepower_model.compile(
    optimizer = tf.optimizers.Adam(learning_rate=0.1),
    loss = 'mean_absolute_error'
)
history = horsepower_model.fit(
    train_features['Horsepower'],
    train_labels,
    epochs=100,
    verbose=2,
    validation_split = 0.2)
hist = pd.DataFrame(history.history)
hist['epoch'] = history.epoch
hist.tail()
#Plot history loss
def plot_loss(history):
  plt.plot(history.history['loss'], label='loss')
  plt.plot(history.history['val_loss'], label='val_loss')
  plt.ylim([0, 10])
  plt.xlabel('Epoch')
  plt.ylabel('Error [MPG]')
  plt.legend()
  plt.grid(True)

plot_loss(history)

# Collect the results on the test set
test_results = {}

test_results['horsepower_model'] = horsepower_model.evaluate(
    test_features['Horsepower'],
    test_labels, verbose=2
)
x = tf.linspace(0.0, 250, 251)
y = horsepower_model.predict(x)
def plot_horsepower(x,y):
    plt.scatter(train_features['Horsepower'], train_labels, label='Data')
    plt.plot(x, y, color='k', label='Predictions')
    plt.xlabel('Horsepower')
    plt.ylabel('MPG')
    plt.legend()

plot_horsepower(x,y)<jupyter_output><empty_output><jupyter_text>Regression with a deep neural network- Single-input DNN- Multiple-input DNN- Layers to consider:- Normalization layers (horsepower_normalizer)- Two hidden, non-linear, Dense layer with ReLU activation- a linear *Dense* single-output layer<jupyter_code>def build_and_compile_model(norm):
    model = keras.Sequential([
        norm,
        layers.Dense(64, activation='relu'),
        layers.Dense(64, activation='relu'),
        layers.Dense(1)
    ])
    model.compile(loss='mean_absolute_error',
            optimizer=tf.keras.optimizers.Adam(0.001)
    )
    return model<jupyter_output><empty_output><jupyter_text>Regression using a DNN and a single input- 'Horsepower' as input- horsepower_normalizer as normalization layer<jupyter_code>dnn_horsepower_model = build_and_compile_model(horsepower_normalizer)
dnn_horsepower_model.summary()
history = dnn_horsepower_model.fit(
    train_features['Horsepower'],
    train_labels,
    validation_split=0.2,
    verbose=1,epochs=100
)
plot_loss(history)
x = tf.linspace(0.0, 250, 251)
y = dnn_horsepower_model.predict(x)

plot_horsepower(x,y)
# Collec the results on the test set for later
test_results['dnn_horsepower_model'] = dnn_horsepower_model.evaluate(
    test_features['Horsepower'], test_labels,
    verbose=0)<jupyter_output><empty_output><jupyter_text>Regression using a DNN and multiple inputs<jupyter_code>dnn_model = build_and_compile_model(normalizer)
dnn_model.summary()<jupyter_output>Model: "sequential_2"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 normalization (Normalizatio  (None, 9)                19        
 n)                                                              
                                                                 
 dense_4 (Dense)             (None, 64)                640       
                                                                 
 dense_5 (Dense)             (None, 64)                4160      
                                                                 
 dense_6 (Dense)             (None, 1)                 65        
                                                                 
=================================================================
Total params: 4,884
Trainable params: 4,865
Non-trainable params: 19
___________________________________________________[...]<jupyter_text>Fit model<jupyter_code>history = dnn_model.fit(
    train_features,
    train_labels,
    validation_split=0.2,
    verbose=1, epochs=100
)
plot_loss(history)
# Collect results on the test set
test_results['dnn_model'] = dnn_model.evaluate(test_features, test_labels, verbose=0)<jupyter_output><empty_output><jupyter_text>Performance<jupyter_code>#Since all models are now trained, lets review the performance
pd.DataFrame(test_results, index=['Mean absolute error [MPG]']).T<jupyter_output><empty_output><jupyter_text>Make prediction<jupyter_code>test_predictions = dnn_model.predict(test_features).flatten()

a = plt.axes(aspect='equal')
plt.scatter(test_labels, test_predictions)
plt.xlabel('True Values [MPG]')
plt.ylabel('Predictions [MPG]')
lims = [0, 50]
plt.xlim(lims)
plt.ylim(lims)
_ = plt.plot(lims, lims)<jupyter_output><empty_output><jupyter_text>Error Distribution<jupyter_code>error = test_predictions - test_labels
plt.hist(error, bins=25)
plt.xlabel('Prediction Error [MPG]')
_ = plt.ylabel('Count')<jupyter_output><empty_output><jupyter_text>Save and load model<jupyter_code>dnn_model.save('dnn_model')
reloaded = tf.keras.models.load_model('dnn_model')

test_results['reloaded'] = reloaded.evaluate(
    test_features, test_labels, verbose=0)<jupyter_output>INFO:tensorflow:Assets written to: dnn_model\assets
{'fraction_non_alphanumeric': 0.11917347429120614, 'fraction_numerical': 0.014608361364728496, 'mean_word_length': 4.366683857658587, 'pattern_counts': {'":': 0, '<': 85, '<?xml version=': 0, '>': 86, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '22', 'dir': 'jupyter-structured-clean-dedup', 'id': '96921', 'license': 'MIT', 'n_tokens_mistral': 3339, 'n_tokens_neox': 3151, 'n_words': 779, 'path': '3_Basic_Regression_Fuel_Prices.ipynb', 'repo_name': 'ASBattu/TensorFlow_Tutorials', 'size': '10405'}