<jupyter_start><jupyter_text>A regression problem aims to predict a continuous numeric value like price or probability. Unlike a classification problem, the model is trained to predict a class from a list of classes. A classic class problem is like predicting the class of a fruit image.Reference:* https://www.tensorflow.org/tutorials/keras/regression<jupyter_code>!pip install -q tensorflow-gpu==2.0.0
!pip install -q git+https://github.com/tensorflow/docs  # use some functions from tensorflow docs 
!pip install -q seaborn  # for pairplot
import pathlib
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
import seaborn as sns
import tensorflow as tf
import tensorflow_docs as tfdocs
import tensorflow_docs.plots
import tensorflow_docs.modeling

print("Tensorflow Version: {}".format(tf.__version__))
print("Eager model: {}".format(tf.executing_eagerly()))
print("GPU {} available".format("is" if tf.config.experimental.list_physical_devices('GPU') else "not"))<jupyter_output>Tensorflow Version: 2.0.0
Eager model: True
GPU is available<jupyter_text>Dataset PreparationHere you are going to use Auto MPG Dataset from [UCI Machine Learning Repository](https://archive.ics.uci.edu/ml/).  Download and Import the Dataset<jupyter_code>dataset_path = tf.keras.utils.get_file("auto-mpg.data", 
                                       "http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data")
dataset_path
col_names = ['MPG','Cylinders','Displacement','Horsepower','Weight',
             'Acceleration', 'Model Year', 'Origin']
raw_dataset = pd.read_csv(dataset_path, names=col_names, 
                          na_values="?", comment="\t", sep=" ", 
                          skipinitialspace=True)
dataset = raw_dataset.copy()
dataset.head(5)<jupyter_output><empty_output><jupyter_text>Clean the Dataset The dataset still contains several unknown values. To keep the dataset simple, we delete the row containing unknown values.<jupyter_code>dataset.isna().sum()
dataset = dataset.dropna()<jupyter_output><empty_output><jupyter_text>The column `Origin` is already a categorical value. We change the value to its categorical text.<jupyter_code>dataset["Origin"] = dataset["Origin"].map(lambda x: {1: "USA", 2: "EU", 3: "Japan"}.get(x))<jupyter_output><empty_output><jupyter_text>Use function `get_dummies()` to create one-hot encoding like columns.<jupyter_code>dataset = pd.get_dummies(dataset, prefix='', prefix_sep='')
dataset.tail(5)<jupyter_output><empty_output><jupyter_text>Split the data into sub-datasets<jupyter_code>train_dataset = dataset.sample(frac=0.8,random_state=0)
test_dataset = dataset.drop(train_dataset.index)<jupyter_output><empty_output><jupyter_text>Inspect the Dataset<jupyter_code>sns.pairplot(train_dataset[["MPG", "Cylinders", "Displacement", "Weight"]], diag_kind="kde")
train_stats = train_dataset.describe()
train_stats.pop("MPG")
train_stats = train_stats.transpose()
train_stats<jupyter_output><empty_output><jupyter_text>Split features from labels Separate the feature which you are going to train a model to predict.<jupyter_code>train_labels = train_dataset.pop('MPG')
test_labels = test_dataset.pop('MPG')
def norm(x):
  return (x - train_stats['mean']) / train_stats['std']
normed_train_data = norm(train_dataset)
normed_test_data = norm(test_dataset)<jupyter_output><empty_output><jupyter_text>Building the Model<jupyter_code>def build_model(inputs):
  x = tf.keras.layers.Dense(units=64, activation='relu', 
                            input_shape=[len(train_dataset.keys())])(inputs)
  x = tf.keras.layers.Dense(units=64, activation='relu')(x)
  x = tf.keras.layers.Dense(units=1)(x)
  return x
inputs = tf.keras.Input(shape=[len(train_dataset.keys())])
outputs = build_model(inputs)
model = tf.keras.Model(inputs, outputs)
model.summary()<jupyter_output>Model: "model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_2 (InputLayer)         [(None, 9)]               0         
_________________________________________________________________
dense_3 (Dense)              (None, 64)                640       
_________________________________________________________________
dense_4 (Dense)              (None, 64)                4160      
_________________________________________________________________
dense_5 (Dense)              (None, 1)                 65        
=================================================================
Total params: 4,865
Trainable params: 4,865
Non-trainable params: 0
_________________________________________________________________<jupyter_text>An Inference Example<jupyter_code>example_batch = normed_train_data[:10]
example_batch
example_result = model.predict(example_batch)
example_result<jupyter_output><empty_output><jupyter_text>Training the Model<jupyter_code>model.compile(optimizer=tf.keras.optimizers.RMSprop(1e-3), 
              loss='mse', 
              metrics=["mae", "mse"])
EPOCHS = 1000

his = model.fit(normed_train_data, train_labels, epochs=EPOCHS, 
                validation_split=0.2, verbose=0,
                callbacks=[tfdocs.modeling.EpochDots()])<jupyter_output>WARNING:tensorflow:Falling back from v2 loop because of error: Failed to find data adapter that can handle input: <class 'pandas.core.frame.DataFrame'>, <class 'NoneType'>

Epoch: 0, loss:560.5284,  mae:22.4126,  mse:560.5284,  val_loss:549.2268,  val_mae:22.1249,  val_mse:549.2268,  
....................................................................................................
Epoch: 100, loss:6.0259,  mae:1.6846,  mse:6.0259,  val_loss:9.3346,  val_mae:2.2935,  val_mse:9.3346,  
....................................................................................................
Epoch: 200, loss:5.2588,  mae:1.5826,  mse:5.2588,  val_loss:9.1171,  val_mae:2.3280,  val_mse:9.1171,  
....................................................................................................
Epoch: 300, loss:4.6884,  mae:1.4774,  mse:4.6884,  val_loss:8.9965,  val_mae:2.1682,  val_mse:8.9965,  
................................................................................................[...]<jupyter_text>View the training details via Pandas<jupyter_code>hist = pd.DataFrame(his.history)
hist["epoch"] = his.epoch
hist.tail(5)
plotter = tfdocs.plots.HistoryPlotter(smoothing_std=2)
plotter.plot({'Basic': his}, metric="mae")
plt.ylim([0, 10])
plt.ylabel('MAE [MPG]')
plotter.plot({'Basic': his}, metric="mse")
plt.ylim([0, 20])
plt.ylabel('MSE [MPG^2]')<jupyter_output><empty_output><jupyter_text>Introducing EarlyStopping<jupyter_code>model = tf.keras.Model(inputs, outputs)
early_stopping = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=30)
model.compile(optimizer=tf.keras.optimizers.RMSprop(1e-3), 
              loss='mse', 
              metrics=["mae", "mse"])
early_his = model.fit(normed_train_data, train_labels, epochs=EPOCHS, 
                      validation_split=0.2, verbose=0, 
                      callbacks=[early_stopping, tfdocs.modeling.EpochDots()])<jupyter_output>WARNING:tensorflow:Falling back from v2 loop because of error: Failed to find data adapter that can handle input: <class 'pandas.core.frame.DataFrame'>, <class 'NoneType'>

Epoch: 0, loss:2.6850,  mae:1.0756,  mse:2.6850,  val_loss:9.9675,  val_mae:2.2819,  val_mse:9.9675,  
................................<jupyter_text>View Training details with the early stopping<jupyter_code>plotter.plot({'Early_Stopping': early_his}, metric='mae')
plt.ylim([0,10])
plt.ylabel("MAE [MPG]")
loss, mae, mse = model.evaluate(normed_test_data, test_labels, verbose=2)<jupyter_output>WARNING:tensorflow:Falling back from v2 loop because of error: Failed to find data adapter that can handle input: <class 'pandas.core.frame.DataFrame'>, <class 'NoneType'>
78/78 - 0s - loss: 7.4749 - mae: 2.0986 - mse: 7.4749<jupyter_text>Make Predictions Finally you can use the model to predict the MPG value.<jupyter_code>test_predictions = model.predict(normed_test_data).flatten()
a = plt.axes(aspect='equal')
plt.scatter(test_labels, test_predictions)
plt.xlabel('True Values [MPG]')
plt.ylabel('Predictions [MPG]')
lims = [0, 50]
plt.xlim(lims)
plt.ylim(lims)
plt.plot(lims, lims)<jupyter_output><empty_output><jupyter_text>Let's take a look at the error distribution.<jupyter_code>error = test_predictions - test_labels
plt.hist(error, bins=25)
plt.xlabel('Prediction Error [MPG]')
plt.ylabel('Count')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.18191328606300128, 'fraction_numerical': 0.03231430896199, 'mean_word_length': 3.9250143102461363, 'pattern_counts': {'":': 0, '<': 66, '<?xml version=': 0, '>': 66, 'https://': 3, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '223753', 'license': 'MIT', 'n_tokens_mistral': 2952, 'n_tokens_neox': 2728, 'n_words': 551, 'path': 'machine_learning/classification_regression/TF2_Regression.ipynb', 'repo_name': 'jiankaiwang/sophia.ml', 'size': '8603'}