<jupyter_start><jupyter_text>Linear Regression in TensorFlow  Import Packages<jupyter_code>import tensorflow as tf
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
tf.__version__<jupyter_output><empty_output><jupyter_text>Loading Data & Preprocessing<jupyter_code>%%bigquery flights_df --verbose
SELECT date,
       airline,
       departure_airport,
       arrival_airport,
       departure_lat,
       departure_lon,
       arrival_lat,
       arrival_lon,
       departure_delay,
       arrival_delay
FROM `bigquery-samples.airline_ontime_data.flights`
Where departure_airport = 'SFO' AND date >= '2009-01-01' AND date <= '2009-12-31'
flights_df.shape
flights_df.head()
flights_processed_df = (flights_df
                        .query('1 <= departure_delay <= 120')
                        .dropna()
                        .filter(['departure_delay', 'arrival_delay'])
                       )
(flights_processed_df
 .sample(n=100, random_state = 123)
 .plot.scatter('departure_delay', 'arrival_delay'))<jupyter_output><empty_output><jupyter_text>Let's try to model this using linear regression.  Simple Linear Regression  Training-Testing-Split Split the data into training and testing set. Separate features from labels. Ignore missing values for now.<jupyter_code>train_df = flights_processed_df.sample(frac=0.8,random_state=123)
test_df = flights_processed_df.drop(train_df.index)

train_labels = train_df.pop('arrival_delay')
test_labels = test_df.pop('arrival_delay')<jupyter_output><empty_output><jupyter_text>Building the Model<jupyter_code># Initialize a Sequential model using TF.Keras API:
tf.keras.backend.clear_session()

model = tf.keras.models.Sequential([
    
    # Adds a densely-connected layer with 1 unit to the model:
    tf.keras.layers.Dense(1, activation='linear', input_shape = [len(train_df.keys())])
    
])


model.compile(optimizer=tf.keras.optimizers.RMSprop(),
              loss='mean_squared_error',
              metrics=['mean_absolute_error', 'mean_squared_error'],
             )<jupyter_output><empty_output><jupyter_text>![](linear_node_1.png)<jupyter_code>model.summary()<jupyter_output>Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                (None, 1)                 2         
=================================================================
Total params: 2
Trainable params: 2
Non-trainable params: 0
_________________________________________________________________<jupyter_text>The model has 2 free parameters which is simple regression y = w0*x0 + b. When we compile the model, random weights and no bias will be initialized.<jupyter_code>model.get_weights()<jupyter_output><empty_output><jupyter_text>Testing the model architecture<jupyter_code>example_batch = train_df[:5]
print(example_batch)

model.predict(example_batch)<jupyter_output>departure_delay
84174               2.0
69887              52.0
36578               5.0
132614             23.0
34780               1.0<jupyter_text>The model is predicting something. Currently it doesn't make sense, because the model wasn't fitted to the data. But we'll work on that now!  Training the model<jupyter_code>history = model.fit(train_df, train_labels, 
                    epochs=10,
                    callbacks=[tf.keras.callbacks.TensorBoard("logs/simple-regression")])
model.get_weights()<jupyter_output><empty_output><jupyter_text>See how the model weights were fitted to our data. Let's see how the fitted regression performs:  Performance evaluation of simple regression on test data<jupyter_code>test_predictions = model.predict(test_df).flatten()
results_df = pd.DataFrame({'Predicted': test_predictions, 'Actual': test_labels})
results_df.head(5)
(results_df
 .sample(n=200, random_state = 123)
 .plot.scatter('Actual', 'Predicted')
)<jupyter_output><empty_output><jupyter_text>The model still shows a lot of scatter.<jupyter_code>error = test_predictions - test_labels
plt.hist(error, bins = 25)
plt.xlabel("Prediction Error [Departure_Delay]")
_ = plt.ylabel("Count")<jupyter_output><empty_output><jupyter_text>More concretely, our model seems to be a little bit too optimistic about arrival delays. We are predicting less delay than actually observed.<jupyter_code>test_mae = abs(error).mean()

test_mae<jupyter_output><empty_output><jupyter_text>On average, we are missing the true value by about 9 Minutes.  Multiple Linear Regression Let's try to improve our model by adding more complexity, i.e. features. Here we will add the distance between the origin and destination airport.  Unfortunately the dataset does not contain the distance. We have to calculate it from the coordinates.<jupyter_code>pip install geopy
import geopy.distance
coords_1 = (38.94, -77.46)
coords_2 = (40.77, -73.87)
geopy.distance.distance(coords_1, coords_2).km
# Calculate distance for each route
def coords_to_distance(row):
    coords_start = (row['departure_lat'], row['departure_lon'])
    coors_end = (row['arrival_lat'], row['arrival_lon'])
    distance = geopy.distance.distance(coords_start, coors_end).km
    return(distance)

def get_route_key(row):
    return row['departure_airport']+row['arrival_airport']

distances_df = (flights_df[['departure_airport', 'arrival_airport', 'departure_lon', 'departure_lat', 'arrival_lon', 'arrival_lat']]
                .drop_duplicates()
                .assign(route_key = get_route_key)
               )
distances_df['distance'] = distances_df.apply(coords_to_distance, axis=1)
distances_df = distances_df[['route_key', 'distance']]

# Add distances to flights_processed_df
flights_processed_df = (flights_df
                        .query('1 <= departure_delay <= 120')
                        .assign(route_key = get_route_key)
                        .merge(distances_df)
                        .dropna()
                        .filter(['departure_delay', 'distance', 'arrival_delay'])
                       )
flights_processed_df.head()
train_df = flights_processed_df.sample(frac=0.8,random_state=123)
test_df = flights_processed_df.drop(train_df.index)

train_labels = train_df.pop('arrival_delay')
test_labels = test_df.pop('arrival_delay')
train_df.head()
# Initialize a Sequential model using TF.Keras API:
tf.keras.backend.clear_session()

model = tf.keras.models.Sequential([
    
    # Adds a densely-connected layer with 1 unit to the model:
    tf.keras.layers.Dense(1, activation='linear', input_shape = [len(train_df.keys())])
    
])

model.compile(optimizer=tf.keras.optimizers.RMSprop(),
              loss='mean_squared_error',
              metrics=['mean_absolute_error', 'mean_squared_error'])<jupyter_output><empty_output><jupyter_text>![](linear_node_n.png)<jupyter_code>model.summary()<jupyter_output>Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                (None, 1)                 3         
=================================================================
Total params: 3
Trainable params: 3
Non-trainable params: 0
_________________________________________________________________<jupyter_text>Recognize that the model has 3 Parameters now: 1 bias and 2 weights for each input feature.<jupyter_code>history = model.fit(train_df, train_labels, 
                    epochs=20,
                    callbacks=[tf.keras.callbacks.TensorBoard("logs/multinominal-regression")])
hist = pd.DataFrame(history.history)
hist['epoch'] = history.epoch
hist.tail()<jupyter_output><empty_output><jupyter_text>Performance evaluation of multiple regression on test data<jupyter_code>test_mae = abs(error).mean()

test_mae<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12903632694248235, 'fraction_numerical': 0.017028254288597375, 'mean_word_length': 4.209592641261498, 'pattern_counts': {'":': 0, '<': 63, '<?xml version=': 0, '>': 59, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '390957', 'license': 'MIT', 'n_tokens_mistral': 2517, 'n_tokens_neox': 2345, 'n_words': 513, 'path': '2.5 Lab Linear Regression.ipynb', 'repo_name': 'PacktPublishing/Google-Cloud-Machine-Learning-with-TensorFlow', 'size': '7928'}