<jupyter_start><jupyter_text>How does a Deep Learning Network Learn  Universal Approximation$$ Z = 2X^2 - 3Y^2 + 5 + \epsilon $$  Create a noisy data<jupyter_code>import numpy as np
import keras
import matplotlib.pyplot as plt
% matplotlib inline
n = 100
start = -10
end = 10
step = (end - start) / n
x = np.arange(start, end, step)
y = np.arange(start, end, step)<jupyter_output><empty_output><jupyter_text>Create this function<jupyter_code>X, Y = np.meshgrid(x,y)
X.shape, Y.shape
C = np.ones((n, n))
e = np.random.rand(n,n)
Z = 2*X*X - 3*Y*Y + 5*C + e<jupyter_output><empty_output><jupyter_text>Plot the function<jupyter_code>from mpl_toolkits.mplot3d import Axes3D
def plot3d(X,Y,Z):
    fig = plt.figure(figsize = (8,8))
    ax = fig.add_subplot(111, projection = '3d')
    ax.plot_surface(X, Y, Z, color = "b")
    plt.show
plot3d(X,Y,Z)<jupyter_output><empty_output><jupyter_text>Create the input and output shape<jupyter_code>input_xy = np.c_[X.reshape(-1), Y.reshape(-1)]
input_xy.shape
output_z = Z.reshape(-1)
output_z.shape
from sklearn.model_selection import train_test_split
input_xy_train, input_xy_test, output_z_train, output_z_test = train_test_split(
    input_xy, output_z, test_size=0.2, random_state=1234)
input_xy_train.shape, input_xy_test.shape, output_z_train.shape, output_z_test.shape<jupyter_output><empty_output><jupyter_text>Create the model and Learn<jupyter_code>from keras.models import Sequential
from keras.layers import Dense
model = Sequential()
model.add(Dense(40, input_dim = 2, activation="relu"))
model.add(Dense(40, activation="relu"))
model.add(Dense(1))
model.compile(loss = "mse", optimizer="adam", metrics=['mse'])
from helpers import MetricHistory
history = MetricHistory()
# Callback to save log files for tensorboard
tbCallBack = keras.callbacks.TensorBoard(log_dir='logs/simple-model')<jupyter_output><empty_output><jupyter_text>After training - run this on the terminal```For Rorodata/opt/rorodata/roro-tensorboard  logs/simple-modelFor localtensorboard --logdir=logs/simple-model```<jupyter_code>%time
history = model.fit(input_xy_train, output_z_train, 
                    batch_size=10, epochs=20, verbose=1, validation_split=0.2,
                   callbacks =[tbCallBack, history])
Z_pred = model.predict(input_xy).reshape(n,n)
plot3d(X,Y,Z_pred)
type(history.history)
len(history.history['loss']), len(history.history['val_loss'])
def plot_metrics(history):
    loss = history.history['loss']
    val_loss = history.history['val_loss']
    
    epochs = range(1, len(loss)+1)
    plt.plot(epochs, loss, 'b', label="Training Loss" )
    plt.plot(epochs, val_loss, 'b', label="Validation Loss" )
    plt.legend()
    plt.show()
plot_metrics(history)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13559941520467836, 'fraction_numerical': 0.01827485380116959, 'mean_word_length': 5.724815724815725, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '474690', 'license': 'MIT', 'n_tokens_mistral': 1056, 'n_tokens_neox': 1024, 'n_words': 199, 'path': 'experiments/Version1/InClass-Intuition.ipynb', 'repo_name': 'SaketJNU/deep-learning', 'size': '2736'}