<jupyter_start><jupyter_text>04. Why Neural Networks Deeply Learn a Mathematical Formula? - Book + Private Lessons [Here ↗](https://sotastica.com/reservar)- Subscribe to my [Blog ↗](https://blog.pythonassembly.com/)- Let's keep in touch on [LinkedIn ↗](www.linkedin.com/in/jsulopz) 😄  Machine Learning, what does it mean? > - The Machine Learns...>> But, **what does it learn?**<jupyter_code>%%HTML
<blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">Machine Learning, what does it mean? ⏯<br><br>· The machine learns...<br><br>Ha ha, not funny! 🤨 What does it learn?<br><br>· A mathematical equation. For example: <a href="https://t.co/sjtq9F2pq7">pic.twitter.com/sjtq9F2pq7</a></p>&mdash; <NAME> (@sotastica) <a href="https://twitter.com/sotastica/status/1449735653328031745?ref_src=twsrc%5Etfw">October 17, 2021</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script><jupyter_output><empty_output><jupyter_text>How does the Machine Learn?  In a Linear Regression<jupyter_code>%%HTML
<iframe width="560" height="315" src="https://www.youtube.com/embed/Ht3rYS-JilE" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe><jupyter_output><empty_output><jupyter_text>In a Neural Network<jupyter_code>%%HTML
<iframe width="560" height="315" src="https://www.youtube.com/embed/IHZwWFHWa-w?start=329" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe><jupyter_output><empty_output><jupyter_text>A Practical Example → [Tesla Autopilot](https://www.tesla.com/AI) An Example where It Fails → [Tesla Confuses Moon with Semaphore](https://twitter.com/Carnage4Life/status/1418920100086784000?s=20)  Load the Data > - Simply execute the following lines of code to load the data.> - This dataset contains **statistics about Car Accidents** (columns)> - In each one of **USA States** (rows) https://www.kaggle.com/fivethirtyeight/fivethirtyeight-bad-drivers-dataset/<jupyter_code>import seaborn as sns

df = sns.load_dataset(name='car_crashes', index_col='abbrev')
df.sample(5)<jupyter_output><empty_output><jupyter_text>Neural Network Concepts in Python  Initializing the `Weights` > - https://keras.io/api/layers/initializers/  How to `kernel_initializer` the weights? $$accidents = speeding \cdot w_1 + alcohol \cdot w_2 \ + ... + \ ins\_losses \cdot w_7$$<jupyter_code>from tensorflow.keras import Sequential, Input
from tensorflow.keras.layers import Dense
df.shape
model = Sequential()
model.add(layer=Input(shape=(6,)))
model.add(layer=Dense(units=3, kernel_initializer='zeros'))
model.add(layer=Dense(units=1))<jupyter_output>Metal device set to: Apple M1<jupyter_text>Make a Prediction with the Neural Network > - Can we make a prediction for for `Washington DC` accidents> - With the already initialized Mathematical Equation?<jupyter_code>X = df.drop(columns='total')
y = df.total
AL = X[:1]
AL<jupyter_output><empty_output><jupyter_text>Observe the numbers for the `weights`<jupyter_code>model.get_weights()<jupyter_output><empty_output><jupyter_text>Predictions vs Reality > 1. Calculate the Predicted Accidents and> 2. Compare it with the Real Total Accidents<jupyter_code>model.predict(x=AL)<jupyter_output>2022-01-05 09:59:02.342564: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
2022-01-05 09:59:02.343180: W tensorflow/core/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz
2022-01-05 09:59:02.375183: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.<jupyter_text>`fit()` the `model` and compare again<jupyter_code>model.compile(loss='mse', metrics=['mse'])
model.fit(X, y, epochs=500, verbose=1)<jupyter_output>Epoch 1/500<jupyter_text>Observe the numbers for the `weights`<jupyter_code>model.get_weights()<jupyter_output><empty_output><jupyter_text>Predictions vs Reality > 1. Calculate the Predicted Accidents and> 2. Compare it with the Real Total Accidents<jupyter_code>y_pred = model.predict(X)
dfsel = df[['total']].copy()
dfsel['pred_zeros_after_fit'] = y_pred
dfsel.head()
mse = ((dfsel.total - dfsel.pred_zeros_after_fit)**2).mean()
mse<jupyter_output><empty_output><jupyter_text>How to `kernel_initializer` the weights to 1?<jupyter_code>dfsel['pred_ones_after_fit'] = y_pred
dfsel.head()
mse = ((dfsel.total - dfsel.pred_ones_after_fit)**2).mean()
mse<jupyter_output><empty_output><jupyter_text>How to `kernel_initializer` the weights to `glorot_uniform` (default)?  Play with the Activation Function > - https://keras.io/api/layers/activations/<jupyter_code>%%HTML
<iframe width="560" height="315" src="https://www.youtube.com/embed/IHZwWFHWa-w?start=558" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe><jupyter_output><empty_output><jupyter_text>Use `sigmoid` activation in last layer<jupyter_code>model = Sequential()
model.add(layer=Input(shape=(6,)))
model.add(layer=Dense(units=3, kernel_initializer='glorot_uniform'))
model.add(layer=Dense(units=1, activation='sigmoid'))
model.compile(loss='mse', metrics=['mse'])<jupyter_output><empty_output><jupyter_text>`fit()` the Model<jupyter_code>model.fit(X, y, epochs=500, verbose=0)<jupyter_output><empty_output><jupyter_text>Predictions vs Reality > 1. Calculate the Predicted Accidents and> 2. Compare it with the Real Total Accidents<jupyter_code>y_pred = model.predict(X)
dfsel['pred_sigmoid'] = y_pred
dfsel.head()
mse = ((dfsel.total - dfsel.pred_sigmoid)**2).mean()
mse<jupyter_output><empty_output><jupyter_text>Observe the numbers for the `weights`> - Have they changed?<jupyter_code>model.get_weights()<jupyter_output><empty_output><jupyter_text>Use `linear` activation in last layer  Use `tanh` activation in last layer  Use `relu` activation in last layer  How are the predictions changing? Why?  Optimizer > - https://keras.io/api/optimizers/available-optimizers Optimizers comparison in GIF → https://mlfromscratch.com/optimizers-explained/adam Tesla's Neural Network Models is composed of 48 models trainned in 70.000 hours of GPU → https://tesla.com/ai 1 Year with a 8 GPU Computer → https://twitter.com/thirdrowtesla/status/1252723358342377472  Use Gradient Descent `SGD`<jupyter_code>model = Sequential()
model.add(layer=Input(shape=(6,)))
model.add(layer=Dense(units=3, kernel_initializer='glorot_uniform'))
model.add(layer=Dense(units=1, activation='sigmoid'))<jupyter_output><empty_output><jupyter_text>`compile()` the model<jupyter_code>model.compile(optimizer='sgd', loss='mse', metrics=['mse'])<jupyter_output><empty_output><jupyter_text>`fit()` the Model<jupyter_code>history = model.fit(X, y, epochs=500, verbose=0)<jupyter_output><empty_output><jupyter_text>Predictions vs Reality > 1. Calculate the Predicted Accidents and> 2. Compare it with the Real Total Accidents<jupyter_code>y_pred = model.predict(X)
dfsel['pred_gsd'] = y_pred
dfsel.head()
mse = ((dfsel.total - dfsel.pred_sgd)**2).mean()
mse<jupyter_output><empty_output><jupyter_text>Observe the numbers for the `weights`> - Have they changed?<jupyter_code>model.get_weights()<jupyter_output><empty_output><jupyter_text>View History<jupyter_code>import matplotlib.pyplot as plt
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'val'], loc='upper left')
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1415495681320098, 'fraction_numerical': 0.029392806497357225, 'mean_word_length': 7.766101694915254, 'pattern_counts': {'":': 0, '<': 113, '<?xml version=': 0, '>': 134, 'https://': 17, 'lorem ipsum': 0, 'www.': 6, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '23', 'dir': 'jupyter-structured-clean-dedup', 'id': '7546', 'license': 'MIT', 'n_tokens_mistral': 2931, 'n_tokens_neox': 2714, 'n_words': 557, 'path': 'II Machine Learning & Deep Learning/01_Why Neural Networks Deeply Learn a Mathematical Formula/04session.ipynb', 'repo_name': 'algoriv92/machine-learning-program', 'size': '7762'}