<jupyter_start><jupyter_text>Classification Uncertainty Analysis in Bayesian Deep Learning with Dropout Variational Inference Here is [astroNN](https://github.com/henrysky/astroNN), please take a look if you are interested in astronomy or how neural network applied in astronomy* **<NAME>** - *Astronomy student, University of Toronto* - [henrysky](https://github.com/henrysky)* Project adviser: **<NAME>** - *Professor, Department of Astronomy and Astrophysics, University of Toronto* - [jobovy](https://github.com/jobovy)* Contact Henry: henrysky.leung [at] utoronto.ca* This tutorial is created on 16/Mar/2018 with Keras 2.1.5, Tensorflow 1.6.0, Nvidia CuDNN 7.0 for CUDA 9.0 (Optional)* Updated on 31/Jan/2020 with Tensorflow 2.1.0, Tensorflow Probability 0.9.0For more resources on Bayesian Deep Learning with Dropout Variational Inference, please refer to [README.md](https://github.com/henrysky/astroNN/tree/master/demo_tutorial/NN_uncertainty_analysis)  First import everything we need<jupyter_code>%matplotlib inline
%config InlineBackend.figure_format='retina'

from tensorflow.keras.datasets import mnist
from tensorflow.keras import utils
import numpy as np
import pylab as plt

from astroNN.models import MNIST_BCNN

# disable eager execution to prevent potential error
import tensorflow as tf
tf.compat.v1.disable_eager_execution()<jupyter_output>WARNING:tensorflow:From c:\users\henry\documents\github\astronn\astroNN\models\base_master_nn.py:22: The name tf.keras.backend.get_session is deprecated. Please use tf.compat.v1.keras.backend.get_session instead.

WARNING! Gaia environment variable GAIA_TOOLS_DATA not set<jupyter_text>Train the neural network on MNIST training set<jupyter_code>(x_train, y_train), (x_test, y_test) = mnist.load_data()

y_train = utils.to_categorical(y_train, 10)
y_train = y_train.astype(np.float32)
x_train = x_train.astype(np.float32)
x_test = x_test.astype(np.float32)

# Create a astroNN neural network instance and set the basic parameter
net = MNIST_BCNN()
net.task = 'classification'
net.max_epochs = 5  # Just use 5 epochs for quick result

# Trian the nerual network
net.train(x_train, y_train)<jupyter_output>Number of Training Data: 54000, Number of Validation Data: 6000
====Message from Normalizer====
You selected mode: 255
Featurewise Center: False
Datawise Center: False
Featurewise std Center: False
Datawise std Center: False
====Message ends====
====Message from Normalizer====
You selected mode: 0
Featurewise Center: False
Datawise Center: False
Featurewise std Center: False
Datawise std Center: False
====Message ends====
WARNING:tensorflow:From C:\Users\Henry\Miniconda3\lib\site-packages\tensorflow_core\python\ops\resource_variable_ops.py:1635: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
WARNING:tensorflow:From c:\users\henry\documents\github\astronn\astroNN\models\base_bayesian_cnn.py:324: Model.fit_generator (from tensorflow.python.keras.engine.training) is deprecated and will be removed i[...]<jupyter_text>Test the neural network on random MNIST images You can see from below, most test images are right except the last one the model has a high uncertainty in it. As a human, you can indeed can argue this 5 is badly written can can be read as 6 or even a badly written 8.<jupyter_code>test_idx = [1, 2, 3, 4, 5, 8]
pred, pred_std = net.test(x_test[test_idx])
for counter, i in enumerate(test_idx):
    plt.figure(figsize=(3, 3), dpi=100)
    plt.title(f'Predicted Digit {pred[counter]}, Real Answer: {y_test[i]:{1}} \n'
              f'Total Uncertainty (Entropy): {(pred_std["total"][counter]):.{2}}')
    plt.imshow(x_test[i])
    plt.show()
    plt.close('all')
    plt.clf()<jupyter_output>WARNING:tensorflow:From c:\users\henry\documents\github\astronn\astroNN\shared\nn_tools.py:77: is_gpu_available (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.config.list_physical_devices('GPU')` instead.
Starting Dropout Variational Inference
WARNING:tensorflow:From c:\users\henry\documents\github\astronn\astroNN\models\base_bayesian_cnn.py:496: Model.predict_generator (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.
Instructions for updating:
Please use Model.predict, which supports generators.
Completed Dropout Variational Inference with 100 forward passes, 0.40s elapsed<jupyter_text>Test the neural network on random MNIST images with 90 degree rotation Since the neural network is trained on MNIST images without any data argumentation, so if we rotate the MNIST images, the images should look 'alien' to the neural network and the neural network should give us a high unceratinty. And indeed the neural network tells us its very uncertain about the prediction with roated images.<jupyter_code>test_rot_idx = [9, 10, 11]

test_rot = x_test[test_rot_idx]

for counter, j in enumerate(test_rot):
    test_rot[counter] = np.rot90(j)

pred_rot, pred_rot_std = net.test(test_rot)

for counter, i in enumerate(test_rot_idx):
    plt.figure(figsize=(3, 3), dpi=100)
    plt.title(f'Predicted Digit {pred_rot[counter]}, Real Answer: {y_test[i]:{1}} \n'
              f'Total Uncertainty (Entropy): {(pred_rot_std["total"][counter]):.{2}}')
    plt.imshow(test_rot[counter])
    plt.show()
    plt.close('all')
    plt.clf()<jupyter_output>Starting Dropout Variational Inference
Completed Dropout Variational Inference with 100 forward passes, 0.30s elapsed
{'fraction_non_alphanumeric': 0.09710962284102925, 'fraction_numerical': 0.01973916108565386, 'mean_word_length': 5.684334511189635, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 15, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '630276', 'license': 'MIT', 'n_tokens_mistral': 1875, 'n_tokens_neox': 1744, 'n_words': 561, 'path': 'demo_tutorial/NN_uncertainty_analysis/Uncertainty_Demo_MNIST.ipynb', 'repo_name': 'AbdulfattahBaalawi/astroNN', 'size': '5680'}