<jupyter_start><jupyter_text>Importing libraries<jupyter_code>import warnings
warnings.filterwarnings('ignore')

import numpy as np
import matplotlib.pyplot as plt
import keras
from keras.datasets import mnist
from keras.models import Sequential,model_from_json
from keras.layers import Dense
from keras.optimizers import RMSprop
import pylab as plt<jupyter_output><empty_output><jupyter_text>Keras is the deep learning library that helps you to code Deep Neural Networks with fewer lines of code   Import data<jupyter_code>batch_size = 128
num_classes = 10
epochs = 2

# the data, split between train and test sets
(x_train, y_train), (x_test, y_test) = mnist.load_data()

x_train = x_train.reshape(60000, 784)
x_test = x_test.reshape(10000, 784)
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')

# Normalize to 0 to 1 range
x_train /= 255
x_test /= 255

print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')
# convert class vectors to binary class matrices
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)<jupyter_output>Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
11493376/11490434 [==============================] - 0s 0us/step
60000 train samples
10000 test samples<jupyter_text>Part 1: Analysis of Activation Functions
  Try other Activation FunctionsAnalyze the accuracy and training time Example Syntax * model.add(Dense(8, activation='tanh'))* model.add(Dense(8, activation='linear'))* model.add(Dense(8, activation='relu'))* model.add(Dense(8, activation='hard_sigmoid'))  Sigmoid Activation Function<jupyter_code>model = Sequential()
model.add(Dense(64, activation='sigmoid', input_shape=(784,)))
model.add(Dense(64, activation='sigmoid'))
model.add(Dense(num_classes, activation='softmax'))

model.summary()
model.compile(loss='categorical_crossentropy',
              optimizer=RMSprop(),
              metrics=['accuracy'])

history = model.fit(x_train, y_train,
                    batch_size=batch_size,
                    epochs=10,
                    verbose=1,
                    validation_data=(x_test, y_test))
score = model.evaluate(x_test, y_test, verbose=0)
print('Test loss:', score[0])
print('Test accuracy:', score[1])<jupyter_output>Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                (None, 64)                50240     
_________________________________________________________________
dense_1 (Dense)              (None, 64)                4160      
_________________________________________________________________
dense_2 (Dense)              (None, 10)                650       
=================================================================
Total params: 55,050
Trainable params: 55,050
Non-trainable params: 0
_________________________________________________________________
Epoch 1/10
469/469 [==============================] - 4s 4ms/step - loss: 1.3583 - accuracy: 0.7003 - val_loss: 0.3443 - val_accuracy: 0.9065
Epoch 2/10
469/469 [==============================] - 1s 3ms/step - loss: 0.3092 - accuracy: 0.9146 - val_lo[...]<jupyter_text>Relu Activation Function<jupyter_code># Write your code here. Use the same architecture as above.<jupyter_output><empty_output><jupyter_text>Write your findings about activations here?

1.   Finding 1
2.   Finding 2


  Part 2: Design Choices: Nodes
 Analyze the parameter size, accuracy and training time  Design a model with small Number of Nodes. For Example 8<jupyter_code>first_layer_size = 8

model = Sequential()
model.add(Dense(first_layer_size, activation='sigmoid', input_shape=(784,)))
model.add(Dense(32, activation='sigmoid'))
model.add(Dense(num_classes, activation='softmax'))

model.summary()
model.compile(loss='categorical_crossentropy',
              optimizer=RMSprop(),
              metrics=['accuracy'])

history = model.fit(x_train, y_train,
                    batch_size=batch_size,
                    epochs=epochs,
                    verbose=1,
                    validation_data=(x_test, y_test))
score = model.evaluate(x_test, y_test, verbose=0)
print('Test loss:', score[0])
print('Test accuracy:', score[1])

w = []
for layer in model.layers:
    weights = layer.get_weights()
    w.append(weights)

layer1 = np.array(w[0][0])
print("Shape of First Layer",layer1.shape)
print("Visualization of First Layer")

import matplotlib.pyplot as plt
fig=plt.figure(figsize=(16, 16))
columns = 8
rows = int(first_layer_size/8)
for i in range(1, columns*rows +1):
    fig.add_subplot(rows, columns, i)
    plt.imshow(layer1[:,i-1].reshape(28,28),cmap='gray')
plt.show()<jupyter_output>_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_11 (Dense)             (None, 8)                 6280      
_________________________________________________________________
dense_12 (Dense)             (None, 32)                288       
_________________________________________________________________
dense_13 (Dense)             (None, 10)                330       
=================================================================
Total params: 6,898
Trainable params: 6,898
Non-trainable params: 0
_________________________________________________________________
Train on 60000 samples, validate on 10000 samples
Epoch 1/2
60000/60000 [==============================] - 15s - loss: 1.8398 - acc: 0.5610 - val_loss: 1.3080 - val_acc: 0.7207
Epoch 2/2
60000/60000 [==============================] - 11s - loss: 0.9784 - acc: 0.7753 - val[...]<jupyter_text>Design a model with large Number of Nodes. For example 128<jupyter_code># Write your code here 

# Use the same layer design from the above cell. Play with the parameter "first_layer_size" above. 
# It is preferred to have first_layer_size as multiplication of 8 (for visualizing the weights properly)<jupyter_output><empty_output><jupyter_text>Write your findings about influence of nodes here?

1.   Finding 1
2.   Finding 2


  Part 3: Design Choices: Layers
 Analyze the parameter size, accuracy and training time  Design a model with small number of layers. For example 1 hidden layer<jupyter_code>model = Sequential()
model.add(Dense(32, activation='relu', input_shape=(784,)))
model.add(Dense(num_classes, activation='softmax'))

model.summary()
model.compile(loss='categorical_crossentropy',
              optimizer=RMSprop(),
              metrics=['accuracy'])

history = model.fit(x_train, y_train,
                    batch_size=batch_size,
                    epochs=10,
                    verbose=1,
                    validation_data=(x_test, y_test))
score = model.evaluate(x_test, y_test, verbose=0)
print('Test loss:', score[0])
print('Test accuracy:', score[1])<jupyter_output>Model: "sequential_2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_5 (Dense)              (None, 32)                25120     
_________________________________________________________________
dense_6 (Dense)              (None, 10)                330       
=================================================================
Total params: 25,450
Trainable params: 25,450
Non-trainable params: 0
_________________________________________________________________
Epoch 1/10
469/469 [==============================] - 2s 3ms/step - loss: 0.7565 - accuracy: 0.7996 - val_loss: 0.2641 - val_accuracy: 0.9260
Epoch 2/10
469/469 [==============================] - 1s 3ms/step - loss: 0.2532 - accuracy: 0.9283 - val_loss: 0.2173 - val_accuracy: 0.9372
Epoch 3/10
469/469 [==============================] - 1s 3ms/step - loss: 0.2057 - accuracy: 0.9[...]<jupyter_text>Lower number of Layers. For example 4 hidden layers<jupyter_code># Write your code here 

# Use the same number of units from the above cell<jupyter_output><empty_output><jupyter_text>Write your findings about influence of layers here?

1.   Finding 1
2.   Finding 2<jupyter_code><jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.16795619569098916, 'fraction_numerical': 0.05106534936317105, 'mean_word_length': 4.12942612942613, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '466385', 'license': 'MIT', 'n_tokens_mistral': 2723, 'n_tokens_neox': 2366, 'n_words': 615, 'path': 'Homework/To Do/Homework 02.ipynb', 'repo_name': 'MinaamAw/GlobalAIHubDLCourse', 'size': '8401'}