<jupyter_start><jupyter_text>Previous: 1.5 Overfitting VS generalization   Keras  1.6 Increasing/Decreasing number of layers  ExplanationThere are three types of layers:- input layer- hidden layer- output layerOne hidden layer is sufficient for the large majority of problems.There is no "magic" rule to calculate the number of hidden layers and nodes of Neural Network, but there are some tips and recommendations.The number of hidden nodes is based on a relationship between:- Number of input and output nodes- Amount of training data available- Complexity of the function that is trying to be learned- The training algorithm- To minimize the error and have a trained network that generalizes well, you need to pick an optimal number of hidden layers, as well as nodes in each hidden layer.- Too few nodes will lead to high error for your system as the predictive factors might be too complex for a small number of nodes to capture- Too many nodes will overfit to your training data and not generalize well<jupyter_code>#previously done
from keras.models import Sequential
from keras.layers.core import Dense, Dropout
from keras.optimizers import SGD, Adam, Adamax
from keras.utils import np_utils
from keras.utils.vis_utils import model_to_dot
from keras.datasets import mnist
from keras.utils import np_utils
%matplotlib inline
import math
import random
import numpy as np
import matplotlib.pyplot as plt
from IPython.display import SVG
#Load MNIST
(X_train, y_train), (X_test, y_test) = mnist.load_data()
#Reshape
X_train = X_train.reshape(60000, 784)
X_test = X_test.reshape(10000, 784)
X_train = X_train.astype('float32')
X_test = X_test.astype('float32')
X_train /= 255
X_test /= 255
Y_train = np_utils.to_categorical(y_train, 10)
Y_test = np_utils.to_categorical(y_test, 10)
#Split
X_train = X_train[0:10000]
X_test = X_test[0:1000]
Y_train = Y_train[0:10000]
Y_test = Y_test[0:1000]

def plot_training_history(history):
    plt.plot(history.history['acc'])
    plt.plot(history.history['val_acc'])
    plt.title('model accuracy')
    plt.ylabel('accuracy')
    plt.xlabel('epoch')
    plt.legend(['train', 'test'], loc='upper left')
    plt.show()
    #loss
    plt.plot(history.history['loss'])
    plt.plot(history.history['val_loss'])
    plt.title('model loss')
    plt.ylabel('loss')
    plt.xlabel('epoch')
    plt.legend(['train', 'test'], loc='upper left')
    plt.show()<jupyter_output>Using TensorFlow backend.<jupyter_text>Example<jupyter_code>model = Sequential()
model.add(Dense(input_dim=28*28, units=500, activation='relu'))
model.add(Dense(units=500, activation='relu'))
model.add(Dense(units=500, activation='relu'))
model.add(Dense(units=500, activation='relu'))
model.add(Dense(units=500, activation='relu'))
model.add(Dense(units=500, activation='relu'))
model.add(Dense(units=500, activation='relu'))
model.add(Dense(units=500, activation='relu'))
model.add(Dense(units=500, activation='relu'))
model.add(Dense(units=500, activation='relu'))
model.add(Dense(units=500, activation='relu'))
model.add(Dense(units=500, activation='relu'))
model.add(Dense(units=10, activation='softmax'))

model.compile(loss='mse',
              optimizer=SGD(lr=0.1),
              metrics=['accuracy'])

BATCH_SIZE=100
NP_EPOCHS=10
history = model.fit(X_train, Y_train,
                    batch_size=BATCH_SIZE, epochs=NP_EPOCHS,
                    verbose=1, validation_data=(X_test, Y_test))
plot_training_history(history)
model = Sequential()
model.add(Dense(input_dim=28*28, units=500, activation='relu'))
model.add(Dense(units=10, activation='softmax'))

model.compile(loss='mse',
              optimizer=SGD(lr=0.1),
              metrics=['accuracy'])

BATCH_SIZE=100
NP_EPOCHS=10
history = model.fit(X_train, Y_train,
                    batch_size=BATCH_SIZE, epochs=NP_EPOCHS,
                    verbose=1, validation_data=(X_test, Y_test))
plot_training_history(history)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11739241151005857, 'fraction_numerical': 0.03132161955691368, 'mean_word_length': 5.185826771653543, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 8, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '494657', 'license': 'MIT', 'n_tokens_mistral': 1373, 'n_tokens_neox': 1278, 'n_words': 337, 'path': 'Keras/.ipynb_checkpoints/keras_06-checkpoint.ipynb', 'repo_name': 'mlwkshops/ML101', 'size': '3927'}