<jupyter_start><jupyter_text>Einführung in Neuronale Netzwerke<jupyter_code>import warnings
warnings.filterwarnings('ignore')
%matplotlib inline
%pylab inline
import matplotlib.pylab as plt
import numpy as np
from distutils.version import StrictVersion
import sklearn
print(sklearn.__version__)

assert StrictVersion(sklearn.__version__ ) >= StrictVersion('0.18.1')
import tensorflow as tf
tf.logging.set_verbosity(tf.logging.ERROR)
print(tf.__version__)

assert StrictVersion(tf.__version__) >= StrictVersion('1.1.0')
import keras
print(keras.__version__)

assert StrictVersion(keras.__version__) >= StrictVersion('2.0.0')<jupyter_output>Using TensorFlow backend.<jupyter_text>Iris mit Neuronalen Netzwerken  Das künstliche Neuron![Das künstliche Neuron](https://djcordhose.github.io/ai/img/sketch/neuron.jpg "Das künstliche Neuron") --- Hands-On Erzeuge eine Python-Implementierung eines Neurons mit zwei Eingabevariablen ohne Activation Funktion* Denke die Werte für w1, w2 und den Bias aus* Kannst du eine Skizze des Graphs der Funktion mit x1 und x2 an den Achsen erstellen?* Was ist das für eine Funktion?---<jupyter_code>%load https://djcordhose.github.io/ai/fragments/neuron.py<jupyter_output><empty_output><jupyter_text>Wir probieren unser Modell mit dem Iris Dataset<jupyter_code>from sklearn.datasets import load_iris
iris = load_iris()
iris.data[0]
neuron_no_activation(5.1, 3.5)<jupyter_output><empty_output><jupyter_text>Wie sollen wir das interpretieren? Damit können wir nicht viel anfangen  Activation Functions<jupyter_code>def centerAxis(uses_negative=False):
    # http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.plot
    ax = plt.gca()
    ax.spines['left'].set_position('center')
    if uses_negative:
        ax.spines['bottom'].set_position('center')
    ax.spines['right'].set_color('none')
    ax.spines['top'].set_color('none')
    ax.xaxis.set_ticks_position('bottom')
    ax.yaxis.set_ticks_position('left')<jupyter_output><empty_output><jupyter_text>Sigmoid<jupyter_code>def np_sigmoid(X):
    return 1 / (1 + np.exp(X * -1))
x = np.arange(-10,10,0.01)
y = np_sigmoid(x)

centerAxis()
plt.plot(x,y,lw=3)<jupyter_output><empty_output><jupyter_text>Relu<jupyter_code>def np_relu(x):
    return np.maximum(0, x)
x = np.arange(-10, 10, 0.01)
y = np_relu(x)

centerAxis()
plt.plot(x,y,lw=3)<jupyter_output><empty_output><jupyter_text>Das komplette Neuron<jupyter_code>w0 = 3
w1 = -4
w2 = 2

import math as math
def sigmoid(x):
    return 1 / (1 + math.exp(x * -1))

def neuron(x1, x2):
    sum = w0 + x1 * w1 + x2 * w2
    return sigmoid(sum)
neuron(5.1, 3.5)<jupyter_output><empty_output><jupyter_text>Unser erste Neuronales Netz mit Keras![Neuron zu Netz](https://djcordhose.github.io/ai/img/sketch/neuron_to_layers.jpg "Neuron zu Netz")<jupyter_code>from keras.layers import Input
inputs = Input(shape=(4, ))
from keras.layers import Dense
fc = Dense(3)(inputs)
from keras.models import Model
model = Model(input=inputs, output=fc)
model.summary()
model.compile(optimizer='adam',
              loss='categorical_crossentropy',
              metrics=['accuracy'])
model.predict(np.array([[ 5.1,  3.5,  1.4,  0.2]]))<jupyter_output><empty_output><jupyter_text>![Fully Connected](https://djcordhose.github.io/ai/img/sketch/fc_nn.jpg "Fully Connected")<jupyter_code>inputs = Input(shape=(4, ))
fc = Dense(3)(inputs)
predictions = Dense(3, activation='softmax')(fc)
model = Model(input=inputs, output=predictions)
model.summary()
model.compile(optimizer='adam',
              loss='categorical_crossentropy',
              metrics=['accuracy'])
model.predict(np.array([[ 5.1,  3.5,  1.4,  0.2]]))<jupyter_output><empty_output><jupyter_text>Training<jupyter_code>X = np.array(iris.data)
y = np.array(iris.target)
X.shape, y.shape
y[100]
# tiny little pieces of feature engeneering
from keras.utils.np_utils import to_categorical

num_categories = 3

y = to_categorical(y, num_categories)
y[100]
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=42, stratify=y)
X_train.shape, X_test.shape, y_train.shape, y_test.shape
# !rm -r tf_log
# tb_callback = keras.callbacks.TensorBoard(log_dir='./tf_log')

# https://keras.io/callbacks/#tensorboard
# To start tensorboard
# tensorboard --logdir=/mnt/c/Users/olive/Development/ml/tf_log
# open http://localhost:6006
# %time model.fit(X_train, y_train, epochs=500, validation_split=0.3, callbacks=[tb_callback])
%time model.fit(X_train, y_train, epochs=500, validation_split=0.3)<jupyter_output>Train on 84 samples, validate on 36 samples
Epoch 1/500
84/84 [==============================] - 0s - loss: 1.6569 - acc: 0.3690 - val_loss: 2.0478 - val_acc: 0.2222
Epoch 2/500
84/84 [==============================] - 0s - loss: 1.6029 - acc: 0.3690 - val_loss: 1.9716 - val_acc: 0.2222
Epoch 3/500
84/84 [==============================] - 0s - loss: 1.5480 - acc: 0.3690 - val_loss: 1.8985 - val_acc: 0.2222
Epoch 4/500
84/84 [==============================] - 0s - loss: 1.4995 - acc: 0.3690 - val_loss: 1.8268 - val_acc: 0.2222
Epoch 5/500
84/84 [==============================] - 0s - loss: 1.4511 - acc: 0.3690 - val_loss: 1.7574 - val_acc: 0.2222
Epoch 6/500
84/84 [==============================] - 0s - loss: 1.4036 - acc: 0.3690 - val_loss: 1.6917 - val_acc: 0.2222
Epoch 7/500
84/84 [==============================] - 0s - loss: 1.3624 - acc: 0.3690 - val_loss: 1.6280 - val_acc: 0.2222
Epoch 8/500
84/84 [==============================] - 0s - loss: 1.3182 - acc: 0.3690 - val_loss: 1.568[...]<jupyter_text>Bewertung<jupyter_code>model.predict(np.array([[ 5.1,  3.5,  1.4,  0.2]]))
X[0], y[0]
train_loss, train_accuracy = model.evaluate(X_train, y_train)
train_loss, train_accuracy
test_loss, test_accuracy = model.evaluate(X_test, y_test)
test_loss, test_accuracy<jupyter_output>30/30 [==============================] - 0s<jupyter_text>--- Hands-On Vollziehe das Notebook bis hier nach und spiele mit den einigen Parametern* Variiere die Anzahl der Neuronen im Hidden Layer. Wieso geht das überhaupt mit 3 Neuronen* Ziehe eine weitere Schicht ein* Kannst du eine Skizze des Graphs der Funktion mit x1 und x2 an den Achsen erstellen?* Was ist das für eine Funktion?---  Stop Here  Optionaler Teil  Model im Keras und TensorFlow Format speichern<jupyter_code>model.save('nn-iris.hdf5')<jupyter_output><empty_output><jupyter_text>Export as raw tf model* https://tensorflow.github.io/serving/serving_basic.html* https://github.com/tensorflow/serving/blob/master/tensorflow_serving/example/mnist_saved_model.py<jupyter_code>import os
from keras import backend as K
K.set_learning_phase(0)
sess = K.get_session()
!rm -r tf
tf.app.flags.DEFINE_integer('model_version', 1, 'version number of the model.')
tf.app.flags.DEFINE_string('work_dir', '/tmp', 'Working directory.')
FLAGS = tf.app.flags.FLAGS
export_path_base = 'tf'
export_path = os.path.join(
  tf.compat.as_bytes(export_path_base),
  tf.compat.as_bytes(str(FLAGS.model_version)))
classification_inputs = tf.saved_model.utils.build_tensor_info(model.input)
classification_outputs_scores = tf.saved_model.utils.build_tensor_info(model.output)
from tensorflow.python.saved_model.signature_def_utils_impl import build_signature_def, predict_signature_def
signature = predict_signature_def(inputs={'inputs': model.input},
                                  outputs={'scores': model.output})
builder = tf.saved_model.builder.SavedModelBuilder(export_path)
builder.add_meta_graph_and_variables(
      sess, 
     tags=[tf.saved_model.tag_constants.SERVING],
      signature_def_map={
          tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: signature
      })
builder.save()
!ls -lhR tf<jupyter_output><empty_output><jupyter_text>Dieses Tensorflow Modell kann man bei Google Cloud ML hochladen und für Berechnungen nutzen<jupyter_code># cd tf
# gsutil cp -R 1 gs://irisnn
# create model and version at https://console.cloud.google.com/mlengine
# gcloud ml-engine predict --model=irisnn --json-instances=./sample_iris.json
# SCORES
# [0.9954029321670532, 0.004596732556819916, 3.3544753819114703e-07]<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1590157914065369, 'fraction_numerical': 0.050556983718937444, 'mean_word_length': 5.343167701863354, 'pattern_counts': {'":': 0, '<': 54, '<?xml version=': 0, '>': 57, 'https://': 8, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '383519', 'license': 'MIT', 'n_tokens_mistral': 3302, 'n_tokens_neox': 3028, 'n_words': 601, 'path': 'notebooks/workshops/d2d/nn-intro.ipynb', 'repo_name': 'subodhchhabra/ai', 'size': '8169'}