<jupyter_start><jupyter_text>Neural Network We'll now use a Neural Network to predict the players identity.<jupyter_code>%matplotlib notebook

import pylab as plt
import numpy as np
import seaborn as sns; sns.set()

import keras
from keras.models import Sequential, Model
from keras.layers import Dense
from keras.optimizers import Adam

from sklearn.decomposition import PCA<jupyter_output>Using TensorFlow backend.<jupyter_text>We'll start with the features and the topology proposed by the last year and train the NN with it. After we'll try with our features (generated by wave instead of each balloon).<jupyter_code>data = np.genfromtxt('../features/kate_data_julien_sarah.csv', delimiter=',')
np.random.shuffle(data)
training_ratio = 0.85
l = len(data)
X = data[:,:-1]
y = data[:,-1]
X_train = X[:int(l*training_ratio)]
X_test = X[int(l*training_ratio):]
y_train = y[:int(l*training_ratio)]/2
y_test = y[int(l*training_ratio):]/2
y_train = keras.utils.np_utils.to_categorical(y_train.astype(int))
y_test = keras.utils.np_utils.to_categorical(y_test.astype(int))<jupyter_output><empty_output><jupyter_text>Dimensionality reduction with PCA<jupyter_code>mu = X_train.mean(axis=0)
U,s,V = np.linalg.svd(X_train - mu, full_matrices=False)
Zpca = np.dot(X_train - mu, V.transpose())

Rpca = np.dot(Zpca[:,:2], V[:2,:]) + mu    # reconstruction
err = np.sum((X_train-Rpca)**2)/Rpca.shape[0]/Rpca.shape[1]
print('PCA reconstruction error with 2 PCs: ' + str(round(err,3)));
print(max(Zpca[:,0]))
print(min(Zpca[:,0]))
print(max(Zpca[:,1]))
print(min(Zpca[:,1]))

print(np.argmax(Zpca[:,0]))
print(np.argmax(Zpca[:,1]))<jupyter_output>PCA reconstruction error with 2 PCs: 7.471
2234.6621633472378
-22.639794956373947
128.40015756257122
-369.87814850662505
94
94<jupyter_text>Building and training of a dnn<jupyter_code>m = Sequential()
m.add(Dense(150,  activation='relu', input_shape=(105,)))
#m.add(Dense(150,  activation='relu')) 
m.add(Dense(150,  activation='relu'))
m.add(Dense(150,  activation='relu'))
m.add(Dense(50,  activation='relu'))
m.add(Dense(2,  activation='sigmoid'))
m.compile(loss='categorical_crossentropy', optimizer = Adam(), metrics=['accuracy'])

history = m.fit(X_train, y_train, batch_size=10, epochs=20, verbose=1, validation_data = (X_test, y_test))
y_pred = m.predict(X_test)
accuracy = m.evaluate(X_test, y_test)[1]
print("Précision old features: %.2f" % accuracy)<jupyter_output>240/240 [==============================] - 0s 37us/step
Précision old features: 0.81<jupyter_text>Now let's try with our features and with a different topology. Since we have computed a 12 dimension feature-vector, it would make no sense to use layers with more than 100 neurons as the last year group did.<jupyter_code>X = np.genfromtxt('../features/features_wave_julian_sarah.csv', delimiter=',')
y = np.genfromtxt('../features/output_wave_julian_sarah.csv', delimiter=',')

p = np.random.permutation(len(X))
X, y = X[p], y[p]

training_ratio = 0.85
l = len(y)
X_train = X[:int(l*training_ratio)]
X_test = X[int(l*training_ratio):]
y_train = y[:int(l*training_ratio)]/2
y_test = y[int(l*training_ratio):]/2

y_train = keras.utils.np_utils.to_categorical(y_train.astype(int))
y_test = keras.utils.np_utils.to_categorical(y_test.astype(int))<jupyter_output><empty_output><jupyter_text>In our case we have 12 dimensions to features instead. Let's apply the NN directly without a PCA to compare later<jupyter_code>m = Sequential()
m.add(Dense(15,  activation='relu', input_shape=(12,)))
m.add(Dense(15,  activation='relu'))
m.add(Dense(15,  activation='relu'))
m.add(Dense(4,  activation='relu'))
m.add(Dense(2,  activation='sigmoid'))
m.compile(loss='categorical_crossentropy', optimizer = Adam(), metrics=['accuracy'])

history = m.fit(X_train, y_train, batch_size=10, epochs=20, verbose=1, validation_data = (X_test, y_test))
accuracy = m.evaluate(X_test, y_test)[1]
print("Précision New features: %.2f" % accuracy)<jupyter_output>240/240 [==============================] - 0s 29us/step
Précision New features: 0.88<jupyter_text>We got a precision of 88%, which is better than the 81% of the last year. But we would need to do an average to compare Let's apply PCA now to reduce the dimension to 3<jupyter_code>model_pca3 = PCA(n_components=3)

# On entraîne notre modèle (fit) sur les données
model_pca3.fit(X)

# On applique le résultat sur nos données :
X_reduced3 = model_pca3.transform(X)
training_ratio = 0.85
l = len(y)
X_train = X_reduced3[:int(l*training_ratio)]
X_test = X_reduced3[int(l*training_ratio):]
y_train = y[:int(l*training_ratio)]/2
y_test = y[int(l*training_ratio):]/2

y_train = keras.utils.np_utils.to_categorical(y_train.astype(int))
y_test = keras.utils.np_utils.to_categorical(y_test.astype(int))
m = Sequential()
m.add(Dense(20,  activation='relu', input_shape=(3,)))
#m.add(Dense(20,  activation='relu')) 
m.add(Dense(20,  activation='relu'))
m.add(Dense(20,  activation='relu'))
m.add(Dense(5,  activation='relu'))
m.add(Dense(2,  activation='sigmoid'))
m.compile(loss='categorical_crossentropy', optimizer = Adam(), metrics=['accuracy'])

history = m.fit(X_train, y_train, batch_size=10, epochs=20, verbose=1, validation_data = (X_test, y_test))<jupyter_output>Train on 1360 samples, validate on 240 samples
Epoch 1/20
1360/1360 [==============================] - 1s 555us/step - loss: 0.5776 - acc: 0.7301 - val_loss: 0.5513 - val_acc: 0.7500
Epoch 2/20
1360/1360 [==============================] - 0s 91us/step - loss: 0.4966 - acc: 0.7831 - val_loss: 0.5360 - val_acc: 0.7583
Epoch 3/20
1360/1360 [==============================] - 0s 87us/step - loss: 0.4722 - acc: 0.7860 - val_loss: 0.5485 - val_acc: 0.7333
Epoch 4/20
1360/1360 [==============================] - 0s 88us/step - loss: 0.4613 - acc: 0.7897 - val_loss: 0.5361 - val_acc: 0.7583
Epoch 5/20
1360/1360 [==============================] - 0s 86us/step - loss: 0.4488 - acc: 0.7949 - val_loss: 0.5459 - val_acc: 0.7458
Epoch 6/20
1360/1360 [==============================] - 0s 87us/step - loss: 0.4395 - acc: 0.7963 - val_loss: 0.5291 - val_acc: 0.7375
Epoch 7/20
1360/1360 [==============================] - 0s 102us/step - loss: 0.4388 - acc: 0.7934 - val_loss: 0.5254 - val_acc: 0.7417
Epoch [...]
{'fraction_non_alphanumeric': 0.1910745932012244, 'fraction_numerical': 0.07346544224262928, 'mean_word_length': 6.054545454545455, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '581999', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 2703, 'n_tokens_neox': 2393, 'n_words': 514, 'path': 'Kinect_project/Neural Network/Neural Newtork.ipynb', 'repo_name': 'netomenoci/data-analysis', 'size': '6207'}