<jupyter_start><jupyter_text>Can we use a neural network to predict LogS? Use a neural network - so all neurons in one layer are connected to all other neurons in the next layer.<jupyter_code>import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
train_desc = pd.read_csv("../data/train_desc.csv", index_col=0)
train = pd.read_csv("../data/train_nonull.csv", index_col=0)
train_desc["Y"] = train["Log S"].values<jupyter_output><empty_output><jupyter_text>We won't be using cross validation, so we will split our data.2/3 will become training, and 1/3 will become validation.<jupyter_code>train, val = train_test_split(train_desc, test_size=1/3, random_state=1)

tx = train[train.columns[:-1]] #all but the last column
ty = train["Y"] #the last column

vx = val[val.columns[:-1]] #all but the last column
vy = val["Y"] #the last column

print(len(tx))
print(len(vx))
tx = np.array(tx)
ty = np.array(ty)

vx = np.array(vx)
vy = np.array(vy)
print(tx.shape)
input_shape = (196,)
epochs = 100<jupyter_output><empty_output><jupyter_text>The neural network modelThere are lots of hyperparameters that can have an effect on training performance. Unfortunately, we never know which are the right ones to begin with, so there's often a bit of a search in the space before we settle on the best model.<jupyter_code>#parameters to change:
activation = "tanh" #can be 'tanh', 'sigmoid', 'relu'
depth = 5 #how many hidden layers there are, can be any non zero integer
size = 0.5 #for every hidden layer, the number of neurons are multiplied by this factor. 
 #0.5 means a hidden layer is reduced by half. Can be any value between 0 and 2.
### The model. Do not change this code. Any parameters you have changed above will be reflected in here.
model = Sequential() 
model.add(Dense(units=196, activation=activation, input_shape=input_shape, kernel_initializer="normal"))

prev = 196

#For 5 hidden layers, reduce in size by half each layer.
for i in range(0,depth):
    prev *= size
    if prev > 1:
        model.add(Dense(units=int(prev), activation="tanh", kernel_initializer="normal"))
    else:
        next

#output layer - keep this code the same 
model.add(Dense(units=1, activation="linear")) 

#optimisation and compiling:
opt = tf.keras.optimizers.SGD(lr=0.01, momentum=0.0, decay=0.0, nesterov=False) #sgd default

model.compile(loss='mean_squared_error', optimizer=opt,
              metrics=['mae'])
model.summary()
history = model.fit(tx,ty, validation_data=(vx,vy),
                      epochs=epochs, verbose=1) #set verbose to 0 if you don't want to see lots of output<jupyter_output>Train on 570 samples, validate on 285 samples
WARNING:tensorflow:From C:\Users\hzv37416\AppData\Local\Continuum\anaconda3\envs\studentml\lib\site-packages\tensorflow\python\ops\math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
Epoch 1/100
570/570 [==============================] - 1s 2ms/sample - loss: 6.5240 - mean_absolute_error: 1.9403 - val_loss: 4.3817 - val_mean_absolute_error: 1.5337
Epoch 2/100
570/570 [==============================] - 0s 156us/sample - loss: 3.6684 - mean_absolute_error: 1.4858 - val_loss: 3.0258 - val_mean_absolute_error: 1.2814
Epoch 3/100
570/570 [==============================] - 0s 146us/sample - loss: 2.6140 - mean_absolute_error: 1.2343 - val_loss: 2.8969 - val_mean_absolute_error: 1.2687
Epoch 4/100
570/570 [==============================] - 0s 152us/sample - loss: 2.5035 - mean_absolute_error: 1.2076 - val_loss: 2.7540 - val_mean_abso[...]<jupyter_text>Visualise the trainingWe hope that over a number of epochs, the model is getting better at both the training and validation sets.This means it is learning!<jupyter_code>plt.figure()
plt.plot(range(1,epochs+1), history.history["loss"], label="Train")
plt.plot(range(1,epochs+1), history.history["val_loss"], label="Validation")
plt.legend()
plt.show()<jupyter_output><empty_output><jupyter_text>Make some predictions using your trained model<jupyter_code># Making predictions

predictions = model.predict(tx)
val_predictions = model.predict(vx)
from sklearn.metrics import r2_score, mean_squared_error
print("Training:")
print(r2_score(ty, predictions))
print(mean_squared_error(ty, predictions))
print("Validation:")
print(r2_score(vy, val_predictions))
print(mean_squared_error(vy, val_predictions))<jupyter_output>Validation:
0.4193864171907459
2.475434352287037
{'fraction_non_alphanumeric': 0.13402061855670103, 'fraction_numerical': 0.04939862542955326, 'mean_word_length': 5.327445652173913, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 20, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '561369', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1689, 'n_tokens_neox': 1506, 'n_words': 460, 'path': 'notebooks/Tensorflow.ipynb', 'repo_name': 'rebecca-mackenzie/ml_cheminformatics', 'size': '4656'}