<jupyter_start><jupyter_text>Set our random seed so that all computations are deterministic<jupyter_code>seed = 21899<jupyter_output><empty_output><jupyter_text>Read in the raw data for the first 100K records of the HCEPDB into a pandas dataframe<jupyter_code>df = pd.read_csv('https://github.com/UWDIRECT/UWDIRECT.github.io/blob/master/Wi18_content/DSMCER/HCEPD_100K.csv?raw=true')
df.head()<jupyter_output><empty_output><jupyter_text>Separate out the predictors from the output<jupyter_code>X = df[['mass', 'voc', 'jsc', 'e_homo_alpha', 'e_gap_alpha', 
        'e_lumo_alpha']].values
Y = df[['pce']].values<jupyter_output><empty_output><jupyter_text>Let's create the test / train split for these data using 80/20.  The `_pn` extension is related to the 'prenormalization' nature of the data.<jupyter_code>X_train_pn, X_test_pn, y_train, y_test = train_test_split(X, Y,
                                                    test_size=0.20,
                                                    random_state=seed)<jupyter_output><empty_output><jupyter_text>Now we need to `StandardScaler` the training data and apply that scale to the test data.<jupyter_code># create the scaler from the training data only and keep it for later use
X_train_scaler = StandardScaler().fit(X_train_pn)
# apply the scaler transform to the training data
X_train = X_train_scaler.transform(X_train_pn)<jupyter_output><empty_output><jupyter_text>Now let's reuse that scaler transform on the test set.  This way we never contaminate the test data with the training data.  We'll start with a histogram of the testing data just to prove to ourselves it is working.<jupyter_code>plt.hist(X_test_pn[:,1])<jupyter_output><empty_output><jupyter_text>OK, bnow apply the training scaler transform to the test and plot a histogram<jupyter_code>X_test = X_train_scaler.transform(X_test_pn)
plt.hist(X_test[:,1])<jupyter_output><empty_output><jupyter_text>Let's create the neural network layoutThis is a simple neural network with no hidden layers and just the inputs transitioned to the output.<jupyter_code>def simple_model():
    # assemble the structure
    model = Sequential()
    model.add(Dense(6, input_dim=6, kernel_initializer='normal', activation='relu'))
    model.add(Dense(1, kernel_initializer='normal'))
    # compile the model
    model.compile(loss='mean_squared_error', optimizer='adam')
    return model<jupyter_output><empty_output><jupyter_text>Train the neural network with the following<jupyter_code># initialize the andom seed as this is used to generate
# the starting weights
np.random.seed(seed)
# create the NN framework
estimator = KerasRegressor(build_fn=simple_model,
        epochs=150, batch_size=25000, verbose=0)
history = estimator.fit(X_train, y_train, validation_split=0.33, epochs=150, 
        batch_size=10000, verbose=0)<jupyter_output><empty_output><jupyter_text>The history object returned by the `fit` call contains the information in a fitting run.<jupyter_code>print(history.history.keys())
print("final MSE for train is %.2f and for validation is %.2f" % 
      (history.history['loss'][-1], history.history['val_loss'][-1]))<jupyter_output><empty_output><jupyter_text>Let's plot it!<jupyter_code># summarize history for loss
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'validation'], loc='upper left')
plt.show()<jupyter_output><empty_output><jupyter_text>Let's get the MSE for the test set.<jupyter_code>test_loss = estimator.model.evaluate(X_test, y_test)
print("test set mse is %.2f" % test_loss)<jupyter_output><empty_output><jupyter_text>NEAT!So our train mse is very similar to the training and validation at the final step!   Let's look at another way to evaluate the set of models using cross validationUse 10 fold cross validation to evaluate the models generated from our training set.  We'll use scikit-learn's tools for this.  Remember, this is only assessing our training set.  If you get negative values, to make `cross_val_score` behave as expected, we have to flip the signs on the results (incompatibility with keras).<jupyter_code>kfold = KFold(n_splits=10, random_state=seed)
results = cross_val_score(estimator, X_train, y_train, cv=kfold)
print("Results: %.2f (%.2f) MSE" % (-1 * results.mean(), results.std()))<jupyter_output><empty_output><jupyter_text>Quick aside, `Pipeline`Let's use scikit learns `Pipeline` workflow to run a k-fold cross validation run on the learned model.With this tool, we create a workflow using the `Pipeline` object.  You provide a list of actions (as named tuples) to be performed.  We do this with `StandardScaler` to eliminate the posibility of training leakage into the cross validation test set during normalization.<jupyter_code>estimators = []
estimators.append(('standardize', StandardScaler()))
estimators.append(('mlp', KerasRegressor(build_fn=simple_model,
        epochs=150, batch_size=25000, verbose=0)))
pipeline = Pipeline(estimators)
kfold = KFold(n_splits=10, random_state=seed)
results = cross_val_score(pipeline, X_train, y_train, cv=kfold)
print('MSE mean: %.4f ; std: %.4f' % (-1 * results.mean(), results.std()))<jupyter_output><empty_output><jupyter_text>Now, let's try a more sophisticated modelLet's use a hidden layer this time.<jupyter_code>def medium_model():
    # assemble the structure
    model = Sequential()
    model.add(Dense(6, input_dim=6, kernel_initializer='normal', activation='relu'))
    model.add(Dense(4, kernel_initializer='normal', activation='relu'))
    model.add(Dense(1, kernel_initializer='normal'))
    # compile the model
    model.compile(loss='mean_squared_error', optimizer='adam')
    return model
# initialize the andom seed as this is used to generate
# the starting weights
np.random.seed(seed)
# create the NN framework
estimator = KerasRegressor(build_fn=medium_model,
        epochs=150, batch_size=25000, verbose=0)
history = estimator.fit(X_train, y_train, validation_split=0.33, epochs=150, 
        batch_size=10000, verbose=0)
print("final MSE for train is %.2f and for validation is %.2f" % 
      (history.history['loss'][-1], history.history['val_loss'][-1]))
# summarize history for loss
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'validation'], loc='upper left')
plt.show()
test_loss = estimator.model.evaluate(X_test, y_test)
print("test set mse is %.2f" % test_loss)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1123304374333181, 'fraction_numerical': 0.015546410608139003, 'mean_word_length': 5.737166324435318, 'pattern_counts': {'":': 0, '<': 61, '<?xml version=': 0, '>': 61, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '10760', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 2218, 'n_tokens_neox': 2093, 'n_words': 581, 'path': 'Wi19_content/DSMCER/L12_Neural_Networks.ipynb', 'repo_name': 'ShahResearchGroup/UWDIRECT.github.io', 'size': '6561'}