<jupyter_start><jupyter_text>Neural FingerprintsWe create atom, bond, and edge tensors from molecule SMILES using `chemml.chem.tensorize_molecules` in order to build neural fingerprints using `chemml.models.NeuralGraphHidden` and `chemml.models.NeuralGraphOutput` modules. These neural fingerprints are then used as features to train a simple feed forward neural network to predict densities of small organic compounds using tensorflow. Here we import a sample dataset from ChemML library which has the SMILES codes for 500 small organic molecules with their densities in $kg/m^3$.<jupyter_code>import numpy as np
from chemml.datasets import load_organic_density
molecules, target, dragon_subset = load_organic_density()
target = np.asarray(target['density_Kg/m3'])<jupyter_output><empty_output><jupyter_text>Building `chemml.chem.Molecule` objects from molecule SMILES.<jupyter_code>from chemml.chem import Molecule
mol_objs_list = []
for smi in molecules['smiles']:
    mol = Molecule(smi, 'smiles')
    mol.hydrogens('add')
    mol.to_xyz('MMFF', maxIters=10000, mmffVariant='MMFF94s')
    mol_objs_list.append(mol)<jupyter_output><empty_output><jupyter_text>Molecule tensors can be used to create neural graph fingerprints using `chemml.models`<jupyter_code>from chemml.chem import tensorise_molecules
xatoms, xbonds, xedges = tensorise_molecules(molecules=mol_objs_list, max_degree=5, 
                                        max_atoms=None, n_jobs=-1, batch_size=100, verbose=True)<jupyter_output>Tensorising molecules in batches of 100 ...
500/500 [==================================================] - 0s 601us/step
Merging batch tensors ...    [DONE]<jupyter_text>Splitting and preprocessing the data<jupyter_code>from sklearn.model_selection import ShuffleSplit
from sklearn.preprocessing import StandardScaler
y_scale = StandardScaler()
rs = ShuffleSplit(n_splits=1, test_size=.20, random_state=42)

for train, test in rs.split(mol_objs_list):
    xatoms_train = xatoms[train]
    xatoms_test = xatoms[test]
    xbonds_train = xbonds[train]
    xbonds_test = xbonds[test]
    xedges_train = xedges[train]
    xedges_test = xedges[test]
    target_train = target[train]
    target_test = target[test]
    target_train = y_scale.fit_transform(target_train.reshape(-1,1))
print('Training data:\n')
print('Atoms: ',xatoms_train.shape)
print('Bonds: ',xbonds_train.shape)
print('Edges: ',xedges_train.shape)
print('Target: ',target_train.shape)

print('\nTesting data:\n')
print('Atoms: ',xatoms_test.shape)
print('Bonds: ',xbonds_test.shape)
print('Edges: ',xedges_test.shape)
print('Target: ',target_test.shape)<jupyter_output>Training data:

Atoms:  (400, 57, 62)
Bonds:  (400, 57, 5, 6)
Edges:  (400, 57, 5)
Target:  (400, 1)

Testing data:

Atoms:  (100, 57, 62)
Bonds:  (100, 57, 5, 6)
Edges:  (100, 57, 5)
Target:  (100,)<jupyter_text>Building the Neural FingerprintsThe atom, bond, and edge tensors are used here to build 200 neural fingerprints of width 8 (i.e., the size atomic neighborhood which will be considered in the convolution process).<jupyter_code>from chemml.models import NeuralGraphHidden, NeuralGraphOutput
from tensorflow.keras.layers import Input, add
import tensorflow as tf
tf.random.set_seed(42)

conv_width = 8
fp_length = 200

num_molecules = xatoms_train.shape[0]
max_atoms = xatoms_train.shape[1]
max_degree = xbonds_train.shape[2]
num_atom_features = xatoms_train.shape[-1]
num_bond_features = xbonds_train.shape[-1]

# Creating input layers for atoms ,bonds and edge information
atoms0 = Input(name='atom_inputs', shape=(max_atoms, num_atom_features),batch_size=None)
bonds = Input(name='bond_inputs', shape=(max_atoms, max_degree, num_bond_features),batch_size=None)
edges = Input(name='edge_inputs', shape=(max_atoms, max_degree), dtype='int32',batch_size=None)

# Defining the convolved atom feature layers                                    
atoms1 = NeuralGraphHidden(conv_width, activation='relu', use_bias=False)([atoms0, bonds, edges])
atoms2 = NeuralGraphHidden(conv_width, activation='relu', use_bias=False)([atoms1, bonds, edges])

# Defining the outputs of each (convolved) atom feature layer to fingerprint
fp_out0 = NeuralGraphOutput(fp_length, activation='softmax')([atoms0,bonds,edges])
fp_out1 = NeuralGraphOutput(fp_length, activation='softmax')([atoms1,bonds,edges])
fp_out2 = NeuralGraphOutput(fp_length, activation='softmax')([atoms2,bonds,edges])

# Sum outputs to obtain fingerprint                                            
final_fp = add([fp_out0, fp_out1, fp_out2])
print('Neural Fingerprint Shape: ',final_fp.shape)<jupyter_output>Neural Fingerprint Shape:  (None, 200)<jupyter_text>Building and training the neural networkHere, we build and train a simple feed forward neural network using `tensorflow.keras` and provide our neural fingerprints as features.<jupyter_code>from tensorflow.keras import Model
from tensorflow.keras.layers import Dense

# Build and compile model for regression.
dense_layer0 = Dense(128,activation='relu',name='dense_layer0',
                     kernel_regularizer=tf.keras.regularizers.l2(0.01))(final_fp)
dense_layer1 = Dense(64,activation='relu',name='dense_layer1',
                     kernel_regularizer=tf.keras.regularizers.l2(0.01))(dense_layer0)
dense_layer2 = Dense(32,activation='relu',name='dense_layer2',
                     kernel_regularizer=tf.keras.regularizers.l2(0.01))(dense_layer1)

main_prediction = Dense(1, activation='linear', name='main_prediction')(dense_layer1)
model = Model(inputs=[atoms0, bonds, edges], outputs=[main_prediction])
model.compile(optimizer='adam', loss='mae')

# Show summary
model.summary()

model.fit([xatoms_train, xbonds_train, xedges_train], target_train, epochs=50,
          steps_per_epoch=None, batch_size=None,verbose=False,validation_split=0.1)<jupyter_output>Model: "model"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
atom_inputs (InputLayer)        [(None, 57, 62)]     0                                            
__________________________________________________________________________________________________
bond_inputs (InputLayer)        [(None, 57, 5, 6)]   0                                            
__________________________________________________________________________________________________
edge_inputs (InputLayer)        [(None, 57, 5)]      0                                            
__________________________________________________________________________________________________
neural_graph_hidden (NeuralGrap (None, 57, 8)        2720        atom_inputs[0][0]            [...]<jupyter_text>Predicting the density of the molecules in our test data and evaluating our model based on it.<jupyter_code>from chemml.utils import regression_metrics

y_pred = model.predict([xatoms_test,xbonds_test,xedges_test])
y_pred = y_scale.inverse_transform(y_pred)
metrics_dict = regression_metrics(target_test, list(y_pred.reshape(-1,)))
mae = metrics_dict['MAE']
r_2 = metrics_dict['r_squared']

print("Mean Absolute Error = {} kg/m^3".format(mae.round(3)))
print("R squared = {}".format(r_2.round(3)))<jupyter_output>Mean Absolute Error = 18.407 kg/m^3
R squared = 0.903
{'fraction_non_alphanumeric': 0.12045914922349764, 'fraction_numerical': 0.025793382849426064, 'mean_word_length': 5.146058091286307, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 9, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '51358', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 2434, 'n_tokens_neox': 2207, 'n_words': 520, 'path': 'docs/ipython_notebooks/Convo_nets.ipynb', 'repo_name': 'hachmannlab/chemml', 'size': '7405'}