<jupyter_start><jupyter_text>Loading Lipophilicity dataset**Lipophilicity:** Experimental results of octanol/water distribution coefficient(logD at pH 7.4).<jupyter_code>lipophilicity_df= pd.read_csv('../input/lipophilicity_descriptors/Lipophilicity_df_revised.csv')
print(lipophilicity_df.shape)
lipophilicity_df.head()
lipophilicity_df.drop(['smiles','CMPD_CHEMBLID'], axis=1, inplace=True)
print(lipophilicity_df.shape)
lipophilicity_df.head()
# Get indices of NaN
#inds = pd.isnull(tox21_df).any(1).nonzero()[0]
# Drop NaN from the dataframe
#tox21_df.dropna(inplace=True)
#print(tox21_df.shape)
#tox21_df.head()
lipophilicity_df = lipophilicity_df.fillna(0)
lipophilicity_df.head()<jupyter_output><empty_output><jupyter_text>The simplified molecular-input line-entry system (**SMILES**) is a specification in form of a line notation for describing the structure of chemical species using short ASCII strings. SMILES can be converted to molecular structure by using RDKIT module.Example: ```pythonfrom rdkit import Chemm = Chem.MolFromSmiles('Cc1ccccc1')```Further reading:* https://www.rdkit.org/docs/GettingStartedInPython.html  Loading molecular descriptorsDescriptors dataframe contains 1625 molecular descriptors (including 3D descriptors) generated on the NCI database using Mordred python module.Further Reading:* https://en.wikipedia.org/wiki/Molecular_descriptor* https://github.com/mordred-descriptor/mordred<jupyter_code>lipophilicity_descriptors_df= pd.read_csv('../input/lipophilicity_descriptors/Lipophilicity_descriptors_df.csv',low_memory=False)
print(lipophilicity_descriptors_df.shape)
lipophilicity_descriptors_df.head()
# function to coerce all data types to numeric

def coerce_to_numeric(df, column_list):
    df[column_list] = df[column_list].apply(pd.to_numeric, errors='coerce')
coerce_to_numeric(lipophilicity_descriptors_df, lipophilicity_descriptors_df.columns)
lipophilicity_descriptors_df.head()
lipophilicity_descriptors_df = lipophilicity_descriptors_df.fillna(0)
lipophilicity_descriptors_df.head()
#tox21_descriptors_df.drop(tox21_descriptors_df.index[inds],inplace=True)
#tox21_descriptors_df.shape<jupyter_output><empty_output><jupyter_text>Scaling and Principal component analysis (PCA)<jupyter_code>lipophilicity_scaler1 = StandardScaler()
lipophilicity_scaler1.fit(lipophilicity_descriptors_df.values)
lipophilicity_descriptors_df = pd.DataFrame(lipophilicity_scaler1.transform(lipophilicity_descriptors_df.values),
                                            columns=lipophilicity_descriptors_df.columns)
nca = NCA1
cn = ['col'+str(x) for x in range(nca)]
lipophilicity_transformer1 = KernelPCA(n_components=nca, kernel='rbf', n_jobs=-1)
lipophilicity_transformer1.fit(lipophilicity_descriptors_df.values)
lipophilicity_descriptors_df = pd.DataFrame(lipophilicity_transformer1.transform(lipophilicity_descriptors_df.values),
                                            columns=cn)
print(lipophilicity_descriptors_df.shape)
lipophilicity_descriptors_df.head()
X_train, X_test, y_train, y_test = train_test_split(lipophilicity_descriptors_df.values,
                                                    lipophilicity_df.values, random_state=32,
                                                    test_size=TEST_RATIO)<jupyter_output><empty_output><jupyter_text>Sklearn SVR Model<jupyter_code>parameters = {'estimator__kernel':['rbf'], 
              'estimator__epsilon':[0.1,0.25,0.5],
              'estimator__C':[1,0.5,0.25], 'estimator__gamma':['auto','scale']}
lipophilicity_svr = GridSearchCV(MultiOutputRegressor(SVR()), 
                        parameters, cv=3, scoring='neg_mean_squared_error',n_jobs=-1)
result = lipophilicity_svr.fit(X_train, y_train)
pred = lipophilicity_svr.predict(X_test)
svr_rmse = np.sqrt(mean_squared_error(y_test,pred))
print(svr_rmse)<jupyter_output>0.7533198612099034<jupyter_text>Keras Neural Network Model<jupyter_code>def huber_loss(y_true, y_pred):
        return tf.losses.huber_loss(y_true,y_pred)
lipophilicity_model = Sequential()
lipophilicity_model.add(Dense(128, input_dim=lipophilicity_descriptors_df.shape[1], 
                              kernel_initializer='he_uniform'))
lipophilicity_model.add(BatchNormalization())
lipophilicity_model.add(Activation('tanh'))
lipophilicity_model.add(Dropout(rate=DROPRATE))
lipophilicity_model.add(Dense(64,kernel_initializer='he_uniform'))
lipophilicity_model.add(BatchNormalization())
lipophilicity_model.add(Activation('tanh'))
lipophilicity_model.add(Dropout(rate=DROPRATE))
lipophilicity_model.add(Dense(32,kernel_initializer='he_uniform'))
lipophilicity_model.add(BatchNormalization())
lipophilicity_model.add(Activation('tanh'))
lipophilicity_model.add(Dropout(rate=DROPRATE))
lipophilicity_model.add(Dense(lipophilicity_df.shape[1],kernel_initializer='he_uniform',activation=None))
lipophilicity_model.compile(loss=huber_loss, optimizer='adam',metrics=['mse'])
checkpoint = ModelCheckpoint('lipophilicity_model.h5', monitor='val_loss', verbose=1, save_best_only=True, save_weights_only=True, mode='min')
hist = lipophilicity_model.fit(X_train, y_train, 
                               validation_split=VAL_RATIO,epochs=EP, batch_size=BATCH_SIZE, 
                               callbacks=[checkpoint])
plt.ylim(0., 1.)
plt.plot(hist.epoch, hist.history["loss"], label="Train loss")
plt.plot(hist.epoch, hist.history["val_loss"], label="Valid loss")
lipophilicity_model.load_weights('lipophilicity_model.h5')
pred = lipophilicity_model.predict(X_test)
nn_rmse = np.sqrt(mean_squared_error(y_test,pred))
print(nn_rmse)<jupyter_output>0.7204267441177015<jupyter_text>Gradient Boosting of Keras Model with SVC<jupyter_code>inp = lipophilicity_model.input
out = lipophilicity_model.layers[-2].output
lipophilicity_model_gb = Model(inp, out)
X_train = lipophilicity_model_gb.predict(X_train)
X_test = lipophilicity_model_gb.predict(X_test)
data = np.concatenate((X_train,X_test),axis=0)
lipophilicity_scaler2 = StandardScaler()
lipophilicity_scaler2.fit(data)
X_train = lipophilicity_scaler2.transform(X_train)
X_test = lipophilicity_scaler2.transform(X_test)
data = np.concatenate((X_train,X_test),axis=0)
nca = NCA2
lipophilicity_transformer2 = KernelPCA(n_components=nca, kernel='rbf', n_jobs=-1)
lipophilicity_transformer2.fit(data)
X_train = lipophilicity_transformer2.transform(X_train)
X_test = lipophilicity_transformer2.transform(X_test)
nca = X_train.shape[1]
parameters = {'estimator__kernel':['rbf'], 
              'estimator__epsilon':[0.1,0.25,0.5],
              'estimator__C':[1,0.5,0.25], 'estimator__gamma':['auto','scale']}
lipophilicity_svr_gb = GridSearchCV(MultiOutputRegressor(SVR()), 
                                    parameters, cv=3, scoring='neg_mean_squared_error',n_jobs=-1)
result = lipophilicity_svr_gb.fit(X_train, y_train)
pred = lipophilicity_svr_gb.predict(X_test)
svr_gb_rmse = np.sqrt(mean_squared_error(y_test,pred))
print(svr_gb_rmse)<jupyter_output>0.7253296192000921<jupyter_text>Gradient Boosting of Keras Model with XGBoost<jupyter_code>parameters = {'estimator__learning_rate':[0.05,0.1,0.15],'estimator__n_estimators':[75,100,125], 'estimator__max_depth':[3,5,7],
              'estimator__booster':['gbtree','dart'],'estimator__reg_alpha':[0.1,0.05],'estimator__reg_lambda':[0.5,1.]}

lipophilicity_xgb_gb = GridSearchCV(MultiOutputRegressor(XGBRegressor(random_state=32)), 
                                    parameters, cv=3, scoring='neg_mean_squared_error',n_jobs=-1)
result = lipophilicity_xgb_gb.fit(X_train, y_train)
pred = lipophilicity_xgb_gb.predict(X_test)
xgb_gb_rmse = np.sqrt(mean_squared_error(y_test,pred))
print(xgb_gb_rmse)<jupyter_output>0.723236299768876<jupyter_text>Saving models, transformer and scaler<jupyter_code>with open('lipophilicity_svr.pkl', 'wb') as fid:
    pickle.dump(lipophilicity_svr, fid)
with open('lipophilicity_transformer1.pkl', 'wb') as fid:
    pickle.dump(lipophilicity_transformer1, fid)
with open('lipophilicity_transformer2.pkl', 'wb') as fid:
    pickle.dump(lipophilicity_transformer2, fid)
with open('lipophilicity_scaler1.pkl', 'wb') as fid:
    pickle.dump(lipophilicity_scaler1, fid)
with open('lipophilicity_scaler2.pkl', 'wb') as fid:
    pickle.dump(lipophilicity_scaler2, fid)
with open('lipophilicity_svr_gb.pkl', 'wb') as fid:
    pickle.dump(lipophilicity_svr_gb, fid)
with open('lipophilicity_xgb_gb.pkl', 'wb') as fid:
    pickle.dump(lipophilicity_xgb_gb, fid)<jupyter_output><empty_output><jupyter_text>For loading saved model```pythonwith open('lipophilicity_svr.pkl', 'rb') as fid:    lipophilicity_svr = pickle.load(fid) ```  Comparision of Results with MoleculeNet resultshttp://moleculenet.ai/full-resultsThe RMSE best score on the test data for the MoleculeNet models is ~0.7, the best score obtained in this kernel is ~0.72 on the test data. **Further Reading:*** https://arxiv.org/pdf/1703.00564.pdf  RMSE<jupyter_code>sns.set(style="whitegrid")
ax = sns.barplot(x=[svr_rmse,nn_rmse,svr_gb_rmse,xgb_gb_rmse],
                 y=['SVR','NN','SVC_GB','XGB_GB'])
ax.set_xlim(0.65,0.8)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11671993827840846, 'fraction_numerical': 0.024357985230904884, 'mean_word_length': 6.593305439330544, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 4, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '312032', 'license': 'MIT', 'n_tokens_mistral': 3531, 'n_tokens_neox': 3217, 'n_words': 419, 'path': 'physical_chemistry_lipophilicity.ipynb', 'repo_name': 'egluma/Chemical-analysis', 'size': '9073'}