<jupyter_start><jupyter_text>Read the antibacterial data from the supporting material from Stokes et al into a dataframe<jupyter_code>ab_df = pd.read_excel("mmc1.xlsx",sheet_name=1,skiprows=1)<jupyter_output><empty_output><jupyter_text>In order to be consistent with previous notebooks, set active and inactive to 1 and 0<jupyter_code>ab_df['active'] = [int(x == "Active") for x in ab_df.Activity]
ab_df.head()<jupyter_output><empty_output><jupyter_text>Use <NAME>'s descriptastorus to generate descritptors<jupyter_code>generator = MakeGenerator(("RDKit2D", "Morgan3Counts"))
desc_list = [generator.process(x) for x in tqdm(ab_df.SMILES)]<jupyter_output><empty_output><jupyter_text>Make sure all of our descriptor calculations worked.  Descriptastorus sets the first element in the returned list to True or False to inidcate the success of the calculations.<jupyter_code>desc_ok = [x[0] for x in desc_list]
pd.Series(desc_ok).value_counts()<jupyter_output><empty_output><jupyter_text>Looks like everything worked, add the descriptors to the dataframe.<jupyter_code>ab_df['desc'] = [x[1:] for x in desc_list]
ab_df.head()<jupyter_output><empty_output><jupyter_text>Create an XGBoost Classifier<jupyter_code>xgb = XGBClassifier()<jupyter_output><empty_output><jupyter_text>Let's build quick model to see how well we predict from the internal data.<jupyter_code>train,test = train_test_split(ab_df)<jupyter_output><empty_output><jupyter_text>Build a model with the data.<jupyter_code>x = np.array([x for x in train.desc])
y = train.active
xgb.fit(x,y)
test_x = np.array([x for x in test.desc])
xgb_prob = xgb.predict_proba(test_x)
xgb_pred = xgb.predict(test_x)<jupyter_output><empty_output><jupyter_text>Make a boxplot of probabilities vs activity lables to ensure that we're looking at this the right way<jupyter_code>ax = sns.boxplot(xgb_pred,xgb_prob[:,1])
ax.set(xlabel="activity",ylabel="probability")<jupyter_output><empty_output><jupyter_text>Calculate a ROC AUC for our model<jupyter_code>roc_auc_score(test.active,xgb_prob[:,1])<jupyter_output><empty_output><jupyter_text>This looks pretty good, now let's build a model with all of the antibiotic data.<jupyter_code>xgb_full = XGBClassifier()
x_full = np.array([x for x in ab_df.desc])
y_full = ab_df.active
xgb_full.fit(x_full,y_full)<jupyter_output><empty_output><jupyter_text>Now that we have a model, we can use it to make som predictions on the Broad Institute's repurposing set.  First, we'll grab this set from the web.<jupyter_code>url = "https://s3.amazonaws.com/data.clue.io/repurposing/downloads/repurposing_samples_20180907.txt"
repurposing_df = pd.read_csv(url, comment = "!", sep="\t")<jupyter_output><empty_output><jupyter_text>Let's take a look at the first few rows to see what's there.<jupyter_code>repurposing_df.head()<jupyter_output><empty_output><jupyter_text>As we saw in the previous notebook, the repurposing data contains duplicates and null values.  Let's get rid of these.<jupyter_code>repurposing_df.drop_duplicates('smiles',inplace=True)
repurposing_df.dropna(axis=0,subset=['smiles'],inplace=True)
repurposing_df.shape<jupyter_output><empty_output><jupyter_text>As above, use descriptastorus to calcualte descriptors<jupyter_code>repurposing_desc_list = [generator.process(x) for x in tqdm(repurposing_df.smiles)]
repurposing_desc_ok = [x[0] for x in repurposing_desc_list]
pd.Series(repurposing_desc_ok).value_counts()
repurposing_df['desc'] = [x[1:] for x in repurposing_desc_list]
repurposing_df.head()<jupyter_output><empty_output><jupyter_text>Format the descriptors for model building<jupyter_code>repurposing_x = np.array([x for x in repurposing_df.desc])<jupyter_output><empty_output><jupyter_text>Predict activity for the repurposing set and put this into the dataframe.<jupyter_code>repurposing_pred = xgb_full.predict_proba(repurposing_x)
repurposing_df['pred'] = repurposing_pred[:,1]<jupyter_output><empty_output><jupyter_text>Sort the dataframe by the predicted probabilities.<jupyter_code>repurposing_df.sort_values("pred",ascending=False,inplace=True)
repurposing_df.head()<jupyter_output><empty_output><jupyter_text>Create a rank order column<jupyter_code>repurposing_df['rank'] = range(0,repurposing_df.shape[0])<jupyter_output><empty_output><jupyter_text>Add a molecule column to the dataframe so that we can do a substructure search.<jupyter_code>repurposing_df['mol'] = [Chem.MolFromSmiles(x) for x in tqdm(repurposing_df.smiles)]<jupyter_output><empty_output><jupyter_text>Create a query molecule from Halicin<jupyter_code>halicin = Chem.MolFromSmarts("Nc1nnc(Sc2ncc(s2)[N+]([O-])=O)s1")
halicin<jupyter_output><empty_output><jupyter_text>Do the substructure search and show the matching row.<jupyter_code>repurposing_df['match'] = [x.HasSubstructMatch(halicin) for x in tqdm(repurposing_df.mol)]
repurposing_df.query("match")<jupyter_output><empty_output><jupyter_text>In this case, Halicin ranks 765, much lower than what we saw with ChemProp<jupyter_code>def rebalance_data(df_in):
    df_active = df_in.query("active == 1")
    df_inactive = df_in.query("active == 0")
    df_active_upsample = resample(df_active,replace=True,n_samples=df_inactive.shape[0])
    df_upsample = df_inactive.append(df_active_upsample)
    return df_upsample<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10999621355547141, 'fraction_numerical': 0.006815600151457781, 'mean_word_length': 9.159615384615385, 'pattern_counts': {'":': 0, '<': 94, '<?xml version=': 0, '>': 94, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '23', 'dir': 'jupyter-structured-clean-dedup', 'id': '300504', 'license': 'MIT', 'n_tokens_mistral': 1973, 'n_tokens_neox': 1922, 'n_words': 381, 'path': '03_antibiotic_xgboost.ipynb', 'repo_name': 'PatWalters/antibiotic', 'size': '5288'}