<jupyter_start><jupyter_text>Load drugbank and calculate fingerprints<jupyter_code>supp = Chem.SDMolSupplier("../data/drugbank.sdf")
drugbank = [m for m in supp if m]
fps = [Chem.GetMorganFingerprintAsBitVect(m,2,nBits=1024) for m in drugbank]<jupyter_output><empty_output><jupyter_text>Prepare full distance matrix<jupyter_code>sparse_mat = distance.pdist(fps,metric="jaccard")
dist_mat = distance.squareform(sparse_mat)<jupyter_output><empty_output><jupyter_text>Calculate new 2D coordinates with MDS<jupyter_code>#----------------------------------------------------------------------
# MDS  embedding of the digits dataset
clf = manifold.MDS(n_components=2, n_init=1, max_iter=100, dissimilarity="precomputed")
X_mds = clf.fit_transform(dist_mat)<jupyter_output><empty_output><jupyter_text>Function visualizing 2D points<jupyter_code>#----------------------------------------------------------------------
# Scale and visualize the embedding vectors
def plot_embedding(X, title=None, highlights=[]):
    fig = plt.figure()
    # ax = plt.subplot(111)
    for i, point in enumerate(X):
        x,y = point
        if i in highlights:
            plt.plot([x],[y],"r.", markersize=4)
        else:
            plt.plot([x],[y],"k.", markersize=1)
    # plt.xticks([]), plt.yticks([])
    if title is not None:
        plt.title(title)
    return fig<jupyter_output><empty_output><jupyter_text>Show drugbank in 2D space<jupyter_code>plot_embedding(X_mds,"MDS")<jupyter_output><empty_output><jupyter_text>Substructures with acetylsalicylic acid<jupyter_code>acet_salicyl_a = Chem.MolFromSmiles("O=C(C)Oc1ccccc1C(=O)O")
acet_salicyl_a_idxs = [i for i,m in enumerate(drugbank) if m.HasSubstructMatch(acet_salicyl_a)]
plot_embedding(X_mds,"MDS",highlights=acet_salicyl_a_idxs)<jupyter_output><empty_output><jupyter_text>Structures similar to our structure<jupyter_code>aspirine_like_structs_idxs = [i for i, dist in enumerate(dist_mat[824,]) if dist<0.5] 
plot_embedding(X_mds,"MDS",highlights=aspirine_like_structs_idxs)<jupyter_output><empty_output><jupyter_text>Picking of diverse set<jupyter_code># This implementation is also working but why calculate distance if we already have it
# def distance_function(index1, index2, *args, fingerprints=fps):
#     return 1 - DataStructs.FingerprintSimilarity(fingerprints[index1],fingerprints[index2])

def full_distance_function(index1, index2, *args, matrix=dist_mat):
    return dist_mat[index1,index2]

picker = MaxMinPicker()
picked = picker.LazyPick(full_distance_function, len(fps), 100, seed=666)
highlighted = tuple(picked)
plot_embedding(X_mds,"MDS",highlights=highlighted)<jupyter_output><empty_output><jupyter_text>What have we picked?<jupyter_code>Draw.MolsToGridImage([drugbank[i] for i in picked[:18]])
# This one makes all structures so tiny that you can't see them in grid (there could be even more problematic onessimilar)
Draw.MolToImage(drugbank[picked[18]])<jupyter_output><empty_output><jupyter_text>Too much inorganic will kill youUsually in chemiformatics, only organic structures are used, so we have to deal with inorganic ones.Let's start with structures without carbon atom<jupyter_code>hasCarbonPatt = Chem.MolFromSmarts("[#6]")
hasCarbons = [(i, m.HasSubstructMatch(hasCarbonPatt)) for i,m in enumerate(drugbank)]
hasCarbonIdxs = [i for i,has in hasCarbons if has]
hasNotCarbonIdxs = [i for i,has in hasCarbons if not has]
len(hasCarbonIdxs)
plot_embedding(X_mds,"MDS",highlights=hasNotCarbonIdxs)
Draw.MolsToGridImage([drugbank[i] for i in hasNotCarbonIdxs[:12]])<jupyter_output><empty_output><jupyter_text>We want definitely to remove such inorganic structures, but is it enough?  Remove inorganic structure Let's remove structures without carbon and with uncommon atom type (not in organic subset)<jupyter_code>onlyOrganicPatt = Chem.MolFromSmarts("[!#6;!#7;!#8;!#16;!F;!Cl;!Br;!I]")
onlyOrganics = [(i, m.HasSubstructMatch(hasCarbonPatt) and not m.HasSubstructMatch(onlyOrganicPatt)) for i,m in enumerate(drugbank)]
onlyOrganicIdxs = [i for i,hasPatt in onlyOrganics if hasPatt]
notOnlyOrganicIdxs = [i for i,hasPatt in onlyOrganics if not hasPatt]
plot_embedding(X_mds,"MDS",highlights=notOnlyOrganicIdxs)
Draw.MolsToGridImage([drugbank[i] for i in notOnlyOrganicIdxs[2:7]])
# Structures at notOnlyOrganicIdxs[1 and 7] are not able to pass sanity check (that's why I skipped them)
# These structures are not sanitizable (we can't easily depict it), you should consider usefullness of such structures
print(Chem.MolToSmiles(drugbank[notOnlyOrganicIdxs[1]]))
notSanitizable = drugbank[notOnlyOrganicIdxs[1]]
Chem.SanitizeMol(notSanitizable)
lou událost vyšetřují, se nyní obracejí k veřejnosti s žádostí o pomoc při zjištění totožnosti napadeného. Uvedli, že z místa odešel jiným směrem než agresor a jeho # but after sanitization (even it failed), everything is awesome
Draw.MolToImage(notSanitizable)<jupyter_output><empty_output><jupyter_text>Do we really want to remove such structures?<jupyter_code>Draw.MolToImage(drugbank[notOnlyOrganicIdxs[13]])<jupyter_output><empty_output><jupyter_text>Or are we too far and we just want to remove inorganic part?How could be this improved:Salt removerhttp://www.rdkit.org/Python_Docs/rdkit.Chem.SaltRemover.SaltRemover-class.htmlNeutralizinghttp://www.rdkit.org/docs/Cookbook.htmlneutralizing-charged-molecules  Remove salts, neutralize, and maybe even standardize<jupyter_code>remover = SaltRemover(defnData='[!C;!c;!#7;!#8;!#16;!Cl;!F;!Br;!I]')
remover2 = SaltRemover()
print([Chem.MolToSmarts(mol) for mol in remover.salts])
print([Chem.MolToSmarts(mol) for mol in remover2.salts])<jupyter_output><empty_output><jupyter_text>Default patterns seem to be good<jupyter_code>print(Chem.MolToSmiles(drugbank[notOnlyOrganicIdxs[13]]))
print(Chem.MolToSmiles(remover2(drugbank[notOnlyOrganicIdxs[13]])))

# In this case, we ommit the issue with fragmented structures as we use fingerprint and both fragments are same<jupyter_output><empty_output><jupyter_text>We removed cation but then we should neutralize structure (it cannot work absolutely, for example quaternary ammonium cations)<jupyter_code>""" contribution from <NAME> """
def _InitialiseNeutralisationReactions():
    patts= (
        # Imidazoles
        ('[n+;H]','n'),
        # Amines
        ('[N+;!H0]','N'),
        # Carboxylic acids and alcohols
        ('[$([O-]);!$([O-][#7])]','O'),
        # Thiols
        ('[S-;X1]','S'),
        # Sulfonamides
        ('[$([N-;X2]S(=O)=O)]','N'),
        # Enamines
        ('[$([N-;X2][C,N]=C)]','N'),
        # Tetrazoles
        ('[n-]','[nH]'),
        # Sulfoxides
        ('[$([S-]=O)]','S'),
        # Amides
        ('[$([N-]C=O)]','N'),
        )
    return [(Chem.MolFromSmarts(x),Chem.MolFromSmiles(y,False)) for x,y in patts]

_reactions=None
def NeutraliseCharges(mol, reactions=None):
    global _reactions
    if reactions is None:
        if _reactions is None:
            _reactions=_InitialiseNeutralisationReactions()
        reactions=_reactions
    replaced = False
    for i,(reactant, product) in enumerate(reactions):
        while mol.HasSubstructMatch(reactant):
            replaced = True
            rms = Chem.ReplaceSubstructs(mol, reactant, product)
            mol = rms[0]
    return mol
print(Chem.MolToSmiles(remover2(drugbank[notOnlyOrganicIdxs[13]])))
print(Chem.MolToSmiles(NeutraliseCharges(remover2(drugbank[notOnlyOrganicIdxs[13]]))))<jupyter_output><empty_output><jupyter_text>Our finale standardization function could be implemented as:<jupyter_code>_saltRemover = SaltRemover()
_inorganicPatt = Chem.MolFromSmarts("[!#6;!#7;!#8;!#16;!F;!Cl;!Br;!I]")
_carbonPatt = Chem.MolFromSmarts("[#6]")
def standardize(mol):
    if mol.HasSubstructMatch(_carbonPatt):
        mol = _saltRemover(mol)
        if mol.GetNumAtoms()==0:
            return None
        else:
            mol = NeutraliseCharges(mol)
            if mol.HasSubstructMatch(_inorganicPatt):
                return None
            else:
                # Sanitize mol, without sanitization some structures can't be drawn or fingerprinted
                Chem.SanitizeMol(mol) # add catch block?
                return mol
    else:
        return None<jupyter_output><empty_output><jupyter_text>Let's make our visualization great (again)remove salts, neutralize and then remove the rest of inogranics<jupyter_code>supp = Chem.SDMolSupplier("../data/drugbank.sdf")
cleanDrugbank = [sm for sm in (standardize(m) for m in supp if m) if sm]
cleanFps = [Chem.GetMorganFingerprintAsBitVect(m,2,nBits=1024) for i,m in enumerate(cleanDrugbank)]
#Prepare full distance matrix

clean_sparse_mat = distance.pdist(cleanFps,metric="jaccard")
clean_dist_mat = distance.squareform(clean_sparse_mat)

#Calculate new 2D coordinates with MDS

#----------------------------------------------------------------------
# MDS  embedding of the digits dataset
clf = manifold.MDS(n_components=2, n_init=1, max_iter=100, dissimilarity="precomputed")
clean_mds = clf.fit_transform(clean_dist_mat)
def clean_distance_function(index1, index2, *args, matrix=clean_dist_mat):
    return matrix[index1,index2]

picker = MaxMinPicker()
clean_picked = picker.LazyPick(clean_distance_function, len(cleanFps), 100, seed=666)
clean_highlighted = tuple(clean_picked)
plot_embedding(clean_mds,"MDS",highlights=clean_highlighted)

Draw.MolsToGridImage([cleanDrugbank[i] for i in clean_picked[:12]])
# Calcium Cyanamide CaCN2
print(Chem.MolToSmiles(cleanDrugbank[clean_picked[3]]))
Draw.MolToImage(cleanDrugbank[clean_picked[3]])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13793825222396652, 'fraction_numerical': 0.012140240711669283, 'mean_word_length': 5.067301587301587, 'pattern_counts': {'":': 0, '<': 71, '<?xml version=': 0, '>': 70, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '377617', 'license': 'MIT', 'n_tokens_mistral': 3485, 'n_tokens_neox': 3320, 'n_words': 679, 'path': 'examples/MDS vizualization, standardization and picking of diverse set.ipynb', 'repo_name': 'lich-uct/cdd2018', 'size': '9563'}