<jupyter_start><jupyter_text>Learning Mol2vec IntroductionIn the previous notebook, RDKit provided some fingerprinting methods so that you can make chemical comparisons, using _numerical vectors_. Depending on your application, however, these fingerprints may not _encode_ the information relevant to your task: this corresponds to a relatively broad area of machine learning, which is "representation learning"; how machines learn and use concrete representations of typically abstract concepts. In this case, we're concerned with _how molecules should be represented_, to tailor for your purpose. For example, maybe you need to know not only which functional groups are present, but also _how many_ and whether they are redundant. Another case maybe if you need to know if an aromatic ring is next to a particular functional group. These features/aspects may not be captured, or at least low-level enough for a machine learning method to extract.In this notebook, we'll look at using `mol2vec` to generate "encodings" or representations of molecules. `mol2vec` is inspired by `word2vec`, which is a widely used algorithm in natural language processing. Before we explain what `mol2vec` does, it's more instructional to start with `word2vec` as it is likely more relatable. Word2VecIn natural language, we form sentences using words. Each word carries a specific meaning, and a sentence of words establishes context that modifies the meaning of words. From a computing perspective, we could easily encode each word, or even letter, as vectors:```pythonhello = [1, 0, 0, 0, 0, 0]goodbye = [0, 1, 0, 0, 0, 0]apple = [0, 0, 1, 0, 0, 0]banana = [0, 0, 0, 1, 0, 0]...```You can then encode an entire dictionary of words into this format, where every word has 1 at a specific position (one-hot encoding). These words then make up a _corpus_, or your vocabulary. Now this is useful if you only cared about the words themselves, but in a sentence each word is independent of one another: this encoding does not reflect the fact that "goodbye" should come after "hello". In this corpus, the words hello/goodbye in English should also be more similar in semantics as they are greetings, compared to apple/banana which are fruits. `word2vec` was developed to generate encodings like the ones above, except in a more semantically useful manner by training machine learning model to convert words into vectors, where it learns how words are related to one another based on common usage. For example, we can take a textbook and throw it into `word2vec`, where it then learns how certain words are used in sentences more frequently than others. On a grander scale, you can do the same with Wikipedia, and the model will learn a larger corpus/vocabulary, and different set of semantics. Alternatively, you can also train it on Tweets, which is guaranteed to produce racist and poorly formed sentences.There are two ways of doing this, although I'll only explain one. The training method is to show the model sentences where one word is omitted out of say ten words (a window), and the job of the model is to predict which word in the corpus has been omitted. The workflow is something like this: using an example sentence "Jane is visiting ____ in September", the model has to find/recommend the most likely word to appear based on the training dataset.  Mol2vecThe `mol2vec` package uses the same machinery as `word2vec`, but applied to molecules. We train a corpus/vocabulary of molecules, where molecules are to sentences as atoms are to words. Instead of training a model to predict words, the model now predicts atoms in molecules. In the same way how words can now carry context, atoms can also carry information about their local environment.<jupyter_code>from pathlib import Path
from tempfile import NamedTemporaryFile
import fileinput
import os

import pandas as pd
from mol2vec import features
from rdkit import Chem
from rdkit.Chem.Draw import IPythonConsole
from gensim.models import word2vec
benzene = Chem.MolFromSmiles("c1ccccc1")<jupyter_output><empty_output><jupyter_text>The `mol2alt_sentence` function generates Morgan identifiers for each atom; the identifier takes into account the atom type, as well as its local environment.<jupyter_code>benzene_sentence = features.mol2alt_sentence(benzene, 1)
print(benzene_sentence)<jupyter_output>['3218693969', '98513984', '3218693969', '98513984', '3218693969', '98513984', '3218693969', '98513984', '3218693969', '98513984', '3218693969', '98513984']<jupyter_text>Organizing SMILES for corpus generationHere we're going to rely on two big datasets to build our corpus. It's important to keep in mind that all of our results will depend heavily on this step: the choice of molecules that will compose our corpus, and the number of examples for training. We'll be combining large, commonly used datasets: the QM9 set and a subset of ZINC15 containing molecules up to 200 amu in mass. Our primary focus here is on small-ish molecules (on the scale of biomolecules), and as such we do not want to swamp our corpus with extremely large molecules. Finally, we're going to also include molecules from the KIDA reaction network, which are astronomically relevant. Take a look at notebook "B1" for combining this data.  Generating a corpus for training<jupyter_code>CORPUSNAME = "mol2vec_corpus.dat"
RADIUS = 1
NJOBS = 4

# create a corpus from the SMILES
features.generate_corpus("collected_smiles.smi", CORPUSNAME, RADIUS, sentence_type="alt", n_jobs=NJOBS, sanitize=False)<jupyter_output>[Parallel(n_jobs=4)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=4)]: Done 123 tasks      | elapsed:    0.9s
[Parallel(n_jobs=4)]: Done 59396 tasks      | elapsed:    6.3s
[Parallel(n_jobs=4)]: Done 187396 tasks      | elapsed:   18.4s
[Parallel(n_jobs=4)]: Done 366596 tasks      | elapsed:   37.3s
[Parallel(n_jobs=4)]: Done 596996 tasks      | elapsed:  1.0min
[Parallel(n_jobs=4)]: Done 878596 tasks      | elapsed:  1.5min
[Parallel(n_jobs=4)]: Done 1211396 tasks      | elapsed:  2.1min
[Parallel(n_jobs=4)]: Done 1537432 out of 1537432 | elapsed:  2.7min finished<jupyter_text>Training the `mol2vec` model<jupyter_code>model = features.train_word2vec_model(CORPUSNAME, "mol2vec_model.pkl", vector_size=300, min_count=1, n_jobs=NJOBS)<jupyter_output>Runtime:  5.77  minutes
{'fraction_non_alphanumeric': 0.060438693388038506, 'fraction_numerical': 0.03992425437904371, 'mean_word_length': 5.195503421309873, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 14, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '328425', 'license': 'MIT', 'n_tokens_mistral': 1935, 'n_tokens_neox': 1685, 'n_words': 883, 'path': 'notebooks/B2-mol2vec.ipynb', 'repo_name': 'laserkelvin/cheminformatics-tutorial', 'size': '6337'}