<jupyter_start><jupyter_text>Encoding protein sequences with one-hot and AAindex encodingThis notebook shows how to encode protein sequences with a combination of one-hot and AAindex encoding.The resulting encoded sequences are used as input to the neural networks.<jupyter_code># reload modules before executing code in order to make development and debugging easier
%load_ext autoreload
%autoreload 2
# this jupyter notebook is running inside of the "notebooks" directory
# for relative paths to work properly, we need to set the current working directory to the root of the project
# for imports to work properly, we need to add the code folder to the system path
import os
from os.path import abspath, join, isdir
import sys
if not isdir("notebooks"):
    # if there's a "notebooks" directory in the cwd, we've already set the cwd so no need to do it again
    os.chdir("..")
module_path = abspath("code")
if module_path not in sys.path:
    sys.path.append(module_path)
import numpy as np
import constants
import encode as enc<jupyter_output><empty_output><jupyter_text>The main function for encoding variants is `enc.encode()`. It supports encoding variants specified in two forms: as a full sequence of amino acids or as a list of mutations in the form &lt;current_aa&gt;&lt;position&gt;&lt;replacement_aa&gt;.  Encode a full sequenceEncoding a variant specified as a full sequence is straightforward. Let's say you want to encode a GB1 variant where the amino acid in position 3 (0-indexed) has been mutated from a K to an A. The full sequence of this variant is:> "MQYALILNGKTLKGETTTEAVDAATAEKVFKQYANDNGVDGEWTYDDATKTFTVTE"<jupyter_code>char_seq = "MQYALILNGKTLKGETTTEAVDAATAEKVFKQYANDNGVDGEWTYDDATKTFTVTE"
K3A = enc.encode(encoding="one_hot,aa_index", char_seqs=char_seq)
K3A<jupyter_output><empty_output><jupyter_text>Encode multiple sequences You can also specify a list of sequences to encode.<jupyter_code>char_seqs = ["MQYALILNGKTLKGETTTEAVDAATAEKVFKQYANDNGVDGEWTYDDATKTFTVTE",
             "MLYALILNGKTLKGETTTEAVDAATAEKVFKQYANDNGVDGEWTYDDATKTFTVTE"]
encoded_seqs = enc.encode(encoding="one_hot,aa_index", char_seqs=char_seqs)
encoded_seqs.shape<jupyter_output><empty_output><jupyter_text>Dimension 1 corresponds to the number of variants, dimension 2 corresponds to the number of amino acids, and dimension 3 corresponds to the number of features for each amino acid (21 for one-hot encoding and 19 for AAindex for a total of 40).  Use a different encoding If you want to test only one-hot or only AAindex encoding, just specify the single encoding in the function call.<jupyter_code>char_seq = "MLYKLILNGKTLKGETTTEAVDAATAEKVFKQYANDNGVDGEWTYDDATKTFTVTE"
Q1L = enc.encode(encoding="one_hot", char_seqs=char_seq)
display(Q1L.shape)
display(Q1L)<jupyter_output><empty_output><jupyter_text>Encode lists of mutationsA more convenient way to specify variants is as a list of mutations in the form &lt;current_aa&gt;&lt;position&gt;&lt;replacement_aa&gt;. The variant from the previous section where the amino acid in position 3 (0-indexed) has been mutated from a K to an A can be simply specified as> "K3A"Multiple mutations in a single variant can be separated by commas:> "Q1L,K3A"If you are specifying a list of mutations, you must also specify the wild type sequence `wt_aa` and offset `wt_offset`. The offset is needed if your sequence has a non-zero starting point. For example, if your sequence starts at position 126, and you have variants starting with "M126", you must specify an offset of 126. This helps maintain 0-based indexing in the internal code.<jupyter_code>variants = ["K3A", "Q1L,K3A"]  # encode two variants
gb1_wt = "MQYKLILNGKTLKGETTTEAVDAATAEKVFKQYANDNGVDGEWTYDDATKTFTVTE"
gb1_offset = 0
encoded_variants = enc.encode(encoding="one_hot,aa_index", variants=variants, wt_aa=gb1_wt, wt_offset=gb1_offset)
encoded_variants.shape<jupyter_output><empty_output><jupyter_text>Verify that these encoded variants are the same as the encoded sequences from above.<jupyter_code>np.all(encoded_variants == encoded_seqs)<jupyter_output><empty_output><jupyter_text>Rather than specifying the wild-type sequence and offset manually, you can define them in `constants.py` and simply specify the dataset name `ds_name` when calling `enc.encode()`.<jupyter_code>encoded_variants = enc.encode(encoding="one_hot,aa_index", variants=variants, ds_name="gb1")
encoded_variants.shape<jupyter_output><empty_output><jupyter_text>We have added entries for all of our datasets in `constants.py`. You can also add your own dataset by matching the format we used for our datasets.<jupyter_code><jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06676750216076059, 'fraction_numerical': 0.009075194468452896, 'mean_word_length': 6.442122186495177, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 36, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '225776', 'license': 'MIT', 'n_tokens_mistral': 1499, 'n_tokens_neox': 1430, 'n_words': 528, 'path': 'notebooks/data_encoding.ipynb', 'repo_name': 'boelkn/nn4dms', 'size': '4628'}