<jupyter_start><jupyter_text>Cafe walkthrough---This notebook walks through on how to use `cafe` and its two main modules: `data_transform` and `ml`.The data used in this example are purely for demonstration purposes. The labels were generated randomly.  Import all necessary libraries---<jupyter_code>from cafe.data_transform import read_fasta, combine_seq, saar, convert_to_values
from cafe.ml import classify

import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline<jupyter_output>Using TensorFlow backend.<jupyter_text>Data loading and transformation---1. Read in the FASTA files using the `read_fasta()` function which returns a `pandas` dataframe so any `pandas`-related functions can be used.<jupyter_code># Load the FASTA files into a pandas dataframe
hc_df = read_fasta('data/D12VH.66.fasta')
lc_df = read_fasta('data/D12VL.20.fasta')<jupyter_output><empty_output><jupyter_text>**The user should make sure all the sequences are aligned as there is no function at the moment to perform sequence alignment.**<jupyter_code># Glimpse the first 5 rows
hc_df.head()
lc_df.head()<jupyter_output><empty_output><jupyter_text>2. Use the `combine_seq()` function to get the different combinations of the entire amino acid sequence.    `combine_seq()` takes two `pandas` dataframes as its arguments.    *This function is not necessary if the entire sequence is known.*<jupyter_code>comb_df = combine_seq(hc_df['Sequence'], lc_df['Sequence']) # Specify the column for each dataframe
comb_df.head()<jupyter_output><empty_output><jupyter_text>3. Here, we perform single amino acid representation transformation where the `saar()` function splits every amino acid in the combined sequence into their own columns, creating a high-dimensional dataset.    **It is important to specify the column of the dataframe when using `saar()`.**<jupyter_code>saar_df = saar(comb_df['Combined Sequence']) # Specify the column
saar_df.head()<jupyter_output><empty_output><jupyter_text>4. Next, we can use the `convert_to_values()` function to create several dataframes (in this case, 7) that each have the amino acids replaced with the values from values.xlsx as well as its associated sheet names.    These dataframes represent our features, i.e., the first dataframe represents AA2INT features and so on.    **The user is responsible for creating this Excel file as it is assumed that the user is interested in testing out certain antibody properties (either from literature or from the lab).**    **Additionally, please follow the format of how the Excel file is structured in the data folder as the user may experience errors if he/she does not do so.**<jupyter_code>feature_list, properties = convert_to_values(saar_df, 'data/values.xlsx')
print('There are {} dataframes.'.format(len(feature_list)))
print(properties)<jupyter_output>There are 7 dataframes.
['AA2INT', 'BIOPHYSICAL', 'HYDROPHOBICITY', 'ISOELECTRIC', 'SASA', 'VOLUME', 'WEIGHT']<jupyter_text>Here, `feature_list[0]` corresponds to AA2INT from `properties`, `feature_list[1]` corresponds to BIOPHYSICAL, and so on.<jupyter_code>aa2int_df = feature_list[0]
aa2int_df.head()
# Check the dimensions of the dataframe (rows, columns)
aa2int_df.shape
hydrophobicity_df = feature_list[2]
hydrophobicity_df.head()
hydrophobicity_df.shape<jupyter_output><empty_output><jupyter_text>Now, read in the Excel file of the labels.**Again, the user is responsible for having this Excel file beforehand.**<jupyter_code>labels_df = pd.read_excel('data/fake_labels.xlsx')
labels_df.head()<jupyter_output><empty_output><jupyter_text>**MAKE SURE THAT `labels_df` AND THE DATAFRAMES IN `df_list` HAVE THE SAME NUMBER OF ROWS.**<jupyter_code># Check the dimensions of the labels dataframe.
labels_df.shape<jupyter_output><empty_output><jupyter_text>Data visualization---We can plot the labels to see the distribution.<jupyter_code>sns.countplot(labels_df['Labels'])
plt.show()
# plt.savefig('class_count.png', dpi = 1200) # For saving the plot<jupyter_output><empty_output><jupyter_text>Classification---5. Now, we can run the `classify()` function to perform machine learning classification.        First:    `classify()` tests 5 machine learning models with their default parameters:    - [Random Forest](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html)    - [Gradient Boosting](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingClassifier.htmlsklearn.ensemble.GradientBoostingClassifier)    - [Decision Tree](https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.htmlsklearn.tree.DecisionTreeClassifier)    - [Extremely Randomized Tree](https://scikit-learn.org/stable/modules/generated/sklearn.tree.ExtraTreeClassifier.htmlsklearn.tree.ExtraTreeClassifier)    - [Logistic Regression with Stochastic Gradient Descent learning](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html)    Specifying `model = 'all'` will test all 5 models and selects the one with the highest score based on the `scoring` argument.        Here is a [link](https://scikit-learn.org/stable/modules/model_evaluation.htmlscoring-parameter) for `scoring` callables.    The user can also specify a specific model he/she wants to try.    For:        - Random Forest, specify `model = 'rf'`        - Gradient Boosting, specify `model = 'gbm'`        - Decision Tree, specify `model = 'dt'`        - Extremely Randomized Tree, specify `model = 'et'`        - Logistic Regression with SGD, specify `model = 'log_sgd'`           Second:         The user can also specify the resampling method if the data is imbalanced by setting `resample_method` to either [`smote_tomek`](https://imbalanced-learn.readthedocs.io/en/stable/under_sampling.htmltomek-links) or [`smote_enn`](https://imbalanced-learn.readthedocs.io/en/stable/under_sampling.htmledited-data-set-using-nearest-neighbours) (the default is `None`).      Third:       `classify()` will return     - the tuned classifier (using [random search](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.RandomizedSearchCV.html))    - a report showing the precision, recall, and f1-score for each class    - [Matthew Correlation Coefficient](https://lettier.github.io/posts/2016-08-05-matthews-correlation-coefficient.html)    - the [log loss](http://wiki.fast.ai/index.php/Log_Loss) (cross-entropy loss) value    - feature importance    - a confusion matrix    - a ROC curve<jupyter_code># Test all the properties from the values Excel file
for prop, feature in zip(properties, feature_list):
    
    print()
    print('='*24)
    print(prop.center(24))
    print('='*24)

    tuned_clf, report, mcc, loss_score, feat_imp, conf_mat, roc_curve = classify(feature, 
                                                                                 labels_df['Labels'], # Specify column
                                                                                 resample_method = 'smote_enn',
                                                                                 model = 'all',
                                                                                 scoring = 'roc_auc_ovo',
                                                                                 cv = 10,
                                                                                 n_iter = 10)

    print('\nClassification report')
    print('---------------------')
    print(report)

    print('\nMatthew Correlation Coefficient: {:.4f}'.format(mcc))

    print('\nLog Loss: {:.4f}'.format(loss_score))
    
    print('\nFeature Importance')
    print('------------------')
    print(feat_imp)

    conf_mat
    plt.show()

    roc_curve
    plt.show()<jupyter_output>========================
         AA2INT         
========================

Selecting model...

Model	Score
-------------
Random Forest: 0.4390 ± 0.0598
Gradient Boosting: 0.4287 ± 0.0512
Decision Tree: 0.4715 ± 0.0386
Extra Trees: 0.4404 ± 0.0454
Logistic Regression (SGD): 0.4392 ± 0.0315

The selected model is ('Decision Tree', 0.4714514185089511)

Starting training...

The best 10-fold cross valdiation score is 0.5057.
The best parameters are:
 DecisionTreeClassifier(ccp_alpha=0.0, class_weight=None, criterion='gini',
                       max_depth=110, max_features='sqrt', max_leaf_nodes=None,
                       min_impurity_decrease=0.0, min_impurity_split=None,
                       min_samples_leaf=1, min_samples_split=10,
                       min_weight_fraction_leaf=0.0, presort='deprecated',
                       random_state=9999, splitter='random')
Training took 0m 43s.

Classification report
---------------------
              precision    recall  f1-score   sup[...]
{'fraction_non_alphanumeric': 0.1102665910380034, 'fraction_numerical': 0.016789563244469653, 'mean_word_length': 3.4057971014492754, 'pattern_counts': {'":': 0, '<': 42, '<?xml version=': 0, '>': 42, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 18, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '228429', 'license': 'MIT', 'n_tokens_mistral': 2691, 'n_tokens_neox': 2515, 'n_words': 736, 'path': 'examples/cafe_example.ipynb', 'repo_name': 'lin-justin/cafe', 'size': '8815'}