<jupyter_start><jupyter_text>PCA + MLP approach - stock 9265For LOB we perform $GDF(k, r, s)$ filtering on concated **bid and ask lists** with K=50 and $r \in {0.01, 0.1, 1.0}$, $ s \in {0.1, 1.0} $. We obtain 50 gdf features with values between 0 and 1.We also introduce:* `queue_imbalance`* `prev_queue_imbalance` (for sample from tick $t$ `prev_queue_imbalance` is `queue_imbalance` value from previous tick $t-1$). In total our feature set is:* `queue_imbalance`* `prev_queue_imbalance`* 50 features: $gdf_i$ for $i \in (0, 50) $Next we perform dimensionality reduction using PCA(n_components=10) on these features. We do NOT perform mean normalization since queue imbalance and prev queue imbalance are between (-1, 1) and all gdf features are between (0, 1). We want queue imbalance to have bigger inpact on the result of the reduction.Next we input 10 features obtained after PCA to MLP algorithm.<jupyter_code>%load_ext autoreload

%autoreload 2
%matplotlib inline
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.cluster import KMeans
from sklearn.svm import SVC
from sklearn.metrics import roc_auc_score, roc_curve

from sklearn import preprocessing
from sklearn.linear_model import LogisticRegression
from sklearn.decomposition import PCA

import warnings
import numpy as np
from collections import OrderedDict

import os

from lob_data_utils import lob, db_result, gdf_pca, model
from lob_data_utils.svm_calculation import lob_svm


sns.set_style('whitegrid')
warnings.filterwarnings('ignore')
data_length = 15000
stock = '9265'
gdf_filename_pattern = 'gdf_{}_' + 'len{}'.format(data_length) + '_r{}_s{}_K50'
gdf_parameters = [(1.0, 1.0), (0.1, 0.1), (0.1, 1.0), (1.0, 0.1), (0.01, 0.1)] # pairs of params (r, s)
gdf_dfs = []
for r, s in gdf_parameters:
    gdf_dfs.append(gdf_pca.SvmGdfResults(
        stock,  r=r, s=s, data_length=data_length, 
        gdf_filename_pattern=gdf_filename_pattern))
def get_kernel_from_method(method):
    return method.split('_')[1]

class CFR():
    def __init__(self, clf, feature_name, is_svm=True):
        self.clf = clf
        self.feature_name = feature_name
        self.is_svm = is_svm

    def get_result(self, gdf_df, should_validate=False, method=None):
        if self.is_svm:
            return gdf_df.train_svm(C=self.clf.C, gamma=self.clf.gamma, coef0=self.clf.coef0, 
                                    kernel=self.clf.kernel, 
                                    feature_name=self.feature_name, should_validate=should_validate)
        else:
            res = gdf_df.train_clf(self.clf, self.feature_name, should_validate=should_validate)
            if method:
                res['kernel'] = method
            return res<jupyter_output><empty_output><jupyter_text>PCA VisualizationI'm not sure how to visualize PCA.<jupyter_code>train_x = gdf_dfs[3].df[gdf_dfs[3].feature_columns_dict['pca_gdf_que_prev10']]
n_components = 10
pca_columns = [f'pca_{i}' for i in range(0, n_components)]


pca = PCA(n_components=n_components)
pca.fit(train_x)
train_x = pca.transform(train_x)
df_pca = pd.DataFrame(data=train_x, columns=pca_columns)
df_pca.index = gdf_dfs[3].df.index
df_pca = pd.concat([df_pca, gdf_dfs[3].df], axis=1)
df_pca[pca_columns + ['gdf_20', 'gdf_30', 'queue_imbalance']].head()
plt.scatter(df_pca.loc[:, 'pca_0'], df_pca.loc[:, 'pca_1'], 
            c=df_pca['mid_price_indicator'], s=1,
           cmap='viridis')
plt.legend()
plt.xlabel('PCA 0')
plt.ylabel('PCA 1')
plt.figure(figsize=(12, 12))
sns.heatmap(df_pca[pca_columns + ['queue_imbalance', 'prev_queue_imbalance', 'mid_price_indicator']].corr(), 
            annot=True)
df_pca[pca_columns].iloc[0:10].plot(figsize=(16, 4))
plt.title('PCA over time')
df_pca.iloc[0:10][[f'gdf_{i}' for i in range(20, 30)]].plot(figsize=(16, 4), legend=None)
plt.title('GDF 20-30 over time')
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(16, 4))
ax1.bar(pca_columns, pca.explained_variance_)
ax1.set_title('Explained variance for PCA')
ax2.bar(pca_columns, pca.explained_variance_ratio_)
ax2.set_title('Explained variance ration for PCA')
print('Explained variance:', pca.explained_variance_)
print('Explained variance ratio: ', pca.explained_variance_ratio_)<jupyter_output>Explained variance: [0.27603016 0.22182191 0.00288695 0.00228761 0.00108238 0.00082394
 0.00055541 0.00047809 0.00037898 0.00030596]
Explained variance ratio:  [0.54325739 0.43656966 0.00568184 0.00450226 0.00213024 0.00162161
 0.00109311 0.00094094 0.00074587 0.00060216]<jupyter_text>MLPWe tried different sets of features as inputs to MLP:* pca_gdf_que_prev10 - we obtain them by reducting all GDF features and queue imbalance and prev queue imbalance using PCA to 10 dimensions* pca_gdf_que10 - all GDF and queue_imbalance reducted to 10 features using PCA* pca_gdf10 - all GDF features reducted to 10 features using PCAAll GDF features are a set of $gdf_i$ for $i \in (0, 50)$Output: mid price indicator Hyperparameters* r - parameter for GDF* s - parameter for GDF* alpha - L2 penalty parameter from set $ [1.0, 0.1, 0.01, 0.0001, 0.001]$* hidden_layer_size - number of hidden layers from set ${(8, 8), (8, 16), (8), (16), (16, 16), (16, 8)}$* activation function - tanh* solver - adam ValidationWe used 10-fold forward anchored cross validation:![val](https://raw.githubusercontent.com/vevurka/mt-lob/master/pictures_for_mt/pic_base/forward_anchored_validation.png)We measure MCC (Matthews Correlation Coefficient) and ROC area score. Choice of best classifier is done using MCC instead of ROC area score, since MCC is not prone to class imbalance issues (different amounts of positives and negatives in data set). Of course for most of the stocks we observed that classes are usually balanced. But also usually classifier with highest MCC score on validation set has also the highest ROC area score.<jupyter_code>df_res = pd.read_csv(f'res_{stock}_mlp.csv')
columns = ['r','s', 'alpha', 'hidden_layer_size', 'features', 'matthews', 'roc_auc', 
           'test_matthews', 'test_roc_auc', 'train_matthews', 'train_roc_auc']
df_res = df_res[columns].sort_values(by='matthews', ascending=False)
df_res.head()<jupyter_output><empty_output><jupyter_text>In the table above `matthews` is a MCC validation score, `roc_auc` is ROC area validation score. These are 5 best results - the best one is on the top, they are sorted by MCC. We recorded test scores for all classifiers, but they were not used for comparision.<jupyter_code>d = df_res[df_res['features'] == 'pca_gdf_que_prev10'][columns].groupby(
    ['r', 's']).head(1).sort_values(by=['matthews'], ascending=False)

fig, (ax0, ax1) = plt.subplots(1, 2, figsize=(16, 4))

sns.heatmap(data=d[[ 'r', 's', 'train_matthews']].pivot(
    index='r', columns='s', values=['train_matthews'])['train_matthews'], ax=ax0, annot=True)
ax0.set_title('Train MCC score for {}'.format(stock))
data = d[[ 'r', 's', 'matthews']].pivot(
    index='r', columns='s', values=['matthews'])['matthews']
sns.heatmap(data=data,  ax=ax1, annot=True)
ax1.set_title('Validation MCC score for {}'.format(stock))<jupyter_output><empty_output><jupyter_text>Heatmaps with score comparision between different GDF parameters r and s.<jupyter_code>log_columns = ['matthews', 'roc_auc', 'test_matthews', 'test_roc_auc', 'train_matthews', 'train_roc_auc']

df_res_log = pd.read_csv('res_overview_all_three_logistic_regression.csv')
df_res_log = df_res_log[df_res_log['stock'] == int(stock)]
df_res_log = df_res_log[log_columns].sort_values(by='matthews', ascending=False)
df_res_log<jupyter_output><empty_output><jupyter_text>This is the result obtained by using Logistic Regression with Queue Imbalance as a feature.<jupyter_code>df_res_best = df_res.head(1)
for c in df_res_log.columns:
    df_res_best[f'log_{c}'] = df_res_log[c].values
df_res_best<jupyter_output><empty_output><jupyter_text>Data frame with best PCA+GDF+MLP and logistic regression scores (there have prefix log_) to make some plots with these results.<jupyter_code>bar_columns = ['matthews', 'log_matthews', 'test_matthews', 
               'log_test_matthews', 'train_matthews', 'log_train_matthews']
df_res_best[bar_columns].unstack().plot(kind='bar')
x = plt.xticks(list(range(len(bar_columns))), bar_columns)

plt.title('MCC score comparision for the MLP classifier and Logistic Regression')
df_res_best[bar_columns]
bar_columns = ['roc_auc', 'log_roc_auc', 'test_roc_auc', 
               'log_test_roc_auc', 'train_roc_auc', 'log_train_roc_auc']
df_res_best[bar_columns].unstack().plot(kind='bar')
x = plt.xticks(list(range(len(bar_columns))), bar_columns)
plt.title('ROC score comparision for the MLP classifier and Logistic Regression')
df_res_best[bar_columns]<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1230945558739255, 'fraction_numerical': 0.042406876790830945, 'mean_word_length': 5.364697301239971, 'pattern_counts': {'":': 0, '<': 28, '<?xml version=': 0, '>': 28, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '325034', 'license': 'MIT', 'n_tokens_mistral': 3430, 'n_tokens_neox': 3113, 'n_words': 800, 'path': 'overview_val10/overview_9265_full-mlp.ipynb', 'repo_name': 'vevurka/mt-lob', 'size': '8725'}