<jupyter_start><jupyter_text>Training a BDTNow we will train a BDT on the data using engineered variables from Charlie and UCSB. The model will be built using Scikit-Learn's Gradient Boosting classifier to best mimic Charlie's BDT.```pythonparams = {}  BDT Parametersbdt = GradientBoostingClassifier(**params)bdt.fit(X, y)```<jupyter_code>%matplotlib inline
from __future__ import print_function, division
import pandas as pd
from sklearn.ensemble import GradientBoostingClassifier
import pickle
best_5 = ["Top Mass", "(b, j2) mass", "(b, j3) mass", "Top Pt", "W ptDR"]
best_10 = []
best_15 = []
best_20 = []
df_train = pd.read_csv("ttH_hadT_cut_train.csv", header=0, index_col=0)
df_test = pd.read_csv("ttH_hadT_cut_test.csv", header=0, index_col=0)
df_raw_train = pd.read_csv("ttH_hadT_cut_raw_train.csv", header=0, index_col=0)
df_raw_test = pd.read_csv("ttH_hadT_cut_raw_test.csv", header=0, index_col=0)
train_X = df_train.iloc[:, 1:]
train_y = df_train.iloc[:, 0]

train5_X = df_train.loc[:, best_5]
train5_y = df_train.iloc[:, 0]

raw_train_X = df_raw_train.iloc[:, 1:]
raw_train_y = df_raw_train.iloc[:, 0]

test_X = df_test.iloc[:, 1:]
test_y = df_test.iloc[:, 0]

test5_X = df_test.loc[:, best_5]
test5_y = df_test.iloc[:, 0]

raw_test_X = df_raw_test.iloc[:, 1:]
raw_test_y = df_raw_test.iloc[:, 0]<jupyter_output><empty_output><jupyter_text>Train a BDT on all features<jupyter_code>params = dict(max_depth=8, learning_rate=0.1, n_estimators=1000, min_samples_leaf=0.045, subsample=0.5, min_samples_split=20)
bdt = GradientBoostingClassifier(**params).fit(train_X, train_y)
bdt.score(test_X, test_y)*100
with open("bdt.pkl", 'wb') as f:
    pickle.dump(bdt, f)<jupyter_output><empty_output><jupyter_text>Train a BDT on the 5 best engineered features<jupyter_code>params = dict(max_depth=8, learning_rate=0.1, n_estimators=1000, min_samples_leaf=0.045, subsample=0.5, min_samples_split=20)
bdt5 = GradientBoostingClassifier(**params).fit(train5_X, train5_y)
bdt5.score(test5_X, test5_y)*100
with open("bdt_eng5.pkl", 'wb') as f:
    pickle.dump(bdt5, f)<jupyter_output><empty_output><jupyter_text>Train a BDT on the basic features<jupyter_code>params = dict(max_depth=8, learning_rate=0.1, n_estimators=1000, min_samples_leaf=0.045, subsample=0.5, min_samples_split=20)
basic_bdt = GradientBoostingClassifier(**params).fit(raw_train_X, raw_train_y)
basic_bdt.score(raw_test_X, raw_test_y)*100
with open("basic_bdt.pkl", 'wb') as f:
    pickle.dump(basic_bdt, f)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13754498200719711, 'fraction_numerical': 0.03838464614154338, 'mean_word_length': 7.627586206896551, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '51591', 'license': 'MIT', 'n_tokens_mistral': 1092, 'n_tokens_neox': 1038, 'n_words': 194, 'path': 'Training/BDT/had_top_bdt.ipynb', 'repo_name': 'mdkdrnevich/DeepHadTopTagger', 'size': '2501'}