<jupyter_start><jupyter_text>Sixthbatch size 2048 lr 1e-3 swish mult2, filtered HL features  Import modules<jupyter_code>%matplotlib inline
from __future__ import division
import sys
import os
os.environ['MKL_THREADING_LAYER']='GNU'
sys.path.append('../')
from Modules.Basics import *
from Modules.Class_Basics import *<jupyter_output>/home/giles/anaconda2/lib/python2.7/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
/home/giles/anaconda2/lib/python2.7/site-packages/statsmodels/compat/pandas.py:56: FutureWarning: The pandas.core.datetools module is deprecated and will be removed in a future version. Please use the pandas.tseries module instead.
  from pandas.core import datetools
Using TensorFlow backend.<jupyter_text>Options<jupyter_code>classTrainFeatures = filtered_features_0
classModel = 'modelSwish'
varSet = "filtered_features_0"

nSplits = 10
ensembleSize = 10
ensembleMode = 'loss'

maxEpochs = 200
compileArgs = {'loss':'binary_crossentropy', 'optimizer':'adam'}
trainParams = {'epochs' : 1, 'batch_size' : 2048, 'verbose' : 0}
modelParams = {'version':classModel, 'nIn':len(classTrainFeatures), 'compileArgs':compileArgs}

print "\nTraining on", len(classTrainFeatures), "features:", [var for var in classTrainFeatures]<jupyter_output>Training on 29 features: ['ntracks', 'ntowers', 'jetMass', 'jetPt', 'max_track_pT', 'max_track_eta', 'min_track_eta', 'sum_tower_Ehad', 'max_tower_E', 'sum_track_pT', 'mean_tower_Eem', 'mean_track_pT', 'tower_Eem_frac', 'min_track_pT', 'sum_tower_Eem', 'min_tower_E', 'jetPhi', 'sum_tower_E', 'min_tower_eta', 'tower_Ehad_frac', 'max_tower_Eem', 'mean_tower_E', 'max_tower_Ehad', 'max_tower_eta', 'mean_tower_eta', 'jetEta', 'mean_tower_Ehad', 'mean_track_eta', 'min_tower_Eem']<jupyter_text>Import data<jupyter_code>trainData = h5py.File(dirLoc + 'train.hdf5', "r+")
valData = h5py.File(dirLoc + 'testing.hdf5', "r+")<jupyter_output><empty_output><jupyter_text>Determine LR<jupyter_code>lrFinder = batchLRFindClassifier(trainData, nSplits, getClassifier, modelParams, trainParams, lrBounds=[1e-7,1e-1], trainOnWeights=False, verbose=0)
compileArgs['lr'] = 5e-2<jupyter_output><empty_output><jupyter_text>Train classifier<jupyter_code>results, histories = batchTrainClassifier(trainData, nSplits, getClassifier, modelParams, trainParams, patience=40, cosAnnealMult=2, trainOnWeights=False, maxEpochs=maxEpochs, verbose=1)<jupyter_output>Using cosine annealing
Running fold 1 / 10
2 classes found, running in binary mode

1 New best found: 0.508108724461
2 New best found: 0.489794467668
3 New best found: 0.48635823451
4 New best found: 0.485745368498
5 New best found: 0.484333938217
6 New best found: 0.483275204111
7 New best found: 0.482808863489
12 New best found: 0.482301944019
14 New best found: 0.482161963763
15 New best found: 0.481994632412
25 New best found: 0.481635074103
27 New best found: 0.481581351159
29 New best found: 0.481316633211
30 New best found: 0.481174811606
31 New best found: 0.481172331273
54 New best found: 0.481048114653
57 New best found: 0.480970660153
58 New best found: 0.480875951349
59 New best found: 0.480838762661
60 New best found: 0.480748042703
61 New best found: 0.480708388922
62 New best found: 0.480659841285
63 New best found: 0.480647975321
Early stopping after 103 epochs
Score is: {'loss': 0.4806479753212906, 'AUC': 0.20351015079544887}
Fold took 333.441s

Running fold 2 / 10
1 N[...]<jupyter_text>Construct ensemble<jupyter_code>with open('train_weights/resultsFile.pkl', 'r') as fin:   
    results = pickle.load(fin)
ensemble, weights = assembleEnsemble(results, ensembleSize, ensembleMode, compileArgs)<jupyter_output>Choosing ensemble by loss
Model 0 is 5 with loss = 0.48061988696352154
Model 1 is 0 with loss = 0.4806479753212906
Model 2 is 3 with loss = 0.48124131055586067
Model 3 is 8 with loss = 0.4812573921754315
Model 4 is 6 with loss = 0.4815456121901997
Model 5 is 4 with loss = 0.4821726593832468
Model 6 is 7 with loss = 0.4824461608940197
Model 7 is 2 with loss = 0.48256393182216895
Model 8 is 9 with loss = 0.4831391624874641
Model 9 is 1 with loss = 0.48375256248171805<jupyter_text>Response on development data<jupyter_code>batchEnsemblePredict(ensemble, weights, trainData, ensembleSize=10, nFolds=nSplits, verbose=1)
print 'Training ROC AUC {}'.format(roc_auc_score(getFeature('targets', trainData, nSplits), getFeature('pred', trainData, nSplits)))<jupyter_output>Training ROC AUC 0.79615706997<jupyter_text>Response on val data<jupyter_code>batchEnsemblePredict(ensemble, weights, valData, ensembleSize=10, nFolds=nSplits, verbose=1)
print 'Testing ROC AUC: {}'.format(roc_auc_score(getFeature('targets', valData, nSplits), getFeature('pred', valData, nSplits)))<jupyter_output>Testing ROC AUC: 0.792439632705<jupyter_text>Evaluation  Import in dataframe<jupyter_code>def convertToDF(datafile, columns={'gen_target', 'gen_weight', 'pred_class'}, nLoad=-1):
    data = pandas.DataFrame()
    data['gen_target'] = getFeature('targets', datafile, nLoad)
    data['pred_class'] = getFeature('pred', datafile, nLoad)
    print len(data), "candidates loaded"
    return data
valData = convertToDF(valData, nLoad=nSplits)
sigVal = (valData.gen_target == 1)
bkgVal = (valData.gen_target == 0)<jupyter_output><empty_output><jupyter_text>MVA distributions<jupyter_code>getClassPredPlot([valData[bkgVal], valData[sigVal]])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11121012297273214, 'fraction_numerical': 0.11923008376403493, 'mean_word_length': 7.180758017492711, 'pattern_counts': {'":': 0, '<': 35, '<?xml version=': 0, '>': 35, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '631419', 'license': 'MIT', 'n_tokens_mistral': 2496, 'n_tokens_neox': 2032, 'n_words': 498, 'path': 'Classifiers/Basic_Features_2048_5e-2_swish_mult2_filteredV0.ipynb', 'repo_name': 'GilesStrong/IML_DataChallenge_2017', 'size': '5611'}