<jupyter_start><jupyter_text>Modified dataset<jupyter_code>features = ['R1', 'R2', 'R3', 'R4', 'R5', 'R5', 'R6', 'R7', 'R8', 'Temp.', 'Humidity']

df_db = group_datafiles_byID('../datasets/preprocessed/HT_Sensor_prep_metadata.dat', '../datasets/preprocessed/HT_Sensor_prep_dataset.dat')
df_db = reclassify_series_samples(df_db)
df_db.head()
# Validación simple de n_iter ejecuciones para decidir la constante de aprendizaje, el número de capas ocultas y
# el número de neuronas
list_alpha = [0.01,0.001]
list_hid_layers = range(2,5)
list_neur = [2,4,8]
n_iter = 5
for alpha in list_alpha:
    print('<<<<<<<<<<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>')
    print('Alpha:',alpha)
    for n_hid_layers in list_hid_layers:
        print('##############################################')
        print('\t Hidden layers:',n_hid_layers)
        for n_neur in list_neur:
            print('==============================================')
            print('\t \t Neurons per layer:',n_neur)
            hyper_sim(df_db,n_iter,n_hid_layers,n_neur,alpha,features)
            print('==============================================')<jupyter_output><<<<<<<<<<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>
Alpha: 0.01
##############################################
	 Hidden layers: 2
==============================================
	 	 Neurons per layer: 2
Train loss: 0.3916246646420336 +- 0.0350024483286658
Accuracy: 0.8284198332623605 +- 0.029885100502695
F1-score: 0.7866374098832829 +- 0.04893277144103149
Recall bananas: 0.11498074622482941 +- 0.1422750013256773
==============================================
==============================================
	 	 Neurons per layer: 4
Train loss: 0.28711141493179976 +- 0.02455610698895882
Accuracy: 0.8344744535174758 +- 0.018372318244468833
F1-score: 0.8231780894289331 +- 0.02653388965763423
Recall bananas: 0.2780092596991882 +- 0.12186590199593533
==============================================
==============================================
	 	 Neurons per layer: 8
Train loss: 0.18114278402911027 +- 0.016025121020812518
Accuracy: 0.7927781405115872 +- 0.04760446139721019
F1-score: 0.79337941531469[...]<jupyter_text>Elegimos 3 capas ocultas, 4 neuronas por capa y un alpha 0.01Ahora probamos a hacer bagging con 20 estimadores.<jupyter_code>n_iter = 5
layers = (4,4,4)
alpha = 0.01

errs_acc = []
errs_f1 = []
rec_ban = []
for i in range(10):
    df_train, df_test = split_series_byID(0.75, df_db)
    df_train, df_test = norm_train_test(df_train,df_test,features_to_norm=features)
    xtrain, ytrain = df_train[features].values, df_train['class'].values
    xtest, ytest = df_test[features].values, df_test['class'].values

    clf_nn = MLPClassifier(
        hidden_layer_sizes=layers,
        max_iter=2000,
        early_stopping=True,
        shuffle=True,
        alpha=alpha,
        learning_rate='adaptive'
        )

    bag = BaggingClassifier(base_estimator=clf_nn,n_estimators=20,n_jobs=3)

    bag.fit(xtrain, ytrain)
    ypred = bag.predict(xtest)
    errs_acc.append(accuracy_score(ytest,ypred))
    errs_f1.append(f1_score(ytest,ypred,average='weighted'))
    rec_ban.append(np.sum(np.logical_and(ytest=='banana',ypred=='banana'))/np.sum(ytest=='banana'))

errs_acc = np.array(errs_acc)
errs_f1 = np.array(errs_f1)
rec_ban = np.array(rec_ban)
print('Accuracy:',np.mean(errs_acc),'+-',np.std(errs_acc))
print('F1-score:',np.mean(errs_f1),'+-',np.std(errs_f1))
print('Recall bananas:',np.mean(rec_ban),'+-',np.std(rec_ban))<jupyter_output>Accuracy: 0.8439662843338537 +- 0.05739720482516567
F1-score: 0.8296699177597638 +- 0.0655385850230548
Recall bananas: 0.25925942717612216 +- 0.08496264509329657<jupyter_text>Windows<jupyter_code>features = ['R1', 'R2', 'R3', 'R4', 'R5', 'R6', 'R7', 'R8', 'Temp.', 'Humidity',
            'R1_mean', 'R2_mean', 'R3_mean', 'R4_mean', 'R5_mean', 'R6_mean', 'R7_mean',
            'R8_mean', 'Temp._mean', 'Humidity_mean', 'R1_std', 'R2_std', 'R3_std', 'R4_std',
            'R5_std', 'R6_std', 'R7_std', 'R8_std', 'Temp._std', 'Humidity_std']

win_df = pd.read_pickle('../datasets/preprocessed/window120_dataset.pkl')
# Validación simple de 5 ejecuciones para decidir la constante de aprendizaje, el número de capas ocultas y
# el número de neuronas
for alpha in [0.01,0.001]:
    print('<<<<<<<<<<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>')
    print('Alpha:',alpha)
    for n_hid_layers in range(2,5):
        print('##############################################')
        print('\t Hidden layers:',n_hid_layers)
        for n_neur in [2,4,8]:
            print('==============================================')
            print('\t \t Neurons per layer:',n_neur)
            hyper_sim(win_df,5,n_hid_layers,n_neur,alpha,features)
            print('==============================================')
hyper_sim(win_df,10,3,4,0.01,features)<jupyter_output>Train loss: 0.24018066820164438 +- 0.023418848195325886
Accuracy: 0.8637574268900602 +- 0.02871614724926834
F1-score: 0.8553617572371873 +- 0.031009063536019333
Recall bananas: 0.39207973741325436 +- 0.092974568441552<jupyter_text>Al igual que antes, elegimos 3 capas ocultas, 4 neuronas por capa y un alpha 0.01. Aunque hay mejores resultados, tienen demasiada desviación típica como para ser fiables.Ahora probamos a hacer bagging con 20 estimadores.<jupyter_code>errs_acc = []
errs_f1 = []
rec_ban = []
for i in range(10):
    df_train, df_test = split_series_byID(0.75, win_df)
    df_train, df_test = norm_train_test(df_train,df_test,features_to_norm=features)
    xtrain, ytrain = df_train[features].values, df_train['class'].values
    xtest, ytest = df_test[features].values, df_test['class'].values

    clf_nn = MLPClassifier(
        hidden_layer_sizes=(4,4,4),
        max_iter=2000,
        early_stopping=True,
        shuffle=True,
        alpha=0.01,
        learning_rate='adaptive'
        )

    bag = BaggingClassifier(base_estimator=clf_nn,n_estimators=20,n_jobs=3)

    bag.fit(xtrain, ytrain)
    ypred = bag.predict(xtest)
    errs_acc.append(accuracy_score(ytest,ypred))
    errs_f1.append(f1_score(ytest,ypred,average='weighted'))
    rec_ban.append(np.sum(np.logical_and(ytest=='banana',ypred=='banana'))/np.sum(ytest=='banana'))

errs_acc = np.array(errs_acc)
errs_f1 = np.array(errs_f1)
rec_ban = np.array(rec_ban)
print('Accuracy:',np.mean(errs_acc),'+-',np.std(errs_acc))
print('F1-score:',np.mean(errs_f1),'+-',np.std(errs_f1))
print('Recall bananas:',np.mean(rec_ban),'+-',np.std(rec_ban))<jupyter_output>==============================================
TEST SET PROPORTIONS:
	Real background percentage: 0.7869345926926833
	Real banana percentage: 0.09259234484327131
	Real wine percentage: 0.12047306246404538
------------------------------------------
Accuracy: 0.895500822775496
Recall on background: 0.9620033661447442
Recall on banana: 0.6426335307999808
Recall on wine: 0.65545067555062
F1-score: 0.8939343748402855
==============================================
==============================================
TEST SET PROPORTIONS:
	Real background percentage: 0.7070396595801938
	Real banana percentage: 0.09927509418729817
	Real wine percentage: 0.19368524623250807
------------------------------------------
Accuracy: 0.818205900161464
Recall on background: 0.9609517635934368
Recall on banana: 0.16950444726810673
Recall on wine: 0.6296159607494084
F1-score: 0.792719333808237
==============================================
==============================================
TEST SET PROPORTIONS:
	Re[...]<jupyter_text>Smote<jupyter_code>features = ['R1', 'R2', 'R3', 'R4', 'R5', 'R6', 'R7', 'R8', 'Temp.', 'Humidity',
            'R1_mean', 'R2_mean', 'R3_mean', 'R4_mean', 'R5_mean', 'R6_mean', 'R7_mean',
            'R8_mean', 'Temp._mean', 'Humidity_mean', 'R1_std', 'R2_std', 'R3_std', 'R4_std',
            'R5_std', 'R6_std', 'R7_std', 'R8_std', 'Temp._std', 'Humidity_std']

win_df = pd.read_pickle('../datasets/preprocessed/window120_dataset.pkl')

over_dict = {'banana': 175000, 'wine': 175000}
under_dict = {'background': 500000}
# Validación simple de 5 ejecuciones para decidir la constante de aprendizaje, el número de capas ocultas y
# el número de neuronas
for alpha in [0.01,0.001]:
    print('<<<<<<<<<<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>')
    print('Alpha:',alpha)
    for n_hid_layers in range(2,5):
        print('##############################################')
        print('\t Hidden layers:',n_hid_layers)
        for n_neur in [2,4,8]:
            print('==============================================')
            print('\t \t Neurons per layer:',n_neur)
            hyper_sim(win_df,5,n_hid_layers,n_neur,alpha,features,over_dict=over_dict,under_dict=under_dict)
            print('==============================================')
# Mejor opción
hyper_sim(win_df,10,3,4,0.01,features,over_dict=over_dict,under_dict=under_dict)
errs_acc = []
errs_f1 = []
rec_ban = []
for i in range(5):
    df_train, df_test = split_series_byID(0.75, win_df)
    df_train, df_test = norm_train_test(df_train,df_test,features_to_norm=features)
    xtrain, ytrain = df_train[features].values, df_train['class'].values
    over_sampling = SMOTE(sampling_strategy=over_dict)
    under_sampling = RandomUnderSampler(sampling_strategy=under_dict)
    xtrain, ytrain = over_sampling.fit_resample(xtrain, ytrain)
    xtrain, ytrain = under_sampling.fit_resample(xtrain, ytrain)
    xtest, ytest = df_test[features].values, df_test['class'].values

    clf_nn = MLPClassifier(
        hidden_layer_sizes=(4,4,4),
        max_iter=2000,
        early_stopping=True,
        shuffle=True,
        alpha=0.01,
        learning_rate='adaptive'
        )

    bag = BaggingClassifier(base_estimator=clf_nn,n_estimators=20,n_jobs=3)

    bag.fit(xtrain, ytrain)
    ypred = bag.predict(xtest)
    errs_acc.append(accuracy_score(ytest,ypred))
    errs_f1.append(f1_score(ytest,ypred,average='weighted'))
    rec_ban.append(np.sum(np.logical_and(ytest=='banana',ypred=='banana'))/np.sum(ytest=='banana'))

errs_acc = np.array(errs_acc)
errs_f1 = np.array(errs_f1)
rec_ban = np.array(rec_ban)
print('Accuracy:',np.mean(errs_acc),'+-',np.std(errs_acc))
print('F1-score:',np.mean(errs_f1),'+-',np.std(errs_f1))
print('Recall bananas:',np.mean(rec_ban),'+-',np.std(rec_ban))<jupyter_output>==============================================
TEST SET PROPORTIONS:
	Real background percentage: 0.7261303338023632
	Real banana percentage: 0.1202765016020479
	Real wine percentage: 0.15359316459558886
------------------------------------------
Accuracy: 0.8245588010917659
Recall on background: 0.8768101138744754
Recall on banana: 0.6473801050072236
Recall on wine: 0.716280353200883
F1-score: 0.8326610440983933
==============================================
==============================================
TEST SET PROPORTIONS:
	Real background percentage: 0.7762145091408189
	Real banana percentage: 0.10652743536457862
	Real wine percentage: 0.11725805549460241
------------------------------------------
Accuracy: 0.873000916448311
Recall on background: 0.9137284029998836
Recall on banana: 0.4742921765822529
Recall on wine: 0.9656184640223094
F1-score: 0.8733368939127423
==============================================
==============================================
TEST SET PROPORTIONS:
	R[...]
{'fraction_non_alphanumeric': 0.25197409280454264, 'fraction_numerical': 0.12439002750421435, 'mean_word_length': 4.998935604044704, 'pattern_counts': {'":': 0, '<': 132, '<?xml version=': 0, '>': 84, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '107741', 'license': 'MIT', 'n_tokens_mistral': 4992, 'n_tokens_neox': 4096, 'n_words': 667, 'path': 'tests/neural_networks_bien.ipynb', 'repo_name': 'AlejandroSantorum/Gas_sensors_home_activity_monitoring', 'size': '11271'}