<jupyter_start><jupyter_text>SPORF Tutorial The purpose of this tutorial is to prove that this pure python implementation of SPORF is identical, in terms of functionality, to the one used in the SPORF paper (<NAME>., et al. "Sparse projection oblique randomer forests." Journal of Machine Learning Research 21.104 (2020): 1-39.). In order to do this, this notebook runs this implementation of SPORF on 3 different data sets: hill valley, acute inflammation task 1, and acute inflammation task 2. Cohen's Kappa (fractional decrease in error rate over the chance error rate) is the metric that is being used to compare the implementations. If this implementation has the same kappa values (for the same data sets) as the one in the SPORF paper, we can say with confidence that this implementation is accurate. The datasets used in this notebook all had kappa values of 100 ± 0 in the SPORF paper implementation, which is also what is found when run on this SPORF implementation, as seen below. Thus, we can say with confidence that this implementation of SPORF is accurate.  Import required packages<jupyter_code>import sys
import numpy as np
import pandas as pd

from proglearn.progressive_learner import ProgressiveLearner
from proglearn.forest import LifelongClassificationForest
from proglearn.voters import TreeClassificationVoter
from proglearn.transformers import TreeClassificationTransformer
from proglearn.transformers import ObliqueTreeClassificationTransformer
from proglearn.deciders import SimpleArgmaxAverage

from sklearn.model_selection import train_test_split, cross_val_score

from functions.sporf_datasets_functions import *<jupyter_output><empty_output><jupyter_text>SPORF Set parameters and run on hill valley without noise data<jupyter_code>max_depth = 10
feature_combinations = 2
density = 0.01
reps = 5
n_trees = 10
task_num = 1

kwargs = {"kwargs" : {"max_depth" : max_depth, "feature_combinations" : feature_combinations, "density" : density}}

kappa, err = test("https://archive.ics.uci.edu/ml/machine-learning-databases/hill-valley/Hill_Valley_without_noise_Training.data", reps, n_trees, task_num,
                            ObliqueTreeClassificationTransformer,
                            kwargs)

print("kappa: ", kappa, ", error:", err)<jupyter_output>Accuracy after iteration  0 :  1.0
Accuracy after iteration  1 :  1.0
Accuracy after iteration  2 :  1.0
Accuracy after iteration  3 :  1.0
Accuracy after iteration  4 :  1.0
kappa:  100.0 , error: 0.0<jupyter_text>Set parameters and run on acute inflammation task 1 data<jupyter_code>max_depth = 10
feature_combinations = 1.5
density = 0.5
reps = 5
n_trees = 10
task_num = 1

kwargs = {"kwargs" : {"max_depth" : max_depth, "feature_combinations" : feature_combinations, "density" : density}}

kappa, err = test("https://archive.ics.uci.edu/ml/machine-learning-databases/acute/diagnosis.data", reps, n_trees, task_num,
                            ObliqueTreeClassificationTransformer,
                            kwargs)

print("kappa: ", kappa, ", error:", err)<jupyter_output>Accuracy after iteration  0 :  1.0
Accuracy after iteration  1 :  1.0
Accuracy after iteration  2 :  1.0
Accuracy after iteration  3 :  1.0
Accuracy after iteration  4 :  1.0
kappa:  100.0 , error: 0.0<jupyter_text>Set parameters and run on acute inflammation task 2 data<jupyter_code>max_depth = 10
feature_combinations = 1.5
density = 0.5
reps = 5
n_trees = 10
task_num = 2

kwargs = {"kwargs" : {"max_depth" : max_depth, "feature_combinations" : feature_combinations, "density" : density}}

kappa, err = test("https://archive.ics.uci.edu/ml/machine-learning-databases/acute/diagnosis.data", reps, n_trees, task_num,
                            ObliqueTreeClassificationTransformer,
                            kwargs)

print("kappa: ", kappa, ", error:", err)<jupyter_output>Accuracy after iteration  0 :  1.0
Accuracy after iteration  1 :  1.0
Accuracy after iteration  2 :  1.0
Accuracy after iteration  3 :  1.0
Accuracy after iteration  4 :  1.0
kappa:  100.0 , error: 0.0<jupyter_text>Random Forest (RF)Now we will run the same datasets on a base Random forest. The goal of this is to show how SPORF can clearly outperform or perform as well as the Random Forest algorithm. As seen by the results below, SPORF has a much higher kappa value, than RF, for the hill valley without noise data and has the same value for the acute inflammation data sets. Having a high kappa value is desired since as mentioned above, it is a measure of how much the error rate over the chance error rate decreases.  Set parameters and run on hill valley without noise data<jupyter_code>max_depth = 10
feature_combinations = 2
density = 0.01
reps = 5
n_trees = 10
task_num = 1

kwargs = {"kwargs" : {"max_depth" : max_depth} }

kappa, err = test("https://archive.ics.uci.edu/ml/machine-learning-databases/hill-valley/Hill_Valley_without_noise_Training.data", reps, n_trees, task_num,
                            TreeClassificationTransformer,
                            kwargs)

print("kappa: ", kappa, ", error:", err)<jupyter_output>Accuracy after iteration  0 :  0.5409836065573771
Accuracy after iteration  1 :  0.5901639344262295
Accuracy after iteration  2 :  0.5901639344262295
Accuracy after iteration  3 :  0.6885245901639344
Accuracy after iteration  4 :  0.5245901639344263
kappa:  17.37704918032787 , error: 5.1130724431784715<jupyter_text>Set parameters and run on acute inflammation task 1 data<jupyter_code>max_depth = 10
feature_combinations = 1.5
density = 0.5
reps = 5
n_trees = 10
task_num = 1

kwargs = {"kwargs" : {"max_depth" : max_depth} }

kappa, err = test("https://archive.ics.uci.edu/ml/machine-learning-databases/acute/diagnosis.data", reps, n_trees, task_num,
                            TreeClassificationTransformer,
                            kwargs)

print("kappa: ", kappa, ", error:", err)<jupyter_output>Accuracy after iteration  0 :  1.0
Accuracy after iteration  1 :  1.0
Accuracy after iteration  2 :  1.0
Accuracy after iteration  3 :  1.0
Accuracy after iteration  4 :  1.0
kappa:  100.0 , error: 0.0<jupyter_text>Set parameters and run on acute inflammation task 2 data<jupyter_code>max_depth = 10
feature_combinations = 1.5
density = 0.5
reps = 5
n_trees = 10
task_num = 2

kwargs = {"kwargs" : {"max_depth" : max_depth} }

kappa, err = test("https://archive.ics.uci.edu/ml/machine-learning-databases/acute/diagnosis.data", reps, n_trees, task_num,
                            TreeClassificationTransformer,
                            kwargs)

print("kappa: ", kappa, ", error:", err)<jupyter_output>Accuracy after iteration  0 :  1.0
Accuracy after iteration  1 :  1.0
Accuracy after iteration  2 :  1.0
Accuracy after iteration  3 :  1.0
Accuracy after iteration  4 :  1.0
kappa:  100.0 , error: 0.0
{'fraction_non_alphanumeric': 0.08063566804002355, 'fraction_numerical': 0.04576221306650971, 'mean_word_length': 3.841168091168091, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '72520', 'license': 'MIT', 'n_tokens_mistral': 2311, 'n_tokens_neox': 2047, 'n_words': 712, 'path': 'docs/tutorials/sporf_datasets.ipynb', 'repo_name': 'emilyachang/progressive-learning', 'size': '6805'}