<jupyter_start><jupyter_text>Gaussian XOR and Gaussian XNOR ExperimentOne key goal of omnidirectional learning is to be able to continually improve upon past performance with the introduction of new data, without forgetting too much of the past tasks. This transfer of information can be evaluated using a variety of metrics; however, here, we use a generalization of Pearl's transfer-benefit ratio (TBR) in both the forward and backward directions. As described in [Vogelstein, et al. (2020)](https://arxiv.org/pdf/2004.12908.pdf), the forward transfer efficiency of task $f_n$ for task $t$ given $n$ samples is:$$FTE^t (f_n) := \mathbb{E} [R^t (f^{t}_n )/R^t (f^{<t}_n )]$$If $FTE^t(f_n)>1$, the algorithm demonstrates positive forward transfer, i.e. past task data has been used to improve performance on the current task.Similarly, the backward transfer efficiency of task $f_n$ for task $t$ given $n$ samples is:$$BTE^t (f_n) := \mathbb{E} [R^t (f_n^{<t} )/R^t (f_n)]$$If $BTE^t(f_n)>1$, the algorithm demonstrates positive backward transfer, i.e. data from the current task has been used to improve performance on past tasks. Omnidirectional learning in a simple environment can therefore be demonstrated using two simple tasks: Gaussian XOR and Gaussian Not-XOR (XNOR). Here, forward transfer efficiency is the ratio of generalization errors for XNOR, whereas backward transfer efficiency is the ratio of generalization errors for XOR. These two tasks share the same discriminant boundaries, so learning can be easily transferred between them.This experiment compares the performance of omnidirectional forests to uncertainty forests in undergoing these tasks.<jupyter_code>import numpy as np

import functions.xor_xnor_functions as fn
from proglearn.sims import generate_gaussian_parity<jupyter_output>Using TensorFlow backend.<jupyter_text>**Note:** This notebook tutorial uses functions stored externally within `functions/xor_xnor_functions.py`, to simplify presentation of code. These functions are imported above, along with other libraries.  Classification ProblemFirst, let's visualize Gaussian XOR and N-XOR.Gaussian XOR is a two-class classification problem, where...- Class 0 is drawn from two Gaussians with $\mu = \pm [0.5, 0.5]^T$ and $\sigma^2 = 0.25I$.- Class 1 is drawn from two Gaussians with $\mu = \pm [0.5, -0.5]^T$ and $\sigma^2 = 0.25I$.Gaussian XNOR has the same distribution as Gaussian XOR, but with the class labels flipped, i.e...- Class 0 is drawn from two Gaussians with $\mu = \pm [0.5, -0.5]^T$ and $\sigma^2 = 0.25I$.- Class 1 is drawn from two Gaussians with $\mu = \pm [0.5, 0.5]^T$ and $\sigma^2 = 0.25I$. Within the proglearn package, we can make use of the simulations within the `sims` folder to generate simulated data. The `generate_gaussian_parity` function within `gaussian_sim.py` can be used to create the Gaussian XOR and XNOR problems. Let's generate data and plot it to see what these problems look like!<jupyter_code># call function to return gaussian xor and xnor data:
X_xor, y_xor = generate_gaussian_parity(1000)
X_xnor, y_xnor = generate_gaussian_parity(1000, angle_params=np.pi/2)

# plot and format:
fn.plot_xor_xnor(X_xor, y_xor, 'Gaussian XOR')
fn.plot_xor_xnor(X_xnor, y_xnor, 'Gaussian XNOR')<jupyter_output><empty_output><jupyter_text>The ExperimentNow that we have generated the data, we can prepare to run the experiment. The function for running the experiment, `experiment`, can be found within `functions/xor_xnor_functions.py`.  We first declare the hyperparameters to be used for the experiment, which are as follows:- `mc_rep`: number of repetitions to run the progressive learning algorithm for- `n_test`: number of xor/xnor data points in the test set- `n_trees`: number of trees- `n_xor`: array containing number of xor data points fed to learner, ranges from 50 to 725 in increments of 25- `n_xnor`: array containing number of xnor data points fed to learner, ranges from 50 to 750 in increments of 25<jupyter_code># define hyperparameters:
mc_rep = 1000
n_test = 1000 
n_trees = 10
n_xor = (100*np.arange(0.5, 7.50, step=0.25)).astype(int)
n_xnor = (100*np.arange(0.25, 7.50, step=0.25)).astype(int)<jupyter_output><empty_output><jupyter_text>Once those are determined, the experiment can be initialized and performed. We iterate over the values in `n_xor` and `n_xnor` sequentially, running each experiment for the number of iterations declared in `mc_rep`.<jupyter_code># running the experiment:

# create empty arrays for storing results
mean_error = np.zeros((6, len(n_xor)+len(n_xnor)))
std_error = np.zeros((6, len(n_xor)+len(n_xnor)))
mean_te = np.zeros((4, len(n_xor)+len(n_xnor)))
std_te = np.zeros((4, len(n_xor)+len(n_xnor)))

# run the experiment
mean_error, std_error, mean_te, std_te = fn.run(mc_rep, n_test, n_trees,n_xor,n_xnor, mean_error, std_error, mean_te, std_te);<jupyter_output><empty_output><jupyter_text>Great! The experiment should now be complete, with the results stored in four arrays: `mean_error`, `std_error`, `mean_te`, and `std_te`.  Visualizing the ResultsNow that the experiment is complete, the results can be visualized by extracting the data from these arrays and plotting it. Here, we again utilize functions from `functions/xor_xnor_functions.py` to help in plotting:- `plot_error-and-eff`: plots generalization error and transfer efficiency for progressive learning and random forests<jupyter_code># plot data
%matplotlib inline
fn.plot_error_and_eff(n_xor, n_xnor, mean_error, mean_te, 'XOR', 'XNOR')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09236586244148362, 'fraction_numerical': 0.019985595966870724, 'mean_word_length': 5.566193853427896, 'pattern_counts': {'":': 0, '<': 22, '<?xml version=': 0, '>': 22, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '391076', 'license': 'MIT', 'n_tokens_mistral': 1819, 'n_tokens_neox': 1662, 'n_words': 694, 'path': 'docs/experiments/xor_xnor_exp.ipynb', 'repo_name': 'PSSF23/progressive-learning', 'size': '5554'}