<jupyter_start><jupyter_text>⬛ PyCEBox TutorialThis tutorial recreates the first example from [_Peeking Inside the Black Box: VisualizingStatistical Learning with Plots of Individual Conditional Expectation_](https://arxiv.org/pdf/1309.6392v2.pdf) using [`pycebox`](https://github.com/AustinRochford/PyCEbox).  For details of `pycebox`'s API, consult the [documentation](http://austinrochford.github.io/PyCEbox/docs/).First we generate 1,000 data points from the model $X_1, X_2, X_3 \sim U(-1, 1)$, $\varepsilon \sim N(0, 1)$,$$y = 0.2 X_1 - 5 X_2 + 10 X_2 \cdot \mathbb{I}(X_3 \geq 0) + \varepsilon,$$where $\mathbb{I}(\cdot)$ is the indicator function.<jupyter_code>N = 1000
df = pd.DataFrame(sp.stats.uniform.rvs(-1, 2, size=(N, 3)),
                  columns=['x1', 'x2', 'x3'])
noise = sp.stats.norm.rvs(size=N)
y = 0.2 * df.x1 - 5 * df.x2 + 10 * df.x2 * (df.x3 >= 0) + noise<jupyter_output><empty_output><jupyter_text>We will study the relationship between $y$ and $X_2$, which is shown below.<jupyter_code>fig, ax = plt.subplots(figsize=(8, 6))

ax.scatter(df.x2, y, c='k', alpha=0.5);

ax.set_xlim(-1.05, 1.05);
ax.set_xlabel('$X_2$');

ax.set_ylabel('$y$');

ax.set_title('Data');<jupyter_output><empty_output><jupyter_text>We fit a [`scikit-learn` `GradientBoostringRegressor`](http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingRegressor.html) to the data.<jupyter_code>gbm = GradientBoostingRegressor()
gbm.fit(df.values, y)<jupyter_output><empty_output><jupyter_text>We can now use `pycebox`'s `ice` function, to generate individual conditional expectation curves with respect to the fitted model and the predictor $X_2$.<jupyter_code>ice_df = ice(df, 'x2', gbm.predict, num_grid_points=100)<jupyter_output><empty_output><jupyter_text>Each column of `ice_df` corresponds to a data point, with the $X_2$ value removed.  Each row corresponds to an $X_2$ value.<jupyter_code>ice_df.head()<jupyter_output><empty_output><jupyter_text>The individual conditional expectation curves in `ice_df` can now be plotting using your visualization package of choice.  `pycebox` includes a convenience function, `ice_plot` for plotting the individual conditional expectation curves using [`matplotlib`](http://matplotlib.org/).<jupyter_code>fig, (data_ax, ice_ax) = plt.subplots(ncols=2, sharex=True, sharey=True, figsize=(16, 6))

data_ax.scatter(df.x2, y, c='k', alpha=0.5);

data_ax.set_xlim(-1.05, 1.05);
data_ax.set_xlabel('$X_2$');

data_ax.set_ylabel('$y$');

data_ax.set_title('Data');

ice_plot(ice_df, frac_to_plot=0.1,
         c='k', alpha=0.25,
         ax=ice_ax);

ice_ax.set_xlabel('$X_2$');

ice_ax.set_ylabel('$y$');

ice_ax.set_title('ICE curves');<jupyter_output>/opt/conda/lib/python3.5/site-packages/ipykernel/__main__.py:14: PerformanceWarning: indexing past lexsort depth may impact performance.<jupyter_text>Inspecting the ICE curves, it seems quite likely that there is an important interaction.  Since we know the data generating process, we know that the key interaction is between $X_2$ and $X_3$.  The function `ice_plot` accepts `color_by` keywords, which accepts either a string, which represents the column in the initial `DataFrame` or a function to apply to the column index in the ICE `DataFrame` to use for coloring the ICE plots.<jupyter_code>fig, (data_ax, ice_ax) = plt.subplots(ncols=2, sharex=True, sharey=True, figsize=(16, 6))

data_ax.scatter(df.x2, y, c='k', alpha=0.5);

data_ax.set_xlim(-1.05, 1.05);
data_ax.set_xlabel('$X_2$');

data_ax.set_ylabel('$y$');

data_ax.set_title('Data');

ice_plot(ice_df, frac_to_plot=0.1,
         color_by='x3', cmap=PuOr,
         ax=ice_ax);

ice_ax.set_xlabel('$X_2$');

ice_ax.set_ylabel('$y$');

ice_ax.set_title('ICE Curves');<jupyter_output><empty_output><jupyter_text>This plot makes the interaction between $X_2$ and $X_3$ quite transparent.Additionally, the `ice_plot` accepts the `plot_points` keyword.  When `plot_points=True`, a the predicted value for each data point is plotted on its ICE curve.<jupyter_code>fig, (data_ax, ice_ax) = plt.subplots(ncols=2, sharex=True, sharey=True, figsize=(16, 6))

data_ax.scatter(df.x2, y, c='k', alpha=0.5);

data_ax.set_xlim(-1.05, 1.05);
data_ax.set_xlabel('$X_2$');

data_ax.set_ylabel('$y$');

data_ax.set_title('Data');

ice_plot(ice_df, frac_to_plot=0.1,
         plot_points=True, point_kwargs={'color': 'k', 'alpha': 0.75},
         color_by='x3', cmap=PuOr,
         ax=ice_ax);

ice_ax.set_xlabel('$X_2$');

ice_ax.set_ylabel('$y$');

ice_ax.set_title('ICE Curves');<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.15389667250437827, 'fraction_numerical': 0.029991243432574432, 'mean_word_length': 5.768888888888889, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 33, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '566975', 'license': 'MIT', 'n_tokens_mistral': 1840, 'n_tokens_neox': 1756, 'n_words': 388, 'path': 'notebooks/PyCEBox Tutorial.ipynb', 'repo_name': 'Thammasorn/PyCEbox', 'size': '4568'}