<jupyter_start><jupyter_text>Callback API demoIn this demo we are going to use the callback API to **track some useful information while training** a SLIM minerWe simply define custom python functions, and submit them to create an instance of `skmine.callbacks.CallBacks`.<jupyter_code>import pandas as pd
import numpy as np

from skmine.callbacks import CallBacks
from skmine.itemsets import SLIM
from skmine.datasets.fimi import fetch_mushroom
mushroom = fetch_mushroom()
mushroom.head()
# set n_iter_no_change to a high value so learning takes longer
slim = SLIM(n_iter_no_change=200)<jupyter_output><empty_output><jupyter_text>------------------- Define your own callbacksWe define custom functions, that will take the result of the function they target as input. Those results will be ingested for later reuseHere we define two methods:1. `post_evaluate` is executed after `SLIM.evaluate`. It tracks sizes for both the data and the model2. `post_gen` is executed after `SLIM.generate_candidates`, and just records the size of the current batch of candidates<jupyter_code>sizes = list()
candidate_sizes = list()

def post_evaluate(data_size, model_size, *args):
    sizes.append((data_size, model_size))

def post_gen(candidates):
    candidate_sizes.append(len(candidates))<jupyter_output><empty_output><jupyter_text>A `skmine.callbacks.CallBacks` is a collection of callbacks.It's a mapping between function names and their dedicated callbacks. When an instance of `skmine.callbacks.CallBacks` is called (`() operator`) on an object, it looks for internal methods and tries to attach the callbacks.<jupyter_code>callbacks = CallBacks(evaluate=post_evaluate, generate_candidates=post_gen)

callbacks(slim)
%time slim.fit(mushroom)<jupyter_output>CPU times: user 3.75 s, sys: 36.7 ms, total: 3.79 s
Wall time: 3.8 s<jupyter_text>Inner view of MDL learning The plot below clearly shows **how SLIM performs compression**.While the blue curve represents the size of the data, red vertical lines emphasize the end of a batch of candidates.**We can clearly dinstinguish the beginning of a batch of candidates, where the learning curve is quite abrupt, from the end of a batch, where it reaches a plateau**.<jupyter_code>df = pd.DataFrame(sizes, columns=['data_size', 'model_size'])
ax = df.data_size.plot()
for s in np.cumsum(candidate_sizes):
    ax.axvline(s, color='r')<jupyter_output><empty_output><jupyter_text>------------------And here is how the model size goes up<jupyter_code>df.model_size.plot()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09230769230769231, 'fraction_numerical': 0.00631163708086785, 'mean_word_length': 5.656167979002625, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '35296', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 781, 'n_tokens_neox': 753, 'n_words': 281, 'path': 'docs/tutorials/callbacks/callbacks.ipynb', 'repo_name': 'sdall/scikit-mine', 'size': '2535'}