<jupyter_start><jupyter_text>Hook callbacks This provides both a standalone class and a callback for registering and automatically deregistering [PyTorch hooks](https://pytorch.org/tutorials/beginner/former_torchies/nn_tutorial.htmlforward-and-backward-function-hooks), along with some pre-defined hooks. Hooks can be attached to any [`nn.Module`](https://pytorch.org/docs/stable/nn.htmltorch.nn.Module), for either the forward or the backward pass.We'll start by looking at the pre-defined hook [`ActivationStats`](/callbacks.hooks.htmlActivationStats), then we'll see how to create our own.<jupyter_code>from fastai.gen_doc.nbdoc import *
from fastai.callbacks.hooks import * 
from fastai import *
from fastai.train import *
from fastai.vision import *
show_doc(ActivationStats)<jupyter_output><empty_output><jupyter_text>[`ActivationStats`](/callbacks.hooks.htmlActivationStats) saves the layer activations in `self.stats` for all `modules` passed to it. By default it will save activations for *all* modules. For instance:<jupyter_code>path = untar_data(URLs.MNIST_SAMPLE)
data = ImageDataBunch.from_folder(path)
learn = ConvLearner(data, models.resnet18, callback_fns=ActivationStats)
learn.fit(1)<jupyter_output><empty_output><jupyter_text>The saved `stats` is a `FloatTensor` of shape `(2,num_batches,num_modules)`. The first axis is `(mean,stdev)`.<jupyter_code>len(learn.data.train_dl),len(learn.activation_stats.modules)
learn.activation_stats.stats.shape<jupyter_output><empty_output><jupyter_text>So this shows the standard deviation (`axis0==1`) of 5th last layer (`axis1==-5`) for each batch (`axis2`):<jupyter_code>plt.plot(learn.activation_stats.stats[1][-5].numpy());
show_doc(Hook)<jupyter_output><empty_output><jupyter_text>Registers and manually deregisters a [PyTorch hook](https://pytorch.org/tutorials/beginner/former_torchies/nn_tutorial.htmlforward-and-backward-function-hooks). Your `hook_func` will be called automatically when forward/backward (depending on `is_forward`) for your module `m` is run, and the result of that function is placed in `self.stored`.<jupyter_code>show_doc(Hook.remove)<jupyter_output><empty_output><jupyter_text>Deregister the hook, if not called already.<jupyter_code>show_doc(Hooks)<jupyter_output><empty_output><jupyter_text>Acts as a `Collection` (i.e. `len(hooks)` and `hooks[i]`) and an `Iterator` (i.e. `for hook in hooks`) of a group of hooks, one for each module in `ms`, with the ability to remove all as a group. Use `stored` to get all hook results. `hook_func` and `is_forward` behavior is the same as [`Hook`](/callbacks.hooks.htmlHook). See the source code for [`HookCallback`](/callbacks.hooks.htmlHookCallback) for a simple example.<jupyter_code>show_doc(Hooks.remove)<jupyter_output><empty_output><jupyter_text>Deregister all hooks created by this class, if not previously called.  Convenience functions for hooks<jupyter_code>show_doc(hook_output)<jupyter_output><empty_output><jupyter_text>Function that creates a [`Hook`](/callbacks.hooks.htmlHook) for `module` that simply stores the output of the layer.<jupyter_code>show_doc(hook_outputs)<jupyter_output><empty_output><jupyter_text>Function that creates a [`Hook`](/callbacks.hooks.htmlHook) for all passed `modules` that simply stores the output of the layers. For example, the (slightly simplified) source code of [`model_sizes`](/callbacks.hooks.htmlmodel_sizes) is:```pythondef model_sizes(m, size):    x = m(torch.zeros(1, in_channels(m), *size))    return [o.stored.shape for o in hook_outputs(m)]```<jupyter_code>show_doc(model_sizes)<jupyter_output><empty_output><jupyter_text>It can be useful to get the size of each layer of a model (e.g. for printing a summary, or for generating cross-connections for a `DynamicUnet`), however they depend on the size of the input. This function calculates the layer sizes by passing in a minimal tensor of `size`.<jupyter_code>show_doc(HookCallback)<jupyter_output><empty_output><jupyter_text>For all `modules`, uses a callback to automatically register a method `self.hook` (that you must define in an inherited class) as a hook. This method must have the signature:```pythondef hook(self, m:Model, input:Tensors, output:Tensors)```If `do_remove` then the hook is automatically deregistered at the end of training. See [`ActivationStats`](/callbacks.hooks.htmlActivationStats) for a simple example of inheriting from this class.  Undocumented Methods - Methods moved below this line will intentionally be hidden<jupyter_code>show_doc(HookCallback.remove)
show_doc(HookCallback.on_train_begin)
show_doc(HookCallback.on_train_end)
show_doc(ActivationStats.hook)
show_doc(ActivationStats.on_batch_end)
show_doc(ActivationStats.on_train_begin)
show_doc(ActivationStats.on_train_end)<jupyter_output><empty_output><jupyter_text>New Methods - Please document or move to the undocumented section<jupyter_code>show_doc(Hook.hook_fn)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10511363636363637, 'fraction_numerical': 0.002637987012987013, 'mean_word_length': 7.557291666666667, 'pattern_counts': {'":': 0, '<': 53, '<?xml version=': 0, '>': 53, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '29059', 'license': 'Apache-2.0', 'n_tokens_mistral': 1608, 'n_tokens_neox': 1576, 'n_words': 425, 'path': 'docs_src/callbacks.hooks.ipynb', 'repo_name': 'dienhoa/fastai_docs', 'size': '4928'}