<jupyter_start><jupyter_text>Basic training functionality<jupyter_code>from fastai.basic_train import *
from fastai.gen_doc.nbdoc import *
from fastai.vision import *
from fastai.distributed import *<jupyter_output><empty_output><jupyter_text>[`basic_train`](/basic_train.htmlbasic_train) wraps together the data (in a [`DataBunch`](/basic_data.htmlDataBunch) object) with a PyTorch model to define a [`Learner`](/basic_train.htmlLearner) object. Here the basic training loop is defined for the [`fit`](/basic_train.htmlfit) method. The [`Learner`](/basic_train.htmlLearner) object is the entry point of most of the [`Callback`](/callback.htmlCallback) objects that will customize this training loop in different ways. Some of the most commonly used customizations are available through the [`train`](/train.htmltrain) module, notably: - [`Learner.lr_find`](/train.htmllr_find) will launch an LR range test that will help you select a good learning rate. - [`Learner.fit_one_cycle`](/train.htmlfit_one_cycle) will launch a training using the 1cycle policy to help you train your model faster. - [`Learner.to_fp16`](/train.htmlto_fp16) will convert your model to half precision and help you launch a training in mixed precision.<jupyter_code>show_doc(Learner, title_level=2)<jupyter_output><empty_output><jupyter_text>The main purpose of [`Learner`](/basic_train.htmlLearner) is to train `model` using [`Learner.fit`](/basic_train.htmlLearner.fit). After every epoch, all *metrics* will be printed and also made available to callbacks.The default weight decay will be `wd`, which will be handled using the method from [Fixing Weight Decay Regularization in Adam](https://arxiv.org/abs/1711.05101) if `true_wd` is set (otherwise it's L2 regularization). If `true_wd` is set it will affect all optimizers, not only Adam. If `bn_wd` is `False`, then weight decay will be removed from batchnorm layers, as recommended in [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://arxiv.org/abs/1706.02677). If `train_bn`, batchnorm layer learnable params are trained even for frozen layer groups.To use [discriminative layer training](Discriminative-layer-training), pass a list of [`nn.Module`](https://pytorch.org/docs/stable/nn.htmltorch.nn.Module) as `layer_groups`; each [`nn.Module`](https://pytorch.org/docs/stable/nn.htmltorch.nn.Module) will be used to customize the optimization of the corresponding layer group.If `path` is provided, all the model files created will be saved in `path`/`model_dir`; if not, then they will be saved in `data.path`/`model_dir`.You can pass a list of [`callback`](/callback.htmlcallback)s that you have already created, or (more commonly) simply pass a list of callback functions to `callback_fns` and each function will be called (passing `self`) on object initialization, with the results stored as callback objects. For a walk-through, see the [training overview](/training.html) page. You may also want to use an [application](applications.html) specific model. For example, if you are dealing with a vision dataset, here the MNIST, you might want to use the [`cnn_learner`](/vision.learner.htmlcnn_learner) method:<jupyter_code>path = untar_data(URLs.MNIST_SAMPLE)
data = ImageDataBunch.from_folder(path)
learn = cnn_learner(data, models.resnet18, metrics=accuracy)<jupyter_output><empty_output><jupyter_text>Model fitting methods<jupyter_code>show_doc(Learner.lr_find)<jupyter_output><empty_output><jupyter_text>Runs the learning rate finder defined in [`LRFinder`](/callbacks.lr_finder.htmlLRFinder), as discussed in [Cyclical Learning Rates for Training Neural Networks](https://arxiv.org/abs/1506.01186).<jupyter_code>learn.lr_find()
learn.recorder.plot()
show_doc(Learner.fit)<jupyter_output><empty_output><jupyter_text>Uses [discriminative layer training](Discriminative-layer-training) if multiple learning rates or weight decay values are passed. To control training behaviour, use the [`callback`](/callback.htmlcallback) system or one or more of the pre-defined [`callbacks`](/callbacks.htmlcallbacks).<jupyter_code>learn.fit(1)
show_doc(Learner.fit_one_cycle)<jupyter_output><empty_output><jupyter_text>Use cycle length `cyc_len`, a per cycle maximal learning rate `max_lr`, momentum `moms`, division factor `div_factor`, weight decay `wd`, and optional callbacks [`callbacks`](/callbacks.htmlcallbacks). Uses the [`OneCycleScheduler`](/callbacks.one_cycle.htmlOneCycleScheduler) callback. Please refer to [What is 1-cycle](/callbacks.one_cycle.htmlWhat-is-1cycle?) for a conceptual background of 1-cycle training policy and more technical details on what do the method's arguments do.<jupyter_code>learn.fit_one_cycle(1)<jupyter_output><empty_output><jupyter_text>See results<jupyter_code>show_doc(Learner.predict)<jupyter_output><empty_output><jupyter_text>`predict` can be used to get a single prediction from the trained learner on one specific piece of data you are interested in.<jupyter_code>learn.data.train_ds[0]<jupyter_output><empty_output><jupyter_text>Each element of the dataset is a tuple, where the first element is the data itself, while the second element is the target label. So to get the data, we need to index one more time.<jupyter_code>data = learn.data.train_ds[0][0]
data
pred = learn.predict(data)
pred<jupyter_output><empty_output><jupyter_text>The first two elements of the tuple are, respectively, the predicted class and label. Label here is essentially an internal representation of each class, since class name is a string and cannot be used in computation. To check what each label corresponds to, run:<jupyter_code>learn.data.classes<jupyter_output><empty_output><jupyter_text>So category 0 is 3 while category 1 is 7.<jupyter_code>probs = pred[2]<jupyter_output><empty_output><jupyter_text>The last element in the tuple is the predicted probabilities. For a categorization dataset, the number of probabilities returned is the same as the number of classes; `probs[i]` is the probability that the `item` belongs to `learn.data.classes[i]`.<jupyter_code>learn.data.valid_ds[0][0]<jupyter_output><empty_output><jupyter_text>You could always check yourself if the probabilities given make sense.<jupyter_code>show_doc(Learner.get_preds)<jupyter_output><empty_output><jupyter_text>It will run inference using the learner on all the data in the `ds_type` dataset and return the predictions; if `n_batch` is not specified, it will run the predictions on the default batch size. If `with_loss`, it will also return the loss on each prediction. Here is how you check the default batch size.<jupyter_code>learn.data.batch_size
preds = learn.get_preds()
preds<jupyter_output><empty_output><jupyter_text>The first element of the tuple is a tensor that contains all the predictions.<jupyter_code>preds[0]<jupyter_output><empty_output><jupyter_text>While the second element of the tuple is a tensor that contains all the target labels.<jupyter_code>preds[1]
preds[1][0]<jupyter_output><empty_output><jupyter_text>For more details about what each number mean, refer to the documentation of [`predict`](/basic_train.htmlpredict).Since [`get_preds`](/basic_train.htmlget_preds) gets predictions on all the data in the `ds_type` dataset, here the number of predictions will be equal to the number of data in the validation dataset.<jupyter_code>len(learn.data.valid_ds)
len(preds[0]), len(preds[1])<jupyter_output><empty_output><jupyter_text>To get predictions on the entire training dataset, simply set the `ds_type` argument accordingly.<jupyter_code>learn.get_preds(ds_type=DatasetType.Train)<jupyter_output><empty_output><jupyter_text>To also get prediction loss along with the predictions and the targets, set `with_loss=True` in the arguments.<jupyter_code>learn.get_preds(with_loss=True)<jupyter_output><empty_output><jupyter_text>Note that the third tensor in the output tuple contains the losses.<jupyter_code>show_doc(Learner.validate)<jupyter_output><empty_output><jupyter_text>Return the calculated loss and the metrics of the current model on the given data loader `dl`. The default data loader `dl` is the validation dataloader. You can check the default metrics of the learner using:<jupyter_code>str(learn.metrics)
learn.validate()
learn.validate(learn.data.valid_dl)
learn.validate(learn.data.train_dl)
show_doc(Learner.show_results)<jupyter_output><empty_output><jupyter_text>Note that the text number on the top is the ground truth, or the target label, the one in the middle is the prediction, while the image number on the bottom is the image data itself.<jupyter_code>learn.show_results()
learn.show_results(ds_type=DatasetType.Train)
show_doc(Learner.pred_batch)<jupyter_output><empty_output><jupyter_text>Note that the number of predictions given equals to the batch size.<jupyter_code>learn.data.batch_size
preds = learn.pred_batch()
len(preds)<jupyter_output><empty_output><jupyter_text>Since the total number of predictions is too large, we will only look at a part of them.<jupyter_code>preds[:10]
item = learn.data.train_ds[0][0]
item
batch = learn.data.one_item(item)
batch
learn.pred_batch(batch=batch)
show_doc(Learner.interpret, full_name='interpret')
jekyll_note('This function only works in the vision application.')<jupyter_output><empty_output><jupyter_text>For more details, refer to [ClassificationInterpretation](/vision.learner.htmlClassificationInterpretation)  Model summary<jupyter_code>show_doc(Learner.summary)<jupyter_output><empty_output><jupyter_text>Test time augmentation<jupyter_code>show_doc(Learner.TTA, full_name = 'TTA')<jupyter_output><empty_output><jupyter_text>Applies Test Time Augmentation to `learn` on the dataset `ds_type`. We take the average of our regular predictions (with a weight `beta`) with the average of predictions obtained through augmented versions of the training set (with a weight `1-beta`). The transforms decided for the training set are applied with a few changes `scale` controls the scale for zoom (which isn't random), the cropping isn't random but we make sure to get the four corners of the image. Flipping isn't random but applied once on each of those corner images (so that makes 8 augmented versions total).  Gradient clipping<jupyter_code>show_doc(Learner.clip_grad)<jupyter_output><empty_output><jupyter_text>Mixed precision training<jupyter_code>show_doc(Learner.to_fp16)<jupyter_output><empty_output><jupyter_text>Uses the [`MixedPrecision`](/callbacks.fp16.htmlMixedPrecision) callback to train in mixed precision (i.e. forward and backward passes using fp16, with weight updates using fp32), using all [NVIDIA recommendations](https://docs.nvidia.com/deeplearning/sdk/mixed-precision-training/index.html) for ensuring speed and accuracy.<jupyter_code>show_doc(Learner.to_fp32)<jupyter_output><empty_output><jupyter_text>Distributed training If you want to use ditributed training or [`torch.nn.DataParallel`](https://pytorch.org/docs/stable/nn.htmltorch.nn.DataParallel) these will directly wrap the model for you.<jupyter_code>show_doc(Learner.to_distributed, full_name='to_distributed')
show_doc(Learner.to_parallel, full_name='to_parallel')<jupyter_output><empty_output><jupyter_text>Discriminative layer training When fitting a model you can pass a list of learning rates (and/or weight decay amounts), which will apply a different rate to each *layer group* (i.e. the parameters of each module in `self.layer_groups`). See the [Universal Language Model Fine-tuning for Text Classification](https://arxiv.org/abs/1801.06146) paper for details and experimental results in NLP (we also frequently use them successfully in computer vision, but have not published a paper on this topic yet). When working with a [`Learner`](/basic_train.htmlLearner) on which you've called `split`, you can set hyperparameters in four ways:1. `param = [val1, val2 ..., valn]` (n = number of layer groups)2. `param = val`3. `param = slice(start,end)`4. `param = slice(end)`If we chose to set it in way 1, we must specify a number of values exactly equal to the number of layer groups. If we chose to set it in way 2, the chosen value will be repeated for all layer groups. See [`Learner.lr_range`](/basic_train.htmlLearner.lr_range) for an explanation of the `slice` syntax).Here's an example of how to use discriminative learning rates (note that you don't actually need to manually call [`Learner.split`](/basic_train.htmlLearner.split) in this case, since fastai uses this exact function as the default split for `resnet18`; this is just to show how to customize it):<jupyter_code># creates 3 layer groups
learn.split(lambda m: (m[0][6], m[1]))
# only randomly initialized head now trainable
learn.freeze()
learn.fit_one_cycle(1)
# all layers now trainable
learn.unfreeze()
# optionally, separate LR and WD for each group
learn.fit_one_cycle(1, max_lr=(1e-4, 1e-3, 1e-2), wd=(1e-4,1e-4,1e-1))
show_doc(Learner.lr_range)<jupyter_output><empty_output><jupyter_text>Rather than manually setting an LR for every group, it's often easier to use [`Learner.lr_range`](/basic_train.htmlLearner.lr_range). This is a convenience method that returns one learning rate for each layer group. If you pass `slice(start,end)` then the first group's learning rate is `start`, the last is `end`, and the remaining are evenly geometrically spaced.If you pass just `slice(end)` then the last group's learning rate is `end`, and all the other groups are `end/10`. For instance (for our learner that has 3 layer groups):<jupyter_code>learn.lr_range(slice(1e-5,1e-3)), learn.lr_range(slice(1e-3))
show_doc(Learner.unfreeze)<jupyter_output><empty_output><jupyter_text>Sets every layer group to *trainable* (i.e. `requires_grad=True`).<jupyter_code>show_doc(Learner.freeze)<jupyter_output><empty_output><jupyter_text>Sets every layer group except the last to *untrainable* (i.e. `requires_grad=False`).What does '**the last layer group**' mean?  In the case of transfer learning, such as `learn = cnn_learner(data, models.resnet18, metrics=error_rate)`, `learn.model`will print out two large groups of layers: (0) Sequential and (1) Sequental in the following structure. We can consider the last conv layer as the break line between the two groups.```Sequential(  (0): Sequential(    (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)    (2): ReLU(inplace)    ...                (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)             (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)      )    )  )  (1): Sequential(    (0): AdaptiveConcatPool2d(      (ap): AdaptiveAvgPool2d(output_size=1)      (mp): AdaptiveMaxPool2d(output_size=1)    )    (1): Flatten()    (2): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)    (3): Dropout(p=0.25)    (4): Linear(in_features=1024, out_features=512, bias=True)    (5): ReLU(inplace)    (6): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)    (7): Dropout(p=0.5)    (8): Linear(in_features=512, out_features=12, bias=True)  ))````learn.freeze` freezes the first group and keeps the second or last group free to train, including multiple layers inside (this is why calling it 'group'), as you can see in `learn.summary()` output. How to read the table below, please see [model summary docs](/callbacks.hooks.htmlmodel_summary).```======================================================================Layer (type)         Output Shape         Param     Trainable ======================================================================.........______________________________________________________________________Conv2d               [1, 512, 4, 4]       2,359,296  False     ______________________________________________________________________BatchNorm2d          [1, 512, 4, 4]       1,024      True      ______________________________________________________________________AdaptiveAvgPool2d    [1, 512, 1, 1]       0          False     ______________________________________________________________________AdaptiveMaxPool2d    [1, 512, 1, 1]       0          False     ______________________________________________________________________Flatten              [1, 1024]            0          False     ______________________________________________________________________BatchNorm1d          [1, 1024]            2,048      True      ______________________________________________________________________Dropout              [1, 1024]            0          False     ______________________________________________________________________Linear               [1, 512]             524,800    True      ______________________________________________________________________ReLU                 [1, 512]             0          False     ______________________________________________________________________BatchNorm1d          [1, 512]             1,024      True      ______________________________________________________________________Dropout              [1, 512]             0          False     ______________________________________________________________________Linear               [1, 12]              6,156      True      ______________________________________________________________________Total params: 11,710,540Total trainable params: 543,628Total non-trainable params: 11,166,912```<jupyter_code>show_doc(Learner.freeze_to)<jupyter_output><empty_output><jupyter_text>From above we know what is layer group, but **what exactly does `freeze_to` do behind the scenes**?The `freeze_to` source code can be understood as the following pseudo-code:```pythondef freeze_to(self, n:int)->None:    for g in self.layer_groups[:n]: freeze     for g in self.layer_groups[n:]: unfreeze```    In other words, for example, `freeze_to(1)` is to freeze layer group 0 and unfreeze the rest layer groups, and `freeze_to(3)` is to freeze layer groups 0, 1, and 2 but unfreeze the rest layer groups (if there are more layer groups left).       Both `freeze` and `unfreeze` [sources](https://github.com/fastai/fastai/blob/master/fastai/basic_train.pyL216) are defined using `freeze_to`:    - When we say `freeze`, we mean that in the specified layer groups the [`requires_grad`](/torch_core.htmlrequires_grad) of all layers with weights (except BatchNorm layers) are set `False`, so the layer weights won't be updated during training.     - when we say `unfreeze`, we mean that in the specified layer groups the [`requires_grad`](/torch_core.htmlrequires_grad) of all layers with weights (except BatchNorm layers) are set `True`, so the layer weights will be updated during training.<jupyter_code>show_doc(Learner.split)<jupyter_output><empty_output><jupyter_text>A convenience method that sets `layer_groups` based on the result of [`split_model`](/torch_core.htmlsplit_model). If `split_on` is a function, it calls that function and passes the result to [`split_model`](/torch_core.htmlsplit_model) (see above for example).  Saving and loading models Simply call [`Learner.save`](/basic_train.htmlLearner.save) and [`Learner.load`](/basic_train.htmlLearner.load) to save and load models. Only the parameters are saved, not the actual architecture (so you'll need to create your model in the same way before loading weights back in). Models are saved to the `path`/`model_dir` directory.<jupyter_code>show_doc(Learner.save)<jupyter_output><empty_output><jupyter_text>If argument `name` is a pathlib object that's an absolute path, it'll override the default base directory (`learn.path`), otherwise the model will be saved in a file relative to `learn.path`.<jupyter_code>learn.save("trained_model")
learn.save("trained_model", return_path=True)
show_doc(Learner.load)<jupyter_output><empty_output><jupyter_text>This method only works after `save` (don't confuse with `export`/[`load_learner`](/basic_train.htmlload_learner) pair).If the `purge` argument is `True` (default) `load` internally calls `purge` with `clear_opt=False` to presever `learn.opt`.<jupyter_code>learn = learn.load("trained_model")<jupyter_output><empty_output><jupyter_text>Deploying your model When you are ready to put your model in production, export the minimal state of your [`Learner`](/basic_train.htmlLearner) with:<jupyter_code>show_doc(Learner.export)<jupyter_output><empty_output><jupyter_text>If argument `fname` is a pathlib object that's an absolute path, it'll override the default base directory (`learn.path`), otherwise the model will be saved in a file relative to `learn.path`. Passing `destroy=True` will destroy the [`Learner`](/basic_train.htmlLearner), freeing most of its memory consumption. For specifics see [`Learner.destroy`](/basic_train.htmlLearner.destroy).This method only works with the [`Learner`](/basic_train.htmlLearner) whose [`data`](/vision.data.htmlvision.data) was created through the [data block API](/data_block.html).Otherwise, you will have to create a [`Learner`](/basic_train.htmlLearner) yourself at inference and load the model with [`Learner.load`](/basic_train.htmlLearner.load).<jupyter_code>learn.export()
learn.export('trained_model.pkl')
path = learn.path
path
show_doc(load_learner)<jupyter_output><empty_output><jupyter_text>This function only works after `export` (don't confuse with `save`/`load` pair).The `db_kwargs` will be passed to the call to `databunch` so you can specify a `bs` for the test set, or `num_workers`.<jupyter_code>learn = load_learner(path)
learn = load_learner(path, 'trained_model.pkl')<jupyter_output><empty_output><jupyter_text>WARNING: If you used any customized classes when creating your learner, you must first define these classes first before executing [`load_learner`](/basic_train.htmlload_learner).You can find more information and multiple examples in [this tutorial](/tutorial.inference.html).  Freeing memoryIf you want to be able to do more without needing to restart your notebook, the following methods are designed to free memory when it's no longer needed. Refer to [this tutorial](/tutorial.resources.html) to learn how and when to use these methods.<jupyter_code>show_doc(Learner.purge)<jupyter_output><empty_output><jupyter_text>If `learn.path` is read-only, you can set `model_dir` attribute in Learner to a full `libpath` path that is writable (by setting `learn.model_dir` or passing `model_dir` argument in the [`Learner`](/basic_train.htmlLearner) constructor).<jupyter_code>show_doc(Learner.destroy)<jupyter_output><empty_output><jupyter_text>If you need to free the memory consumed by the [`Learner`](/basic_train.htmlLearner) object, call this method. It can also be automatically invoked through [`Learner.export`](/basic_train.htmlLearner.export) via its `destroy=True` argument.  Other methods<jupyter_code>show_doc(Learner.init)<jupyter_output><empty_output><jupyter_text>Initializes all weights (except batchnorm) using function `init`, which will often be from PyTorch's [`nn.init`](https://pytorch.org/docs/stable/nn.htmltorch-nn-init) module.<jupyter_code>show_doc(Learner.mixup)<jupyter_output><empty_output><jupyter_text>Uses [`MixUpCallback`](/callbacks.mixup.htmlMixUpCallback).<jupyter_code>show_doc(Learner.backward)
show_doc(Learner.create_opt)<jupyter_output><empty_output><jupyter_text>You generally won't need to call this yourself - it's used to create the [`optim`](https://pytorch.org/docs/stable/optim.htmlmodule-torch.optim) optimizer before fitting the model.<jupyter_code>show_doc(Learner.dl)
learn.dl()
learn.dl(DatasetType.Train)
show_doc(Recorder, title_level=2)<jupyter_output><empty_output><jupyter_text>A [`Learner`](/basic_train.htmlLearner) creates a [`Recorder`](/basic_train.htmlRecorder) object automatically - you do not need to explicitly pass it to `callback_fns` - because other callbacks rely on it being available. It stores the smoothed loss, hyperparameter values, and metrics for each batch, and provides plotting methods for each. Note that [`Learner`](/basic_train.htmlLearner) automatically sets an attribute with the snake-cased name of each callback, so you can access this through `Learner.recorder`, as shown below.  Plotting methods<jupyter_code>show_doc(Recorder.plot)<jupyter_output><empty_output><jupyter_text>This is mainly used with the learning rate finder, since it shows a scatterplot of loss vs learning rate.<jupyter_code>path = untar_data(URLs.MNIST_SAMPLE)
data = ImageDataBunch.from_folder(path)
learn = cnn_learner(data, models.resnet18, metrics=accuracy)
learn.lr_find()
learn.recorder.plot()
show_doc(Recorder.plot_losses)<jupyter_output><empty_output><jupyter_text>Note that validation losses are only calculated once per epoch, whereas training losses are calculated after every batch.<jupyter_code>learn.fit_one_cycle(5)
learn.recorder.plot_losses()
show_doc(Recorder.plot_lr)
learn.recorder.plot_lr()
learn.recorder.plot_lr(show_moms=True)
show_doc(Recorder.plot_metrics)<jupyter_output><empty_output><jupyter_text>Note that metrics are only collected at the end of each epoch, so you'll need to train at least two epochs to have anything to show here.<jupyter_code>learn.recorder.plot_metrics()<jupyter_output><empty_output><jupyter_text>Callback methods You don't call these yourself - they're called by fastai's [`Callback`](/callback.htmlCallback) system automatically to enable the class's functionality. Refer to [`Callback`](/callback.htmlCallback) for more details.<jupyter_code>show_doc(Recorder.on_backward_begin)
show_doc(Recorder.on_batch_begin)
show_doc(Recorder.on_epoch_end)
show_doc(Recorder.on_train_begin)<jupyter_output><empty_output><jupyter_text>Inner functions The following functions are used along the way by the [`Recorder`](/basic_train.htmlRecorder) or can be called by other callbacks.<jupyter_code>show_doc(Recorder.add_metric_names)
show_doc(Recorder.format_stats)<jupyter_output><empty_output><jupyter_text>Module functions Generally you'll want to use a [`Learner`](/basic_train.htmlLearner) to train your model, since they provide a lot of functionality and make things easier. However, for ultimate flexibility, you can call the same underlying functions that [`Learner`](/basic_train.htmlLearner) calls behind the scenes:<jupyter_code>show_doc(fit)<jupyter_output><empty_output><jupyter_text>Note that you have to create the [`Optimizer`](https://pytorch.org/docs/stable/optim.htmltorch.optim.Optimizer) yourself if you call this function, whereas [`Learn.fit`](/basic_train.htmlfit) creates it for you automatically.<jupyter_code>show_doc(train_epoch)<jupyter_output><empty_output><jupyter_text>You won't generally need to call this yourself - it's what [`fit`](/basic_train.htmlfit) calls for each epoch.<jupyter_code>show_doc(validate)<jupyter_output><empty_output><jupyter_text>This is what [`fit`](/basic_train.htmlfit) calls after each epoch. You can call it if you want to run inference on a [`DataLoader`](https://pytorch.org/docs/stable/data.htmltorch.utils.data.DataLoader) manually.<jupyter_code>show_doc(get_preds)
show_doc(loss_batch)<jupyter_output><empty_output><jupyter_text>You won't generally need to call this yourself - it's what [`fit`](/basic_train.htmlfit) and [`validate`](/basic_train.htmlvalidate) call for each batch. It only does a backward pass if you set `opt`.  Other classes<jupyter_code>show_doc(LearnerCallback, title_level=3)
show_doc(RecordOnCPU, title_level=3)<jupyter_output><empty_output><jupyter_text>Open This NotebookOpen in GCP Notebooks  Undocumented Methods - Methods moved below this line will intentionally be hidden<jupyter_code>show_doc(Learner.tta_only)
show_doc(Learner.TTA)
show_doc(RecordOnCPU.on_batch_begin)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10129458007284267, 'fraction_numerical': 0.012873679275900617, 'mean_word_length': 6.338449325218312, 'pattern_counts': {'":': 0, '<': 241, '<?xml version=': 0, '>': 242, 'https://': 13, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '60', 'dir': 'jupyter-structured-clean-dedup', 'id': '24734', 'license': 'Apache-2.0', 'n_tokens_mistral': 8604, 'n_tokens_neox': 8174, 'n_words': 2592, 'path': 'docs_src/basic_train.ipynb', 'repo_name': 'jamesgolden1/fastai', 'size': '27731'}