<jupyter_start><jupyter_text>Optimizer
> Utility functions for FastAI optimizers in Pytorch models. Some utility functions for training Pytorch models with FastAI fine tuning method. The code is from [this repository](https://github.com/muellerzr/fastai_minima/blob/master/fastai_minima/optimizer.pyL159).<jupyter_code># export
import torch.nn as nn
from fastcore.foundation import L
from fastai.optimizer import OptimWrapper
# export
def params(m):
    "Return all parameters of `m` (Pytorch model)."
    return [p for p in m.parameters()]
# export
def convert_params(o:list) -> list:
    """
    Converts `o` into Pytorch-compatable param groups
    `o` should be a set of layer-groups that should be split in the optimizer
    Example:
    ```python
    def splitter(m): return convert_params([[m.a], [m.b]])
    ```
    Where `m` is a model defined as:
    ```python
    class RegModel(Module):
      def __init__(self): self.a,self.b = nn.Parameter(torch.randn(1)),nn.Parameter(torch.randn(1))
      def forward(self, x): return x*self.a + self.b
    ```
    """
    if not isinstance(o[0], dict):
        splitter = []
        for group in o:
            if not isinstance(group[0], nn.parameter.Parameter):
                group = L(group).map(params)[0]
            splitter.append({'params':group})
        return splitter
    return o<jupyter_output><empty_output><jupyter_text>In the FastAI library is largely used transfer learning with layer-group learning rate freezing. 
The `convert_params` function returns a list of parameters for specific layers in a model that allows discriminative learning rates.<jupyter_code>#export
def smp_splitter(model):
    return convert_params([[model.encoder],[model.decoder],[model.segmentation_head]])
#export
def opt_func(params, torch_opt, *args, **kwargs):
    """Pytorch Optimizer for fastai Learner."""
    return OptimWrapper(params, torch_opt, *args, **kwargs)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1110550227158001, 'fraction_numerical': 0.004038364462392731, 'mean_word_length': 3.7078384798099764, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 11, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '272884', 'license': 'Apache-2.0', 'n_tokens_mistral': 625, 'n_tokens_neox': 603, 'n_words': 179, 'path': 'nbs/03_optimizer.ipynb', 'repo_name': 'marcomatteo/steel-segmentation-nbdev', 'size': '1981'}