<jupyter_start><jupyter_text>⚔`timm` for Transfer Learning       Fast.ai Version: v2.0.17  Draft Date: 2020-12-04           记录`fast.ai` 使用中的一些尝试 ...    预训练模型一览：[pytorch-image-models](https://github.com/rwightman/pytorch-image-models)<jupyter_code>from fastai.vision.all import *
import fastai
fastai.__version__<jupyter_output><empty_output><jupyter_text>标记 提示💡: 蓝色给大家一些新的提示要点，可以进行拓展操作.    经验分享👣分享一些经验，踩过的坑.    知识回顾📕: 这部分会对学过的知识进行关联和回顾，便于大家进行巩固.     概要✅：绿色部分为章节内容核心点，提醒关注.   重要📌    红色标识了重点注意事项，提醒大家注意.    🤔--这是被隐藏的内容，思考之后再打开👇>>打开之后你将看到内容的详情 ...>>  尝试<jupyter_code>path = untar_data(URLs.PETS)
files = get_image_files(path/'images')
pat = r'/([^/]+)_\d+.*'
item_tfms = RandomResizedCrop(460, min_scale=0.75, ratio=(1.,1.))
batch_tfms = [*aug_transforms(size=224, max_warp=0), Normalize.from_stats(*imagenet_stats)]
bs=16
pets = DataBlock(blocks=(ImageBlock, CategoryBlock),
                 get_items=get_image_files,
                 splitter=RandomSplitter(0.2),
                 get_y=RegexLabeller(pat = pat),
                 item_tfms=item_tfms,
                 batch_tfms=batch_tfms)
dls = pets.dataloaders(path/'images', bs=bs)<jupyter_output><empty_output><jupyter_text>创建模型BODY fastai has a create_body function, whcih is called during cnn_learner, that will take a model architecuture and slice off the last Linear layer (resulting in a "body" that outputs unpooled features).<jupyter_code>create_body??
from timm import create_model
from fastai.vision.learner import _update_first_layer

def create_timm_body(arch:str, pretrained=True, cut=None, n_in=3):
    '''create a body from any model in the `timm` library. '''
    model = create_model(arch, pretrained=pretrained, num_classes=0, global_pool='')
    _update_first_layer(model, n_in, pretrained)
    if cut is None:
        ll = list(enumerate(model.children()))
        cut = next(i for i,o in reversed(ll) if has_pool_type(o))
    if isinstance(cut, int): return nn.Sequential(*list(model.children())[:cut])
    elif callable(cut): return cut(model)
    else: raise NamedError("cut must be either integer or function")
body = create_timm_body('efficientnet_b3a', pretrained=True)
nf = num_features_model(body)*2; nf
head = create_head(nf, dls.c)
net = nn.Sequential(body, head)
learn = Learner(dls, net, splitter=default_split)
learn.freeze()
unfrozen_params = filter(lambda p: not p.requires_grad, learn.model.parameters())
unfrozen_params = sum([np.prod(p.size()) for p in unfrozen_params])
model_parameters = filter(lambda p: p.requires_grad, learn.model.parameters())
frozen_params = sum([np.prod(p.size()) for p in model_parameters])
unfrozen_params, frozen_params
learn.fine_tune(3)<jupyter_output><empty_output><jupyter_text>Refector<jupyter_code>def create_timm_model(arch:str, n_out, cut=None, pretrained=True, n_in=3, init=nn.init.kaiming_normal_, custom_head=None,
                     concat_pool=True, **kwargs):
    "Create custom architecture using `arch`, `n_in` and `n_out` from the `timm` library"
    body = create_timm_body(arch, pretrained, None, n_in)
    if custom_head is None:
        nf = num_features_model(nn.Sequential(*body.children())) * (2 if concat_pool else 1)
        head = create_head(nf, n_out, concat_pool=concat_pool, **kwargs)
    else: head = custom_head
    model = nn.Sequential(body, head)
    if init is not None: apply_init(model[1], init)
    return model
def timm_learner(dls, arch:str, loss_func=None, pretrained=True, cut=None, splitter=None,
                y_range=None, config=None, n_out=None, normalize=True, **kwargs):
    "Build a convnet style learner from `dls` and `arch` using the `timm` library"
    if config is None: config = {}
    if n_out is None: n_out = get_c(dls)
    assert n_out, "`n_out` is not defined, and could not be inferred from data, set `dls.c` or pass `n_out`"
    if y_range is None and 'y_range' in config: y_range = config.pop('y_range')
    model = create_timm_model(arch, n_out, default_split, pretrained, y_range=y_range, **config)
    learn = Learner(dls, model, loss_func=loss_func, splitter=default_split, **kwargs)
    if pretrained: learn.freeze()
    return learn
learn = timm_learner(dls, 'efficientnet_b3a')<jupyter_output><empty_output><jupyter_text>其它  List timm Models<jupyter_code>import timm
timm.list_models()[:10]<jupyter_output><empty_output><jupyter_text>Searching Models<jupyter_code>timm.list_models('*efficientnet*')[:10]
timm.list_models('*b3a')[:10]
timm.list_models('resne*t*', pretrained=True)[:10]<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12635779206384393, 'fraction_numerical': 0.010418975836843272, 'mean_word_length': 5.072678331090175, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 29, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '219172', 'license': 'MIT', 'n_tokens_mistral': 1782, 'n_tokens_neox': 1776, 'n_words': 342, 'path': '3.fastai/02.timm_transferLearning.ipynb', 'repo_name': 'ddebby/AI-Lab', 'size': '4511'}