<jupyter_start><jupyter_text>fastai Multi Core TPU Training Example (Multi Core) - CIFAR Installation and Setup Downgrade from pytorch 1.8 (colab default) and install pytorch 1.7.1 because fastai (as of 2.3.0) doesn't support pytorch 1.8 as of yet.<jupyter_code>!pip install -qqq --no-cache-dir torch==1.7.1+cu101 torchvision==0.8.2+cu101 torchtext==0.8.0 -f https://download.pytorch.org/whl/torch_stable.html<jupyter_output>[K     |████████████████████████████████| 735.4MB 1.1MB/s 
[K     |████████████████████████████████| 12.8MB 48.8MB/s 
[K     |████████████████████████████████| 7.0MB 6.9MB/s 
[?25h<jupyter_text>Install latest fastai (2.3.0)<jupyter_code>!curl -s https://course19.fast.ai/setup/colab | bash
!pip install -Uqq fastai==2.3.0<jupyter_output>[K     |████████████████████████████████| 194kB 7.0MB/s 
[K     |████████████████████████████████| 61kB 5.3MB/s 
[?25h<jupyter_text>Setup colab directories to show downloaded fastai data and models Install pytorch-xla 1.7<jupyter_code>!pip install -Uqq cloud-tpu-client==0.10 https://storage.googleapis.com/tpu-pytorch/wheels/torch_xla-1.7-cp37-cp37m-linux_x86_64.whl<jupyter_output>[K     |████████████████████████████████| 133.6MB 32kB/s 
[K     |████████████████████████████████| 61kB 3.5MB/s 
[31mERROR: earthengine-api 0.1.258 has requirement google-api-python-client<2,>=1.12.1, but you'll have google-api-python-client 1.8.0 which is incompatible.[0m
[?25h<jupyter_text>Install the fastai_xla_extensions package<jupyter_code>!pip install -Uqq fastai_xla_extensions
# !pip install -Uqq git+https://github.com/butchland/fastai_xla_extensions.git<jupyter_output><empty_output><jupyter_text>Check which versions of pytorch and fastai packages are installed.<jupyter_code>!pip freeze | grep torch
!pip freeze | grep fast<jupyter_output>torch==1.7.1+cu101
torch-xla==1.7
torchsummary==1.5.1
torchtext==0.8.0
torchvision==0.8.2+cu101
fastai==2.3.0
fastai-xla-extensions==0.0.11
fastcore==1.3.19
fastdtw==0.3.4
fastprogress==1.0.0
fastrlock==0.6<jupyter_text>Start of codeImport fastai and fastai_xla_extension packages<jupyter_code>from fastai.vision.all import *
from fastai_xla_extensions.all import *
path = untar_data(URLs.CIFAR)
data = DataBlock(
    blocks=(ImageBlock, CategoryBlock),
    get_items=get_image_files,
    get_y=parent_label,
    splitter=GrandparentSplitter(valid_name='test'),
    item_tfms=Resize(32),
    batch_tfms=[Normalize.from_stats(*imagenet_stats)]
)
data.summary(path)
dls = data.dataloaders(path, bs=64)
dls.show_batch()
learner = cnn_learner(dls, resnet18, metrics=accuracy, concat_pool=False)
learner.xla_lr_find(stop_div=False)
learner.xla_fine_tune(3, freeze_epochs=3, base_lr=1.4e-1)
learner.save('stage-1')
learner.unfreeze()
learner.xla_lr_find()
learner.summary()
learner.xla_fit_one_cycle(5, lr_max=slice(2.2e-6, 2e-4))
learner.summary()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.20769764216366157, 'fraction_numerical': 0.05617198335644938, 'mean_word_length': 6.1234567901234565, 'pattern_counts': {'":': 0, '<': 22, '<?xml version=': 0, '>': 22, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '586848', 'license': 'Apache-2.0', 'n_tokens_mistral': 1428, 'n_tokens_neox': 1233, 'n_words': 186, 'path': 'samples/fastai-multi-core-tpu-training-sample.ipynb', 'repo_name': 'hixio-mh/fastai_xla_extensions', 'size': '2884'}