<jupyter_start><jupyter_text>04 - DataBlock SummaryThis notebook shows how to work `DataBlock.summary()` and what it can do!<jupyter_code>#Run once per session
!pip install fastai -q --upgrade
from fastai.vision.all import *<jupyter_output><empty_output><jupyter_text>We'll use `ImageWoof` like we did in previous notebooks<jupyter_code>path = untar_data(URLs.IMAGEWOOF)
lbl_dict = dict(
  n02086240= 'Shih-Tzu',
  n02087394= 'Rhodesian ridgeback',
  n02088364= 'Beagle',
  n02089973= 'English foxhound',
  n02093754= 'Australian terrier',
  n02096294= 'Border terrier',
  n02099601= 'Golden retriever',
  n02105641= 'Old English sheepdog',
  n02111889= 'Samoyed',
  n02115641= 'Dingo'
)
batch_tfms = [*aug_transforms(size=224, max_warp=0), Normalize.from_stats(*imagenet_stats, cuda=False)]
item_tfms = Resize(128)
bs=64
pets = DataBlock(blocks=(ImageBlock, CategoryBlock),
                 get_items=get_image_files,
                 splitter=RandomSplitter(),
                 get_y=Pipeline([parent_label, lbl_dict.__getitem__]),
                 item_tfms=item_tfms,
                 batch_tfms=batch_tfms)<jupyter_output><empty_output><jupyter_text>Now to run `.summary`, we need to send in what our `DataBlock` expects. In this case it's a path (think how we make our `DataLoaders` from the `DataBlock`)<jupyter_code>pets.summary(path)<jupyter_output>Setting-up type transforms pipelines
Collecting items from /root/.fastai/data/imagewoof2
Found 12954 items
2 datasets of sizes 10364,2590
Setting up Pipeline: PILBase.create
Setting up Pipeline: parent_label -> dict.__getitem__ -> Categorize

Building one sample
  Pipeline: PILBase.create
    starting from
      /root/.fastai/data/imagewoof2/train/n02096294/n02096294_3256.JPEG
    applying PILBase.create gives
      <fastai.vision.core.PILImage image mode=RGB size=200x200 at 0x7F2A65CA2668>
  Pipeline: parent_label -> dict.__getitem__ -> Categorize
    starting from
      /root/.fastai/data/imagewoof2/train/n02096294/n02096294_3256.JPEG
    applying parent_label gives
      n02096294
    applying dict.__getitem__ gives
      Border terrier
    applying Categorize gives
      TensorCategory(2)

Final sample: (<fastai.vision.core.PILImage image mode=RGB size=200x200 at 0x7F2A66EA4EF0>, TensorCategory(2))


Setting up after_item: Pipeline: Resize -> ToTensor
Setting up before_batch: Pipel[...]<jupyter_text>What we find is it will go through **each** and every single part of our `DataBlock`, test it on an item, and we can see what popped out! **But!** What if we are using the `Datasets` instead? Let's go through how to utilize it<jupyter_code>tfms = [[PILImage.create], [parent_label, Categorize()]]
item_tfms = [ToTensor(), Resize(128)]
batch_tfms = [FlipItem(), RandomResizedCrop(128, min_scale=0.35),
              IntToFloatTensor(), Normalize.from_stats(*imagenet_stats, cuda=False)]
items = get_image_files(path)
split_idx = GrandparentSplitter(valid_name='val')(items)
dsets = Datasets(items, tfms, splits=split_idx)
dls = dsets.dataloaders(after_item=item_tfms, after_batch=batch_tfms, bs=64)<jupyter_output><empty_output><jupyter_text>We'll want to grab the first item from our set<jupyter_code>x = dsets.train[0]
x<jupyter_output><empty_output><jupyter_text>And pass it into any `after_item` or `after_batch` transform `Pipeline`. We can list them by calling them<jupyter_code>dls.train.after_item
dls.train.after_batch<jupyter_output><empty_output><jupyter_text>And now we can pass in our item through the `Pipeline` like so:(`x[0]` has our input and `x[1]` has our `y`)<jupyter_code>for f in dls.train.after_item:
  name = f.name
  x = f(x)
  print(name, x[0])
for f in dls.train.after_batch:
  name = f.name
  x = f(x)
  print(name, x[0])<jupyter_output>FlipItem TensorImage([[[176, 176, 175,  ..., 162, 160, 156],
         [166, 164, 162,  ..., 159, 160, 161],
         [152, 151, 148,  ..., 157, 159, 157],
         ...,
         [131, 105,  93,  ...,  23,  18,  20],
         [ 93,  59,  54,  ...,  38,  21,  28],
         [ 74,  50,  66,  ...,  32,  29,  23]],

        [[187, 187, 186,  ..., 176, 172, 169],
         [178, 176, 174,  ..., 173, 174, 174],
         [163, 161, 157,  ..., 169, 172, 170],
         ...,
         [146, 107,  85,  ...,  52,  49,  40],
         [120,  56,  44,  ...,  55,  41,  36],
         [111,  59,  62,  ...,  46,  37,  24]],

        [[195, 195, 193,  ..., 184, 180, 177],
         [187, 184, 181,  ..., 181, 182, 182],
         [169, 167, 164,  ..., 177, 180, 176],
         ...,
         [102,  61,  44,  ...,  14,  11,  10],
         [ 74,  30,  21,  ...,  22,   9,   9],
         [ 56,  26,  30,  ...,  29,  10,   7]]], dtype=torch.uint8)
RandomResizedCrop TensorImage([[[176, 176, 175,  ..., 162, 160, 156],
   [...]
{'fraction_non_alphanumeric': 0.13864306784660768, 'fraction_numerical': 0.10450906026127266, 'mean_word_length': 3.3872458410351203, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 34, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '442846', 'license': 'MIT', 'n_tokens_mistral': 2116, 'n_tokens_neox': 1731, 'n_words': 440, 'path': 'Computer Vision/04_DBlock_Summary.ipynb', 'repo_name': 'ANGELLOPARR/Practical-Deep-Learning-for-Coders-2.0', 'size': '4746'}