<jupyter_start><jupyter_text>Computer Vision Interpret [`vision.interpret`](/vision.interpret.htmlvision.interpret) is the module that implements custom [`Interpretation`](/train.htmlInterpretation) classes for different vision tasks by inheriting from it.<jupyter_code>from fastai.gen_doc.nbdoc import *
from fastai.vision import *
from fastai.vision.interpret import *
show_doc(SegmentationInterpretation)
show_doc(SegmentationInterpretation.top_losses)
show_doc(SegmentationInterpretation._interp_show)
show_doc(SegmentationInterpretation.show_xyz)
show_doc(SegmentationInterpretation._generate_confusion)
show_doc(SegmentationInterpretation._plot_intersect_cm)<jupyter_output><empty_output><jupyter_text>Let's show how [`SegmentationInterpretation`](/vision.interpret.htmlSegmentationInterpretation) can be used once we train a segmentation model.  train<jupyter_code>camvid = untar_data(URLs.CAMVID_TINY)
path_lbl = camvid/'labels'
path_img = camvid/'images'
codes = np.loadtxt(camvid/'codes.txt', dtype=str)
get_y_fn = lambda x: path_lbl/f'{x.stem}_P{x.suffix}'
data = (SegmentationItemList.from_folder(path_img)
        .split_by_rand_pct()
        .label_from_func(get_y_fn, classes=codes)
        .transform(get_transforms(), tfm_y=True, size=128)
        .databunch(bs=16, path=camvid)
        .normalize(imagenet_stats))
data.show_batch(rows=2, figsize=(7,5))
learn = unet_learner(data, models.resnet18)
learn.fit_one_cycle(3,1e-2)
learn.save('mini_train')
jekyll_warn("Following results will not make much sense with this underperforming model but functionality will be explained with ease")<jupyter_output><empty_output><jupyter_text>interpret<jupyter_code>interp = SegmentationInterpretation.from_learner(learn)<jupyter_output><empty_output><jupyter_text>Since `FlattenedLoss of CrossEntropyLoss()` is used we reshape and then take the mean of pixel losses per image. In order to do so we need to pass `sizes:tuple` to `top_losses()`<jupyter_code>top_losses, top_idxs = interp.top_losses(sizes=(128,128))
top_losses, top_idxs
plt.hist(to_np(top_losses), bins=20);plt.title("Loss Distribution");<jupyter_output><empty_output><jupyter_text>Next, we can generate a confusion matrix similar to what we usually have for classification. Two confusion matrices are generated: `mean_cm` which represents the global label performance and `single_img_cm` which represents the same thing but for each individual image in dataset.Values in the matrix are calculated as:\begin{align}\ CM_{ij} & = IOU(Predicted , True | True) \\\end{align}Or in plain english: ratio of pixels of predicted label given the true pixels<jupyter_code>learn.data.classes
mean_cm, single_img_cm = interp._generate_confusion()
mean_cm.shape, single_img_cm.shape<jupyter_output><empty_output><jupyter_text>`_plot_intersect_cm` first displays a dataframe showing per class score using the IOU definition we made earlier. These are the diagonal values from the confusion matrix which is displayed after. `NaN` indicate that these labels were not present in our dataset, in this case validation set. As you can imagine it also helps you to maybe construct a better representing validation set.<jupyter_code>df = interp._plot_intersect_cm(mean_cm, "Mean of Ratio of Intersection given True Label")<jupyter_output><empty_output><jupyter_text>Next let's look at the single worst prediction in our dataset. It looks like this dummy model just predicts everything as `Road` :)<jupyter_code>i = top_idxs[0]
df = interp._plot_intersect_cm(single_img_cm[i], f"Ratio of Intersection given True Label, Image:{i}")<jupyter_output><empty_output><jupyter_text>Finally we will visually inspect this single prediction<jupyter_code>interp.show_xyz(i, sz=15)
jekyll_warn("""With matplotlib colormaps the max number of unique qualitative colors is 20. 
So if len(classes) > 20 then close class indexes may be plotted with the same color.
Let's fix this together :)""")
interp.c2i
show_doc(ObjectDetectionInterpretation)
jekyll_warn("ObjectDetectionInterpretation is not implemented yet. Feel free to implement it :)")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08800585080448561, 'fraction_numerical': 0.007069722086786933, 'mean_word_length': 7.3905930470347645, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 34, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '17535', 'license': 'Apache-2.0', 'n_tokens_mistral': 1309, 'n_tokens_neox': 1263, 'n_words': 354, 'path': 'docs_src/vision.interpret.ipynb', 'repo_name': 'iamanuragmahendru/fastai', 'size': '4102'}