<jupyter_start><jupyter_text>Image classification - Real project example with CIFAR-10 datasetThis notebook will exemplify how to do image classification in Ipyannotator using one of the most commonly used datasets in deep learning: [CIFAR-10](https://www.cs.toronto.edu/~kriz/cifar.html). The dataset contains 60000 32x32 images in 10 classes.  Setup data for a fictive greenfield projectThe first step is to download the dataset. The next cell will use the [pooch](https://github.com/fatiando/pooch) library to easily fetch the data files from s3.<jupyter_code>from pathlib import Path
import pooch
file_path = pooch.retrieve(
    # URL to one of Pooch's test files
    url="https://s3.amazonaws.com/fast-ai-imageclas/cifar10.tgz",
    known_hash="sha256:637c5814e11aefcb6ee76d5f59c67ddc8de7f5b5077502a195b0833d1e3e4441",
)<jupyter_output><empty_output><jupyter_text>Pooch retrieves the data to your local machine. The next cell will display the exact path where the files were downloaded.<jupyter_code>file_path<jupyter_output><empty_output><jupyter_text>Since the CITAR-10 dataset is downloaded as a compressed `tar` file, the next cells will extract the files. Ipyannotator has some internal tools to manipulate data, which is the case of the `_extract_tar` function used below to extract the files and move them to a new folder `tmp`.<jupyter_code>from ipyannotator.datasets.download import _extract_tar
_extract_tar(file_path, Path('/tmp'))<jupyter_output><empty_output><jupyter_text>Ipyannotator uses the following path setup:```project_root││─── images│└─── results```The `project root` is the folder that contains folders for the image raw data and the annotation results. `Images` is the folder that contains all images that can displayed by the navigator and are used to create the dataset by the annotator. The `results` folder stores the dataset. The folder names can be chosen by the user. By default Ipyannotator uses `images` and `results`.The next cell defines a project root called `user_project` and creates a new folder called `images` inside of it.<jupyter_code>project_root = Path('user_project')
(project_root / 'images').mkdir(parents=True, exist_ok=True)<jupyter_output><empty_output><jupyter_text>Once the folder structure is created, the files are downloaded and extracted, they will be moved to the `images` folder. The next cell copies the 200 random images from the CIFAR-10 dataset to the Ipyannotator path structure.<jupyter_code>import shutil
import random

classes = "airplane  automobile  bird  cat  deer  dog  frog  horse  ship  truck".split()
for i in range(1, 200):
    rand_class = random.randint(0, 9)
    shutil.copy(
        Path('/tmp') / "cifar10/train" / classes[rand_class] / f"{i:04}.png",
        project_root / 'images')<jupyter_output><empty_output><jupyter_text>Story In the current step we have 200 images from random classes and we need to classify them. The first step is to have a look at the images before checking which classes need to be set in the classification.Ipyannotator uses an API to ensure easy access to the annotators. The next cell will import the `Annotator` factory, that provides a simple function `InputImage` to explore images.<jupyter_code>from ipyannotator.mltypes import InputImage
from ipyannotator.annotator import Annotator<jupyter_output><empty_output><jupyter_text>CIFAR-10 uses 32x32 px color images. The small size of the images makes the visualization difficult. Therefore, the `fit_canvas` property will be used in the next cell to improve the visual appearance, displaying the image at the same size of the `InputImage`.<jupyter_code>input_ = InputImage(image_width=100, image_height=100, image_dir='images', fit_canvas=True)<jupyter_output><empty_output><jupyter_text>To use the `Annotator` factory, a simple pair of `Input/Output` is used. Omitting the output, Ipyannotator will use `NoOutput` as default. In this case, the user can only navigate across the input images and labels/classes are not displayed in the explore function.<jupyter_code>Annotator(input_).explore()<jupyter_output><empty_output><jupyter_text>As mentioned before, the Ipyannotator path setup provides some default names for the folders. These names can be changed using the `Settings` property. The next cells demonstrates how to use the settings property to customize the folder structure.<jupyter_code>from ipyannotator.base import Settings
settings = Settings(
    project_path=Path('user_project'),
    image_dir='images',
    result_dir='results'
)
anni = Annotator(input_, settings=settings)
anni.explore()<jupyter_output><empty_output><jupyter_text>Once the user has gained an overview on the input image dataset, the user can define classes to label the images. Using `OutputLabel` you can define the classes that will be used to label the images. The `class_labels` property at `OutputLabel` allows an array of classes to be used in the classification. Since CIFAR-10 uses 10 classes, these are going to be used in the next cells.<jupyter_code>from ipyannotator.mltypes import OutputLabel
output_ = OutputLabel(class_labels=classes)
anni = Annotator(input_, output_, settings)
anni.explore()<jupyter_output><empty_output><jupyter_text>To create your own dataset you just have to call the `create` step at the `Annotator` factory. This step will allow users to associate the classes to a image.<jupyter_code>anni.create()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07171828908554573, 'fraction_numerical': 0.018067846607669618, 'mean_word_length': 5.841109709962169, 'pattern_counts': {'":': 0, '<': 45, '<?xml version=': 0, '>': 45, 'https://': 3, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '498071', 'license': 'Apache-2.0', 'n_tokens_mistral': 1625, 'n_tokens_neox': 1548, 'n_words': 627, 'path': 'nbs/20_image_classification_user_story.ipynb', 'repo_name': 'itepifanio/ipyannotator', 'size': '5424'}