<jupyter_start><jupyter_text><jupyter_code>from fastai import *
from fastai.vision import *<jupyter_output><empty_output><jupyter_text>Content from this notebook has been inspired from the fast.ai [docs](https://docs.fast.ai/vision.data.html) and course-v3 Part 1. In this notebook we'll go through some ways of creating your own image dataset and training a simple classifier. We'll also go through how to create an `ImageDataBunch` object for Image Classification, Object Detection and Segmentation tasks. `fastai` library uses a style called the Data Block API to simplify this process  Creating your own dataset 梅？桜？We will scrape plum blossom and cherry blossom images from google and classifying them. fastai provides a [notebook](https://github.com/fastai/course-v3/blob/master/nbs/dl1/lesson2-download.ipynb) that illustrates how to scrape images.<jupyter_code>#get csv files with urls of images
#!wget https://raw.githubusercontent.com/Machine-Learning-Tokyo/MLT-x-fastai/master/mini-lessons/data/sakura.csv
#!wget https://raw.githubusercontent.com/Machine-Learning-Tokyo/MLT-x-fastai/master/mini-lessons/data/sakura.csv
#!mkdir data
#!mv sakura.csv data
#!mv ume.csv data
# Path is a pathlib object that you can use to perform bash style file system operations
path = Path('data')
path,path.ls()
folders = ['sakura','ume']
files = ['sakura.csv','ume.csv']
def get_images(folders):
    #make folders of every class and download image into folder from urls
    for i in range(len(folders)):
        dest = path/folders[i]
        dest.mkdir(parents=True, exist_ok=True)
        download_images(path/files[i], dest, max_pics=200)
# If you have problems download, try with `max_workers=0` to see exceptions:
get_images(folders)
def check_images(folders):
    for c in folders:
        print(c)
        #Check if the images in `path` aren't broken, maybe resize them and copy it in `dest`.
        verify_images(path/c, delete=True, max_size=1024)
check_images(folders)
len((path/folders[0]).ls()), len((path/folders[1]).ls())
np.random.seed(42)
data = ImageDataBunch.from_folder(path, train=".", ds_tfms=get_transforms(),valid_pct=0.1, size=224, num_workers=4)
data.show_batch(rows=3, figsize=(7,8))
learn = create_cnn(data, models.resnet34, metrics=error_rate)
learn.fit_one_cycle(4)
learn.unfreeze()
learn.fit_one_cycle(2, max_lr=slice(3e-5,3e-4))
interp = ClassificationInterpretation.from_learner(learn)
interp.plot_confusion_matrix()<jupyter_output><empty_output><jupyter_text>Do not the cells below if running on colab. It will crash because colab doesn't support ipython widgets<jupyter_code>from fastai.widgets import *
dataset, idxs = DatasetFormatter().from_toplosses(learn)
ImageCleaner(dataset, idxs, path)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11147421931735657, 'fraction_numerical': 0.012345679012345678, 'mean_word_length': 5.622596153846154, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '123445', 'license': 'MIT', 'n_tokens_mistral': 944, 'n_tokens_neox': 900, 'n_words': 252, 'path': 'mini-lessons/vision/sakura_ume.ipynb', 'repo_name': 'Machine-Learning-Tokyo/MLT-x-fastai', 'size': '2754'}