<jupyter_start><jupyter_text>AI Open SessionYou will be building a Neural Net that can categorise images. We will be:1. Creating a dataset by downloading images from Google Images2. Training a model to recognise those images3. Testing it on new images to see if it can recognise those too.Based on a notebook from the [fastai course](https://course.fast.ai/) by <NAME> and <NAME>. Inspired by [<NAME>](https://www.pyimagesearch.com/2017/12/04/how-to-create-a-deep-learning-dataset-using-google-images/)*  Install dependenciesClick the "Play" button on the left of the cell below to run it.Alternatively, click on the cell, then use Shift-Enter to run it<jupyter_code>!pip install "torch==1.4" "torchvision==0.5.0"
from fastai.vision import *<jupyter_output><empty_output><jupyter_text>1. Creating your own dataset from Google Images  Get a list of URLs  Search and scroll Go to [Google Images](http://images.google.com) and search for the images you are interested in. The more specific you are in your Google Search, the better the results and the less manual pruning you will have to do.Scroll down until you've seen all the images you want to download, or until you see a button that says 'Show more results'. All the images you scrolled past are now available to download. To get more, click on the button, and continue scrolling. The maximum number of images Google Images shows is 700.It is a good idea to put things you want to exclude into the search query, for instance if you are searching for the Eurasian wolf, "canis lupus lupus", it might be a good idea to exclude other variants:    "canis lupus lupus" -dog -arctos -familiaris -baileyi -occidentalisYou can also limit your results to show particular types of images.  Download into file Now you must run some Javascript code in your browser which will save the URLs of all the images you want for you dataset.Before running the following commands, you may want to disable ad blocking extensions (uBlock, AdBlockPlus etc.) in Chrome. Otherwise the window.open() command doesn't work. In **Google Chrome** press **Ctrl-Shift-j** on Windows/Linux and Cmd-Opt-j on macOS, and a small window the javascript 'Console' will appear. In **Firefox** press **Ctrl-Shift-k** on Windows/Linux or Cmd-Opt-k on macOS. Paste the following into the console window and hit Enter```javascripturls=Array.from(document.querySelectorAll('.rg_i')).map(el=> el.hasAttribute('data-src')?el.getAttribute('data-src'):el.getAttribute('data-iurl'));window.open('data:text/csv;charset=utf-8,' + escape(urls.join('\n')));``` You should now have a file in your Downloads directory called `download`Use file explorer to go to the Downloads directory and **rename the file** - if these are spiderman images, for example, call it `spiderman.txt` Repeat the above steps until you have 2 or 3 different categories of images  Get Access to Google Drive1. Run the cell below2. Click on the link that appears3. Select your Google account and allow access4. Paste the access code into box below & hit Enter<jupyter_code>from google.colab import drive
drive.mount('/content/gdrive')
!ls "gdrive/My Drive/Colab Notebooks"<jupyter_output><empty_output><jupyter_text>Upload files to Google Drive1. Go to your [Google Drive](https://drive.google.com/drive/u/0/my-drive)2. Go to the `Colab Notebooks` folder3. Create a new folder called `data`4. Go to the new data folder5. Drag & drop your .txt files from File Explorer into the data folder6. Wait for the upload to completeIf that all went smoothly, when you run the cell below you should see the files you just uploaded.<jupyter_code>!ls "gdrive/My Drive/Colab Notebooks/data"<jupyter_output><empty_output><jupyter_text>Download the images to your Google DriveFirst, edit the list of categories below to match your .txt file names:<jupyter_code>categories = ['spiderman', 'superman', 'hulk', 'wonderwoman']
# Create some folders to store the images for each category
path = Path('gdrive/My Drive/Colab Notebooks/data')
for cat in categories:
  dest = path/cat
  dest.mkdir(parents=True, exist_ok=True)
# Now download the images - may take a few minutes!
for cat in categories:
  file=f"{cat}.txt"
  print(path/file)
  download_images(path/file, path/cat, max_pics=200)
# If you have problems downloading, try with `max_workers=0` to see exceptions:
# download_images(path/file, dest, max_pics=20, max_workers=0)<jupyter_output><empty_output><jupyter_text>Then we can remove any images that can't be opened:<jupyter_code>for c in categories:
    print(c)
    verify_images(path/c, delete=True, max_size=500)<jupyter_output><empty_output><jupyter_text>View dataGo to your Google Drive, and look at the images which are now in your new category sub-folders - delete anything that isn't a useful image for that category. Create a database of the new images<jupyter_code>np.random.seed(42)
data = ImageDataBunch.from_folder(path, train=".", valid_pct=0.2,
        ds_tfms=get_transforms(), size=224, num_workers=4).normalize(imagenet_stats)<jupyter_output><empty_output><jupyter_text>Good! Let's take a look at some of our pictures then.<jupyter_code>data.show_batch(rows=3, figsize=(7,8))<jupyter_output><empty_output><jupyter_text>2. Train modelCreate a "learner" this has our model, and the the functions needed to train it.We are using a "resnet34" which is a commonly used architecture for image recognition. The 34 refers to the number of "layers" (or how complex the model is)<jupyter_code>learn = cnn_learner(data, models.resnet34, callback_fns=[ShowGraph], metrics=accuracy)<jupyter_output><empty_output><jupyter_text>Now we need to find a good "Learning Rate" for our data. If we pick too low a Learning Rate, it will take forever to train our model. Too high and the process goes unstable and the model does not learn at all<jupyter_code>learn.lr_find(); learn.recorder.plot()<jupyter_output><empty_output><jupyter_text>We want a value towards the bottom of the steepest part of the slope, but not too close to where the curve starts to turn upwards againNow we start training our model:<jupyter_code>learn.fit_one_cycle(5, max_lr=3e-3)<jupyter_output><empty_output><jupyter_text>Save our model:<jupyter_code>learn.save('stage-1')<jupyter_output><empty_output><jupyter_text>By default, fastai stops us from training all the layers in the model at first. This allows the model to learn "the basics" about our categories. But now we want to do some "fine tuning" on the whole model, so we "unfreeze" it:<jupyter_code>learn.unfreeze()
learn.lr_find()
# If the plot is not showing try to give a start and end learning rate
# learn.lr_find(start_lr=1e-5, end_lr=1e-1)
learn.recorder.plot()<jupyter_output><empty_output><jupyter_text>You can think of our model as having 2 parts - the "generic image recognition" bit and the "recognise my categories" bit - we are going to train these two parts at different learning rates - the lower one is for the "generic image recognition"<jupyter_code>learn.fit_one_cycle(3, max_lr=slice(3e-5,1e-4))
learn.save('stage-2')<jupyter_output><empty_output><jupyter_text>Interpretation<jupyter_code>#learn.load('stage-2');
interp = ClassificationInterpretation.from_learner(learn)
interp.plot_confusion_matrix()
interp.plot_top_losses(9, figsize=(15,15))<jupyter_output><empty_output><jupyter_text>3. Putting Our Model Into Production First thing first, let's export the content of our `Learner` object for production:<jupyter_code>learn.export()<jupyter_output><empty_output><jupyter_text>This will create a file named 'export.pkl' in the directory where we were working that contains everything we need to deploy our model (the model, the weights but also some metadata like the classes or the transforms/normalization used).<jupyter_code>img = open_image(path/'spiderman'/'00000009.jpg')
img<jupyter_output><empty_output><jupyter_text>We create our `Learner` in production environment like this, just make sure that `path` contains the file 'export.pkl' from before.<jupyter_code>learn2 = load_learner(path)<jupyter_output><empty_output><jupyter_text>Now we can use our new learner to do a prediction on the image above<jupyter_code>pred_class,pred_idx,outputs = learn2.predict(img)
pred_class.obj<jupyter_output><empty_output><jupyter_text>Use ipywidgets to make an "App"<jupyter_code>import ipywidgets as widgets
from PIL import Image as pim
import numpy as np
import io
def pil2tensor(image:Union[NPImage,NPArray],dtype:np.dtype)->TensorImage:
    "Convert PIL style `image` array to torch style image tensor."
    a = np.asarray(image)
    if a.ndim==2 : a = np.expand_dims(a,2)
    a = np.transpose(a, (1, 0, 2))
    a = np.transpose(a, (2, 1, 0))
    return torch.from_numpy(a.astype(dtype, copy=False) )

def create_image(fn:bytes, div:bool=True, convert_mode:str='RGB')->Image:
    "Return `Image` object created from image in bytes `fn`."
    with warnings.catch_warnings():
        warnings.simplefilter("ignore", UserWarning) # EXIF warning from TiffPlugin
        x = PIL.Image.open(io.BytesIO(fn)).convert(convert_mode)
    x = pil2tensor(x,np.float32)
    if div: x.div_(255)
    return Image(x)

btn_upload = widgets.FileUpload(multiple=False)
lbl_pred = widgets.Label()
btn_run = widgets.Button(description='Classify')

out_pl = widgets.Output()
out_pl.clear_output()

def on_click_classify(change):
    img = create_image(btn_upload.data[-1])
    out_pl.clear_output()
    with out_pl: display(img)
    pred,pred_idx,probs = learn2.predict(img)
    lbl_pred.value = f'Prediction: {pred}; Probability: {probs[pred_idx]*100:.02f}%'

btn_run.on_click(on_click_classify)
widgets.VBox([widgets.Label('Select your image'), 
      btn_upload, btn_run, out_pl, lbl_pred])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09477258695204775, 'fraction_numerical': 0.011936612471701996, 'mean_word_length': 5.616065350578625, 'pattern_counts': {'":': 0, '<': 80, '<?xml version=': 0, '>': 83, 'https://': 3, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 12, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '19', 'dir': 'jupyter-structured-clean-dedup', 'id': '404738', 'license': 'Apache-2.0', 'n_tokens_mistral': 3087, 'n_tokens_neox': 2983, 'n_words': 1082, 'path': '01-FastAi-Image-Recognition.ipynb', 'repo_name': 'nigel-croft-bluefruit/AI-Open-Session', 'size': '9745'}