<commit_before>{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "name": "aiy_retrain_classification.ipynb",
      "private_outputs": true,
      "provenance": [],
      "collapsed_sections": [
        "license"
      ],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "license"
      },
      "source": [
        "##### *Copyright 2020 Google LLC*\n",
        "*Licensed under the Apache License, Version 2.0 (the \"License\")*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "both",
        "id": "rKwqeqWBXANA"
      },
      "source": [
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hRTa3Ee15WsJ"
      },
      "source": [
        "# Retrain a classification model for the AIY Vision Kit (with TF1)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TaX0smDP7xQY"
      },
      "source": [
        "In this tutorial, we'll use TensorFlow to retrain an image classification model (MobileNet) with a flowers dataset, and compile it into the TensorFlow format that's compatible with the AIY Vision Bonnet (included in the AIY Vision Kit).\n",
        "\n",
        "All the code examples on this page are executable from your web browser, but you must execute them in order. So although you can run each code block individually, we recommend that you run everything by selecting **Runtime > Run all** in the Colab toolbar. That allows the training to get started while you read the tutorial.\n",
        "\n",
        "**Note:** Although the code on this page executes in the cloud, you **should not run this on your AIY Vision Kit**. This web app is still quite complex and the Raspberry Pi Zero cannot run it effectively (it will be very slow and you'll have a bad time). Instead, use your desktop computer to run the Colab tutorial and then transfer the files to the Raspberry Pi (as described below)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "viewin-badges"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/google/aiyprojects-raspbian/blob/aiyprojects/tutorials/vision/aiy_retrain_classification.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open in Colab\"></a>\n",
        "&nbsp;&nbsp;&nbsp;&nbsp;\n",
        "<a href=\"https://github.com/google/aiyprojects-raspbian/blob/aiyprojects/tutorials/vision/aiy_retrain_classification.ipynb\" target=\"_parent\"><img src=\"https://img.shields.io/static/v1?logo=GitHub&label=&color=333333&style=flat&message=View%20on%20GitHub\" alt=\"View in GitHub\"></a>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GTCYQg_be8C0"
      },
      "source": [
        "## Import the required libraries"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XZBTCYDICwOG"
      },
      "source": [
        "First, we need to remove the version of TensorFlow that's included with Google Colab by default, and replace it with TensorFlow 1.7, as required by the following training scripts."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3Ms7m4MAPZ1R"
      },
      "source": [
        "! pip uninstall tensorflow tensorboard -y"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hLjaKNp9inlt"
      },
      "source": [
        "! pip install -I absl-py==0.9 jupyter-client==6.1.5 tornado==5.1.0 folium==0.2.1 imgaug==0.2.5 tensorflow==1.7 tensorboard==1.7"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_Uq-26JBk4Z8"
      },
      "source": [
        "The output above  might say \"You must restart the runtime\" but you should ignore that. You **do not** need to restart."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iBMcobPHdD8O"
      },
      "source": [
        "import tensorflow as tf\n",
        "assert tf.__version__.startswith('1.7')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v77rlkCKW0IJ"
      },
      "source": [
        "## Prepare the training data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j4QOy2uA3P_p"
      },
      "source": [
        "First let's download and organize the flowers dataset we'll use to retrain the model (it contains 5 flower classes).\n",
        "\n",
        "Pay attention to this part so you can reproduce it with your own images dataset. In particular, notice that the \"flower_photos\" directory contains an appropriately-named directory for each class. (If you want to retrain the model with different photos, we'll discuss this more at the very end.)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "q9pYqideeXiI"
      },
      "source": [<commit_msg>Upgrade Colab TF version to 1.13.1<commit_after>{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "name": "aiy_retrain_classification.ipynb",
      "private_outputs": true,
      "provenance": [],
      "collapsed_sections": [
        "license"
      ],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "license"
      },
      "source": [
        "##### *Copyright 2020 Google LLC*\n",
        "*Licensed under the Apache License, Version 2.0 (the \"License\")*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "both",
        "id": "rKwqeqWBXANA"
      },
      "source": [
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hRTa3Ee15WsJ"
      },
      "source": [
        "# Retrain a classification model for the AIY Vision Kit (with TF1)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TaX0smDP7xQY"
      },
      "source": [
        "In this tutorial, we'll use TensorFlow to retrain an image classification model (MobileNet) with a flowers dataset, and compile it into the TensorFlow format that's compatible with the AIY Vision Bonnet (included in the AIY Vision Kit).\n",
        "\n",
        "All the code examples on this page are executable from your web browser, but you must execute them in order. So although you can run each code block individually, we recommend that you run everything by selecting **Runtime > Run all** in the Colab toolbar. That allows the training to get started while you read the tutorial.\n",
        "\n",
        "**Note:** Although the code on this page executes in the cloud, you **should not run this on your AIY Vision Kit**. This web app is still quite complex and the Raspberry Pi Zero cannot run it effectively (it will be very slow and you'll have a bad time). Instead, use your desktop computer to run the Colab tutorial and then transfer the files to the Raspberry Pi (as described below)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "viewin-badges"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/google/aiyprojects-raspbian/blob/aiyprojects/tutorials/vision/aiy_retrain_classification.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open in Colab\"></a>\n",
        "&nbsp;&nbsp;&nbsp;&nbsp;\n",
        "<a href=\"https://github.com/google/aiyprojects-raspbian/blob/aiyprojects/tutorials/vision/aiy_retrain_classification.ipynb\" target=\"_parent\"><img src=\"https://img.shields.io/static/v1?logo=GitHub&label=&color=333333&style=flat&message=View%20on%20GitHub\" alt=\"View in GitHub\"></a>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GTCYQg_be8C0"
      },
      "source": [
        "## Import the required libraries"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XZBTCYDICwOG"
      },
      "source": [
        "First, we need to remove the version of TensorFlow that's included with Google Colab by default, and replace it with TensorFlow 1.13.1, as required by the following training scripts."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3Ms7m4MAPZ1R"
      },
      "source": [
        "! pip uninstall tensorflow tensorboard -y"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hLjaKNp9inlt"
      },
      "source": [
        "! pip install -I absl-py==0.9 jupyter-client==6.1.5 tornado==5.1.0 folium==0.2.1 imgaug==0.2.5 tensorflow==1.13.1 tensorboard==1.13.1"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_Uq-26JBk4Z8"
      },
      "source": [
        "The output above  might say \"You must restart the runtime\" but you should ignore that. You **do not** need to restart."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iBMcobPHdD8O"
      },
      "source": [
        "import tensorflow as tf\n",
        "assert tf.__version__.startswith('1.13.1')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v77rlkCKW0IJ"
      },
      "source": [
        "## Prepare the training data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j4QOy2uA3P_p"
      },
      "source": [
        "First let's download and organize the flowers dataset we'll use to retrain the model (it contains 5 flower classes).\n",
        "\n",
        "Pay attention to this part so you can reproduce it with your own images dataset. In particular, notice that the \"flower_photos\" directory contains an appropriately-named directory for each class. (If you want to retrain the model with different photos, we'll discuss this more at the very end.)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "q9pYqideeXiI"
      },
      "source": [
{'fraction_non_alphanumeric': 0.14423314398616943, 'fraction_numerical': 0.013007326911994731, 'mean_word_length': 2.2796976241900646, 'pattern_counts': {'":': 158, '<': 15, '<?xml version=': 0, '>': 17, 'https://': 10, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3012035', 'n_tokens_mistral': 3908, 'n_tokens_neox': 3768, 'n_words': 1113}