<commit_before>{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rX8mhOLljYeM"
      },
      "source": [
        "##### Copyright 2019 The TensorFlow Authors."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "BZSlp3DAjdYf"
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3wF5wszaj97Y"
      },
      "source": [
        "# TensorFlow 2 quickstart for experts"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DUNzJc4jTj6G"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://www.tensorflow.org/tutorials/quickstart/advanced\"><img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" />View on TensorFlow.org</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/quickstart/advanced.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://github.com/tensorflow/docs/blob/master/site/en/tutorials/quickstart/advanced.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a href=\"https://storage.googleapis.com/tensorflow_docs/docs/site/en/tutorials/quickstart/advanced.ipynb\"><img src=\"https://www.tensorflow.org/images/download_logo_32px.png\" />Download notebook</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hiH7AC-NTniF"
      },
      "source": [
        "This is a [Google Colaboratory](https://colab.research.google.com/notebooks/welcome.ipynb) notebook file. Python programs are run directly in the browser—a great way to learn and use TensorFlow. To follow this tutorial, run the notebook in Google Colab by clicking the button at the top of this page.\n",
        "\n",
        "1. In Colab, connect to a Python runtime: At the top-right of the menu bar, select *CONNECT*.\n",
        "2. Run all the notebook code cells: Select *Runtime* > *Run all*."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eOsVdx6GGHmU"
      },
      "source": [
        "Download and install TensorFlow 2. Import TensorFlow into your program:\n",
        "\n",
        "Note: Upgrade `pip` to install the TensorFlow 2 package. See the [install guide](https://www.tensorflow.org/install) for details."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QS7DDTiZGRTo"
      },
      "source": [
        "Import TensorFlow into your program:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0trJmd6DjqBZ"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "print(\"TensorFlow version:\", tf.__version__)\n",
        "\n",
        "from tensorflow.keras.layers import Dense, Flatten, Conv2D\n",
        "from tensorflow.keras import Model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7NAbSZiaoJ4z"
      },
      "source": [
        "Load and prepare the [MNIST dataset](http://yann.lecun.com/exdb/mnist/)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JqFRS6K07jJs"
      },
      "outputs": [],
      "source": [
        "mnist = tf.keras.datasets.mnist\n",
        "\n",
        "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
        "x_train, x_test = x_train / 255.0, x_test / 255.0\n",
        "\n",
        "# Add a channels dimension\n",
        "x_train = x_train[..., tf.newaxis].astype(\"float32\")\n",
        "x_test = x_test[..., tf.newaxis].astype(\"float32\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k1Evqx0S22r_"
      },
      "source": [
        "Use `tf.data` to batch and shuffle the dataset:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8Iu_quO024c2"
      },
      "outputs": [],
      "source": [
        "train_ds = tf.data.Dataset.from_tensor_slices(\n",
        "    (x_train, y_train)).shuffle(10000).batch(32)\n",
        "\n",
        "test_ds = tf.data.Dataset.from_tensor_slices((x_test, y_test)).batch(32)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BPZ68wASog_I"
      },
      "source": [
        "Build the `tf.keras` model using the Keras [model subclassing API](https://www.tensorflow.org/guide/keras#model_subclassing):"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "h3IKyzTCDNGo"
      },
      "outputs": [],
      "source": [
        "class MyModel(Model):\n",
        "  def __init__(self):\n",
        "    super(MyModel, self).__init__()\n",
        "    self.conv1 = Conv2D(32, 3, activation='relu')\n",
        "    self.flatten = Flatten()\n",
        "    self.d1 = Dense(128, activation='relu')\n",
        "    self.d2 = Dense(10)\n",
        "\n",
        "  def call(self, x):\n",
        "    x = self.conv1(x)\n",
        "    x = self.flatten(x)\n",
        "    x = self.d1(x)\n",
        "    return self.d2(x)\n",
        "\n",
        "# Create an instance of the model\n",
        "model = MyModel()"
      ]<commit_msg>Updated correct link for model subclassing API<commit_after>{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rX8mhOLljYeM"
      },
      "source": [
        "##### Copyright 2019 The TensorFlow Authors."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "BZSlp3DAjdYf"
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3wF5wszaj97Y"
      },
      "source": [
        "# TensorFlow 2 quickstart for experts"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DUNzJc4jTj6G"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://www.tensorflow.org/tutorials/quickstart/advanced\"><img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" />View on TensorFlow.org</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/quickstart/advanced.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://github.com/tensorflow/docs/blob/master/site/en/tutorials/quickstart/advanced.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a href=\"https://storage.googleapis.com/tensorflow_docs/docs/site/en/tutorials/quickstart/advanced.ipynb\"><img src=\"https://www.tensorflow.org/images/download_logo_32px.png\" />Download notebook</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hiH7AC-NTniF"
      },
      "source": [
        "This is a [Google Colaboratory](https://colab.research.google.com/notebooks/welcome.ipynb) notebook file. Python programs are run directly in the browser—a great way to learn and use TensorFlow. To follow this tutorial, run the notebook in Google Colab by clicking the button at the top of this page.\n",
        "\n",
        "1. In Colab, connect to a Python runtime: At the top-right of the menu bar, select *CONNECT*.\n",
        "2. Run all the notebook code cells: Select *Runtime* > *Run all*."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eOsVdx6GGHmU"
      },
      "source": [
        "Download and install TensorFlow 2. Import TensorFlow into your program:\n",
        "\n",
        "Note: Upgrade `pip` to install the TensorFlow 2 package. See the [install guide](https://www.tensorflow.org/install) for details."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QS7DDTiZGRTo"
      },
      "source": [
        "Import TensorFlow into your program:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0trJmd6DjqBZ"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "print(\"TensorFlow version:\", tf.__version__)\n",
        "\n",
        "from tensorflow.keras.layers import Dense, Flatten, Conv2D\n",
        "from tensorflow.keras import Model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7NAbSZiaoJ4z"
      },
      "source": [
        "Load and prepare the [MNIST dataset](http://yann.lecun.com/exdb/mnist/)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JqFRS6K07jJs"
      },
      "outputs": [],
      "source": [
        "mnist = tf.keras.datasets.mnist\n",
        "\n",
        "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
        "x_train, x_test = x_train / 255.0, x_test / 255.0\n",
        "\n",
        "# Add a channels dimension\n",
        "x_train = x_train[..., tf.newaxis].astype(\"float32\")\n",
        "x_test = x_test[..., tf.newaxis].astype(\"float32\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k1Evqx0S22r_"
      },
      "source": [
        "Use `tf.data` to batch and shuffle the dataset:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8Iu_quO024c2"
      },
      "outputs": [],
      "source": [
        "train_ds = tf.data.Dataset.from_tensor_slices(\n",
        "    (x_train, y_train)).shuffle(10000).batch(32)\n",
        "\n",
        "test_ds = tf.data.Dataset.from_tensor_slices((x_test, y_test)).batch(32)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BPZ68wASog_I"
      },
      "source": [
        "Build the `tf.keras` model using the Keras [model subclassing API](https://www.tensorflow.org/guide/keras/custom_layers_and_models#the_model_class):"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "h3IKyzTCDNGo"
      },
      "outputs": [],
      "source": [
        "class MyModel(Model):\n",
        "  def __init__(self):\n",
        "    super(MyModel, self).__init__()\n",
        "    self.conv1 = Conv2D(32, 3, activation='relu')\n",
        "    self.flatten = Flatten()\n",
        "    self.d1 = Dense(128, activation='relu')\n",
        "    self.d2 = Dense(10)\n",
        "\n",
        "  def call(self, x):\n",
        "    x = self.conv1(x)\n",
        "    x = self.flatten(x)\n",
        "    x = self.d1(x)\n",
        "    return self.d2(x)\n",
        "\n",
        "# Create an instance of the model\n",
        "model = MyModel()"
      ]
{'fraction_non_alphanumeric': 0.17779301935680536, 'fraction_numerical': 0.013412589544276788, 'mean_word_length': 1.9743880326382592, 'pattern_counts': {'":': 136, '<': 47, '<?xml version=': 0, '>': 49, 'https://': 24, 'lorem ipsum': 0, 'www.': 16, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '517442', 'n_tokens_mistral': 4625, 'n_tokens_neox': 4486, 'n_words': 904}