<commit_before>{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-0BQWhvAP2jb"
      },
      "source": [
        "\n",
        "\u003ca href=\"https://colab.research.google.com/github/google-research/t5x/blob/main/t5x/notebooks/introduction.ipynb\" target=\"_parent\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bqZYp90PIa1t"
      },
      "source": [
        "# Overview\n",
        "\n",
        "T5X is a modular, composable, research-friendly framework for high-performance, configurable, self-service training, evaluation, and inference of sequence models (starting with language) at many scales.\n",
        "\n",
        "It is essentially a new and improved implementation of the [T5 codebase](https://github.com/google-research/text-to-text-transfer-transformer/blob/main/README.md) (based on Mesh TensorFlow) in JAX and Flax.\n",
        "\n",
        "# Getting Started\n",
        "\n",
        "In the following Colab, we present an introductory tutorial to get you started interacting with the T5X codebase. In particular, we'll introduce the major components of the T5X codebase and get you started running training, inference, and evaluation on natural text inputs.\n",
        "\n",
        "Note: If you are a using public colab, please use its `Connect to a local runtime` option by following the [setup guide](https://github.com/google-research/t5x/blob/main/t5x/notebooks/README.md)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jIGSIHzD7YPO"
      },
      "outputs": [],
      "source": [
        "import functools\n",
        "import os\n",
        "\n",
        "import tensorflow as tf\n",
        "import jax\n",
        "from jax import random\n",
        "from jax.experimental import multihost_utils\n",
        "import jax.numpy as jnp\n",
        "from flax import linen\n",<commit_msg>fix import statements.<commit_after>{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-0BQWhvAP2jb"
      },
      "source": [
        "\n",
        "\u003ca href=\"https://colab.research.google.com/github/google-research/t5x/blob/main/t5x/notebooks/introduction.ipynb\" target=\"_parent\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bqZYp90PIa1t"
      },
      "source": [
        "# Overview\n",
        "\n",
        "T5X is a modular, composable, research-friendly framework for high-performance, configurable, self-service training, evaluation, and inference of sequence models (starting with language) at many scales.\n",
        "\n",
        "It is essentially a new and improved implementation of the [T5 codebase](https://github.com/google-research/text-to-text-transfer-transformer/blob/main/README.md) (based on Mesh TensorFlow) in JAX and Flax.\n",
        "\n",
        "# Getting Started\n",
        "\n",
        "In the following Colab, we present an introductory tutorial to get you started interacting with the T5X codebase. In particular, we'll introduce the major components of the T5X codebase and get you started running training, inference, and evaluation on natural text inputs.\n",
        "\n",
        "Note: If you are a using public colab, please use its `Connect to a local runtime` option by following the [setup guide](https://github.com/google-research/t5x/blob/main/t5x/notebooks/README.md)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jIGSIHzD7YPO"
      },
      "outputs": [],
      "source": [
        "import functools\n",
        "import os\n",
        "\n",
        "import clu.data.dataset_iterator\n",
        "import tensorflow as tf\n",
        "import jax\n",
        "from jax import random\n",
        "from jax.experimental import multihost_utils\n",
        "import jax.numpy as jnp\n",
        "from flax import linen\n",
{'fraction_non_alphanumeric': 0.14117369785529107, 'fraction_numerical': 0.015083667216592034, 'mean_word_length': 2.4420113544201136, 'pattern_counts': {'":': 30, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 8, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4491050', 'n_tokens_mistral': 1408, 'n_tokens_neox': 1337, 'n_words': 351}