<commit_before>{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LHt2xZUSqIzV"
      },
      "source": [
        "\u003ca href=\"https://colab.research.google.com/github/magenta/mt3/blob/main/mt3/colab/music_transcription_with_transformers.ipynb\" target=\"_parent\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e\n",
        "\n",
        "# Music Transcription with Transformers\n",
        "\n",
        "This notebook is an interactive demo of a few [music transcription models](g.co/magenta/mt3) created by Google's [Magenta](g.co/magenta) team.  You can upload audio and have one of our models automatically transcribe it.\n",
        "\n",
        "\u003cimg src=\"https://magenta.tensorflow.org/assets/transcription-with-transformers/architecture_diagram.png\" alt=\"Transformer-based transcription architecture\"\u003e\n",
        "\n",
        "The notebook supports two pre-trained models:\n",
        "1. the piano transcription model from [our ISMIR 2021 paper](https://archives.ismir.net/ismir2021/paper/000030.pdf)\n",
        "1. the multi-instrument transcription model from [our ICLR 2022 paper](https://openreview.net/pdf?id=iMSjopcOn0p)\n",
        "\n",
        "**Caveat**: neither model is trained on singing.  If you upload audio with vocals, you will likely get weird results.  Multi-instrument transcription is still not a completely-solved problem and so you may get weird results regardless.\n",
        "\n",
        "In any case, we hope you have fun transcribing!  Feel free to tweet any interesting output at [@GoogleMagenta](https://twitter.com/googlemagenta)...\n",
        "\n",
        "### Instructions for running:\n",
        "\n",
        "* Make sure to use a GPU runtime, click:  __Runtime \u003e\u003e Change Runtime Type \u003e\u003e GPU__\n",
        "* Press ▶️ on the left of each cell to execute the cell\n",
        "* In the __Load Model__ cell, choose either `ismir2021` for piano transcription or `mt3` for multi-instrument transcription\n",
        "* In the __Upload Audio__ cell, choose an MP3 or WAV file from your computer when prompted\n",
        "* Transcribe the audio using the __Transcribe Audio__ cell (it may take a few minutes depending on the length of the audio)\n",
        "\n",
        "---\n",
        "\n",
        "This notebook sends basic usage data to Google Analytics.  For more information, see [Google's privacy policy](https://policies.google.com/privacy)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "ibSG_uu0QXgc"
      },
      "outputs": [],
      "source": [
        "# Copyright 2021 Google LLC. All Rights Reserved.\n",
        "\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "\n",
        "#     http://www.apache.org/licenses/LICENSE-2.0\n",
        "\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License.\n",
        "# ==============================================================================\n",
        "\n",
        "#@title Setup Environment\n",
        "#@markdown Install MT3 and its dependencies (may take a few minutes).\n",
        "\n",
        "!apt-get update -qq \u0026\u0026 apt-get install -qq libfluidsynth1 build-essential libasound2-dev libjack-dev\n",
        "\n",
        "!pip install --upgrade pip\n",
        "!pip install nest-asyncio\n",
        "!pip install pyfluidsynth\n",
        "# pin CLU for python 3.7 compatibility\n",
        "!pip install clu==0.0.7\n",
        "\n",
        "# install t5x\n",
        "!git clone --branch=main https://github.com/google-research/t5x\n",
        "# pin T5X for python 3.7 compatibility\n",<commit_msg>don't upgrade pip in transcription colab to avoid thorough and lengthy dependency resolution<commit_after>{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LHt2xZUSqIzV"
      },
      "source": [
        "\u003ca href=\"https://colab.research.google.com/github/magenta/mt3/blob/main/mt3/colab/music_transcription_with_transformers.ipynb\" target=\"_parent\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e\n",
        "\n",
        "# Music Transcription with Transformers\n",
        "\n",
        "This notebook is an interactive demo of a few [music transcription models](g.co/magenta/mt3) created by Google's [Magenta](g.co/magenta) team.  You can upload audio and have one of our models automatically transcribe it.\n",
        "\n",
        "\u003cimg src=\"https://magenta.tensorflow.org/assets/transcription-with-transformers/architecture_diagram.png\" alt=\"Transformer-based transcription architecture\"\u003e\n",
        "\n",
        "The notebook supports two pre-trained models:\n",
        "1. the piano transcription model from [our ISMIR 2021 paper](https://archives.ismir.net/ismir2021/paper/000030.pdf)\n",
        "1. the multi-instrument transcription model from [our ICLR 2022 paper](https://openreview.net/pdf?id=iMSjopcOn0p)\n",
        "\n",
        "**Caveat**: neither model is trained on singing.  If you upload audio with vocals, you will likely get weird results.  Multi-instrument transcription is still not a completely-solved problem and so you may get weird results regardless.\n",
        "\n",
        "In any case, we hope you have fun transcribing!  Feel free to tweet any interesting output at [@GoogleMagenta](https://twitter.com/googlemagenta)...\n",
        "\n",
        "### Instructions for running:\n",
        "\n",
        "* Make sure to use a GPU runtime, click:  __Runtime \u003e\u003e Change Runtime Type \u003e\u003e GPU__\n",
        "* Press ▶️ on the left of each cell to execute the cell\n",
        "* In the __Load Model__ cell, choose either `ismir2021` for piano transcription or `mt3` for multi-instrument transcription\n",
        "* In the __Upload Audio__ cell, choose an MP3 or WAV file from your computer when prompted\n",
        "* Transcribe the audio using the __Transcribe Audio__ cell (it may take a few minutes depending on the length of the audio)\n",
        "\n",
        "---\n",
        "\n",
        "This notebook sends basic usage data to Google Analytics.  For more information, see [Google's privacy policy](https://policies.google.com/privacy)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "ibSG_uu0QXgc"
      },
      "outputs": [],
      "source": [
        "# Copyright 2021 Google LLC. All Rights Reserved.\n",
        "\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "\n",
        "#     http://www.apache.org/licenses/LICENSE-2.0\n",
        "\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License.\n",
        "# ==============================================================================\n",
        "\n",
        "#@title Setup Environment\n",
        "#@markdown Install MT3 and its dependencies (may take a few minutes).\n",
        "\n",
        "!apt-get update -qq \u0026\u0026 apt-get install -qq libfluidsynth1 build-essential libasound2-dev libjack-dev\n",
        "\n",
        "!pip install nest-asyncio\n",
        "!pip install pyfluidsynth\n",
        "# pin CLU for python 3.7 compatibility\n",
        "!pip install clu==0.0.7\n",
        "\n",
        "# install t5x\n",
        "!git clone --branch=main https://github.com/google-research/t5x\n",
        "# pin T5X for python 3.7 compatibility\n",
{'fraction_non_alphanumeric': 0.14956521739130435, 'fraction_numerical': 0.022492753623188404, 'mean_word_length': 2.5941666666666667, 'pattern_counts': {'":': 24, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 16, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4328317', 'n_tokens_mistral': 2670, 'n_tokens_neox': 2550, 'n_words': 813}