<commit_before>{
 "metadata": {
  "name": "",
  "signature": "sha256:26be3747209e886db08a0058c8a09bc10e73e0f751c2a409d339ea69f4a6124c"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Spectral Features"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "For classification, we're going to be using the new features in our arsenal: cherishing those \"spectral moments\" (centroid, bandwidth, skewness, kurtosis) and also examining other spectral statistics."
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Training Data"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "First, we want to analyze and feature extract a small collection of audio samples - storing their feature data as our \"training data\".  The commands below read all of the drum example .wav files from the MIR web site into an array, snareFileList.  \n",
      "\n",
      "Let's define a function to retrieve a list of URLs from a text file."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import urllib2\n",
      "\n",
      "def process_corpus(corpus_URL):\n",
      "    \"\"\"Read a list of files to process from the text file at corpusURL. Return a list of URLs\"\"\" \n",
      "    # Open and read each line\n",
      "    url_list_text_data = urllib2.urlopen(corpus_URL) # it's a file like object and works just like a file\n",
      "    for file_URL in url_list_text_data: # files are iterable\n",
      "        yield file_URL.rstrip()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 1
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Use these commands to read in a list of filenames (samples) in a directory, replacing the URL with a URL to a list of URLs (one per line) indicating where the audio / drum samples are stored."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "snares_URL = \"https://ccrma.stanford.edu/workshops/mir2014/SnareCorpus.txt\"\n",
      "snare_file_list = [audio_file_URL for audio_file_URL in process_corpus(snares_URL)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 2
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "kicks_URL = \"https://ccrma.stanford.edu/workshops/mir2014/KickCorpus.txt\"\n",
      "kick_file_list = [audio_file_URL for audio_file_URL in process_corpus(kicks_URL)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 3
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "To access the filenames contained in the array, use the square brackets [ ] to get to the element that you want to access. For example, to access the text URL file name of the first file in the list, you would type:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "snare_URL = snare_file_list[0]\n",
      "snare_URL"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 4,
       "text": [
        "'https://ccrma.stanford.edu/workshops/mir2014/audio/drum%20samples/snares/SNARE_01_01.WAV'"
       ]
      }
     ],
     "prompt_number": 4
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "When we feature extract a sample collection, we need to sequentially access audio files, segment them (or not), and feature extract them.  Loading a lot of audio files into memory is not always a feasible or desirable operation, so you will create a loop which loads an audio file, feature extracts it, and closes  the audio file.  Note that the only information that we retain in memory are the features that are extracted.\n",
      "\n",
      "Create a loop which reads in an audio file, extracts the zero crossing rate, and some spectral statistics. You can use the \"in\" operator to retrieve each audio file URL from process_corpus(), as used above. The feature information for each audio file (the \"feature vector\") should be stored as a feature array, with columns being the features and rows for each file. For example:\n",
      "\n",
      "        featuresSnare =\n",
      "\n",
      "             0.5730    1.9183    2.9713    0.0004 0.0002\n",
      "             0.4750    1.4834    2.4463    0.0004  0.0012\n",
      "             0.5900    2.2857    3.1788    0.0003  0.0041\n",
      "             0.5090    1.6622    2.6369    0.0004  0.0051\n",
      "             0.4860    1.4758    2.2085    0.0004  0.0021\n",
      "             0.6060    2.2119    3.2798    0.0004  0.0651\n",
      "             0.4990    2.0607    2.7654    0.0004  0.0721\n",
      "             0.6360    2.3153    3.0256    0.0003  0.0221\n",
      "             0.5490    2.0137    3.0342    0.0004  0.0016\n",
      "             0.5900    2.2857    3.1788    0.0003  0.0012\n",
      " \n",
      " Within your loop, here's a reminder how to read in your wav files, using an array of audio file URLs:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import urllib\n",
      "from essentia.standard import MonoLoader\n",
      "\n",
      "sample_rate = 44100\n",
      "urllib.urlretrieve(snare_URL, filename='/tmp/localfile.wav')\n",
      "audio = MonoLoader(filename = '/tmp/localfile.wav', sampleRate = sample_rate)()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 5
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Here's an example of how to feature extract the first frame from the current audio file..."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      " frameSize = 0.100 * sample_rate   # 100ms\n",
      " currentFrame = audio[0 : frameSize]\n",
      " # featuresSnare[i, 0] = zcr(currentFrame)\n",
      " # centroid, bandwidth, skew, kurtosis = spectralMoments(currentFrame, sample_rate, 8192)\n",
      " # featuresSnare[i, 1:4] = [centroid, bandwidth, skew, kurtosis]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 14
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "4.  First, extract all of the feature data for the kick drums and store it in a feature array.  (For my example, above, I'd put it in \"featuresKick\")\n",
      "\n",
      "5.  Next, extract all of the feature data for the snares, storing them in a different array. \n",
      "Again, the kick and snare features should be separated in two different arrays!\n",
      " \n",
      "OK, no more help.  The rest is up to you!"
     ]
    }
   ],
   "metadata": {}
  }
 ]
}<commit_msg>Added in spectral moments computation<commit_after>{
 "metadata": {
  "name": "",
  "signature": "sha256:620336a15b319b335fbc26f830d5f6b5507392500835305cf3d7fd579934ff6e"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Spectral Features"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "For classification, we're going to be using the new features in our arsenal: cherishing those \"spectral moments\" (centroid, bandwidth, skewness, kurtosis) and also examining other spectral statistics."
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Training Data"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "First, we want to analyze and feature extract a small collection of audio samples - storing their feature data as our \"training data\".  The commands below read all of the drum example .wav files from the MIR web site into an array, snareFileList.  \n",
      "\n",
      "Let's define a function to retrieve a list of URLs from a text file."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import urllib2\n",
      "\n",
      "def process_corpus(corpus_URL):\n",
      "    \"\"\"Read a list of files to process from the text file at corpusURL. Return a list of URLs\"\"\" \n",
      "    # Open and read each line\n",
      "    url_list_text_data = urllib2.urlopen(corpus_URL) # it's a file like object and works just like a file\n",
      "    for file_URL in url_list_text_data: # files are iterable\n",
      "        yield file_URL.rstrip()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 1
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Use these commands to read in a list of filenames (samples) in a directory, replacing the URL with a URL to a list of URLs (one per line) indicating where the audio / drum samples are stored."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "snares_URL = \"https://ccrma.stanford.edu/workshops/mir2014/SnareCorpus.txt\"\n",
      "snare_file_list = [audio_file_URL for audio_file_URL in process_corpus(snares_URL)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 2
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "kicks_URL = \"https://ccrma.stanford.edu/workshops/mir2014/KickCorpus.txt\"\n",
      "kick_file_list = [audio_file_URL for audio_file_URL in process_corpus(kicks_URL)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 3
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "To access the filenames contained in the array, use the square brackets [ ] to get to the element that you want to access. For example, to access the text URL file name of the first file in the list, you would type:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "snare_URL = snare_file_list[0]\n",
      "snare_URL"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 4,
       "text": [
        "'https://ccrma.stanford.edu/workshops/mir2014/audio/drum%20samples/snares/SNARE_01_01.WAV'"
       ]
      }
     ],
     "prompt_number": 4
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "When we feature extract a sample collection, we need to sequentially access audio files, segment them (or not), and feature extract them.  Loading a lot of audio files into memory is not always a feasible or desirable operation, so you will create a loop which loads an audio file, feature extracts it, and closes  the audio file.  Note that the only information that we retain in memory are the features that are extracted.\n",
      "\n",
      "Create a loop which reads in an audio file, extracts the zero crossing rate, and some spectral statistics. You can use the \"in\" operator to retrieve each audio file URL from process_corpus(), as used above. The feature information for each audio file (the \"feature vector\") should be stored as a feature array, with columns being the features and rows for each file. For example:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import numpy as np\n",
      "\n",
      "number_of_files = 20\n",
      "number_of_features = 5\n",
      "features_snare = np.zeros([number_of_files, number_of_features])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 15
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      " For example, when populated, features_snare might look like:\n",
      " \n",
      "    features_snare =\n",
      "\n",
      "             0.5730    1.9183    2.9713    0.0004  0.0002\n",
      "             0.4750    1.4834    2.4463    0.0004  0.0012\n",
      "             0.5900    2.2857    3.1788    0.0003  0.0041\n",
      "             0.5090    1.6622    2.6369    0.0004  0.0051\n",
      "             0.4860    1.4758    2.2085    0.0004  0.0021\n",
      "             0.6060    2.2119    3.2798    0.0004  0.0651\n",
      "             0.4990    2.0607    2.7654    0.0004  0.0721\n",
      "             0.6360    2.3153    3.0256    0.0003  0.0221\n",
      "             0.5490    2.0137    3.0342    0.0004  0.0016\n",
      "             0.5900    2.2857    3.1788    0.0003  0.0012\n",
      " \n",
      " Within your loop, here's a reminder how to read in your wav files, using an array of audio file URLs:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import urllib\n",
      "from essentia.standard import MonoLoader\n",
      "\n",
      "sample_rate = 44100\n",
      "urllib.urlretrieve(snare_URL, filename='/tmp/localfile.wav')\n",
      "audio = MonoLoader(filename = '/tmp/localfile.wav', sampleRate = sample_rate)()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "ename": "IOError",
       "evalue": "[Errno socket error] _ssl.c:320: Invalid SSL protocol variant specified.",
       "output_type": "pyerr",
       "traceback": [
        "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[0;31mIOError\u001b[0m                                   Traceback (most recent call last)",
        "\u001b[0;32m<ipython-input-16-f4740563a986>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0msample_rate\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m44100\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0murllib\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0murlretrieve\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msnare_URL\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfilename\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'/tmp/localfile.wav'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      6\u001b[0m \u001b[0maudio\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mMonoLoader\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilename\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m'/tmp/localfile.wav'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msampleRate\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msample_rate\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
        "\u001b[0;32m/sw/lib/python2.7/urllib.pyc\u001b[0m in \u001b[0;36murlretrieve\u001b[0;34m(url, filename, reporthook, data)\u001b[0m\n\u001b[1;32m     92\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0m_urlopener\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     93\u001b[0m         \u001b[0m_urlopener\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mFancyURLopener\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 94\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0m_urlopener\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mretrieve\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0murl\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfilename\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mreporthook\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     95\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0murlcleanup\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     96\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0m_urlopener\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
        "\u001b[0;32m/sw/lib/python2.7/urllib.pyc\u001b[0m in \u001b[0;36mretrieve\u001b[0;34m(self, url, filename, reporthook, data)\u001b[0m\n\u001b[1;32m    238\u001b[0m             \u001b[0;32mexcept\u001b[0m \u001b[0mIOError\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    239\u001b[0m                 \u001b[0;32mpass\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 240\u001b[0;31m         \u001b[0mfp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mopen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0murl\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    241\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    242\u001b[0m             \u001b[0mheaders\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minfo\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
        "\u001b[0;32m/sw/lib/python2.7/urllib.pyc\u001b[0m in \u001b[0;36mopen\u001b[0;34m(self, fullurl, data)\u001b[0m\n\u001b[1;32m    206\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    207\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mdata\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 208\u001b[0;31m                 \u001b[0;32mreturn\u001b[0m \u001b[0mgetattr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0murl\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    209\u001b[0m             \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    210\u001b[0m                 \u001b[0;32mreturn\u001b[0m \u001b[0mgetattr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0murl\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
        "\u001b[0;32m/sw/lib/python2.7/urllib.pyc\u001b[0m in \u001b[0;36mopen_https\u001b[0;34m(self, url, data)\u001b[0m\n\u001b[1;32m    435\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mrealhost\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mh\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mputheader\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Host'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrealhost\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    436\u001b[0m             \u001b[0;32mfor\u001b[0m \u001b[0margs\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0maddheaders\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mh\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mputheader\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 437\u001b[0;31m             \u001b[0mh\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mendheaders\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    438\u001b[0m             \u001b[0merrcode\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0merrmsg\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mheaders\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mh\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgetreply\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    439\u001b[0m             \u001b[0mfp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mh\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgetfile\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
        "\u001b[0;32m/sw/lib/python2.7/httplib.pyc\u001b[0m in \u001b[0;36mendheaders\u001b[0;34m(self, message_body)\u001b[0m\n\u001b[1;32m    989\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    990\u001b[0m             \u001b[0;32mraise\u001b[0m \u001b[0mCannotSendHeader\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 991\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_send_output\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmessage_body\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    992\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    993\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mrequest\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmethod\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0murl\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbody\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mheaders\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m{\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
        "\u001b[0;32m/sw/lib/python2.7/httplib.pyc\u001b[0m in \u001b[0;36m_send_output\u001b[0;34m(self, message_body)\u001b[0m\n\u001b[1;32m    842\u001b[0m             \u001b[0mmsg\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0mmessage_body\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    843\u001b[0m             \u001b[0mmessage_body\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 844\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmsg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    845\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mmessage_body\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    846\u001b[0m             \u001b[0;31m#message_body was not a string (i.e. it is a file) and\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
        "\u001b[0;32m/sw/lib/python2.7/httplib.pyc\u001b[0m in \u001b[0;36msend\u001b[0;34m(self, data)\u001b[0m\n\u001b[1;32m    804\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msock\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    805\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mauto_open\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 806\u001b[0;31m                 \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconnect\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    807\u001b[0m             \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    808\u001b[0m                 \u001b[0;32mraise\u001b[0m \u001b[0mNotConnected\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
        "\u001b[0;32m/sw/lib/python2.7/httplib.pyc\u001b[0m in \u001b[0;36mconnect\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m   1196\u001b[0m                 \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msock\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msock\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1197\u001b[0m                 \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_tunnel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1198\u001b[0;31m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msock\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mssl\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwrap_socket\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msock\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkey_file\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcert_file\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1199\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1200\u001b[0m     \u001b[0m__all__\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"HTTPSConnection\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
        "\u001b[0;32m/sw/lib/python2.7/ssl.pyc\u001b[0m in \u001b[0;36mwrap_socket\u001b[0;34m(sock, keyfile, certfile, server_side, cert_reqs, ssl_version, ca_certs, do_handshake_on_connect, suppress_ragged_eofs, ciphers)\u001b[0m\n\u001b[1;32m    390\u001b[0m                      \u001b[0mdo_handshake_on_connect\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mdo_handshake_on_connect\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    391\u001b[0m                      \u001b[0msuppress_ragged_eofs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0msuppress_ragged_eofs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 392\u001b[0;31m                      ciphers=ciphers)\n\u001b[0m\u001b[1;32m    393\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    394\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
        "\u001b[0;32m/sw/lib/python2.7/ssl.pyc\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, sock, keyfile, certfile, server_side, cert_reqs, ssl_version, ca_certs, do_handshake_on_connect, suppress_ragged_eofs, ciphers)\u001b[0m\n\u001b[1;32m    144\u001b[0m                                         \u001b[0mkeyfile\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcertfile\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    145\u001b[0m                                         \u001b[0mcert_reqs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mssl_version\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mca_certs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 146\u001b[0;31m                                         ciphers)\n\u001b[0m\u001b[1;32m    147\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mdo_handshake_on_connect\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    148\u001b[0m                 \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdo_handshake\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
        "\u001b[0;31mIOError\u001b[0m: [Errno socket error] _ssl.c:320: Invalid SSL protocol variant specified."
       ]
      }
     ],
     "prompt_number": 16
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Here's an example of how to feature extract the first frame from the current audio file, using Essentia's [ZeroCrossingRate](http://essentia.upf.edu/documentation/reference/streaming_ZeroCrossingRate.html) and [CentralMoments](http://essentia.upf.edu/documentation/reference/std_CentralMoments.html) classes..."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from essentia.standard import ZeroCrossingRate, CentralMoments, Spectrum, Windowing\n",
      "zcr = ZeroCrossingRate()\n",
      "hamming_window = Windowing(type = 'hamming') # we need to window the frame to avoid FFT artifacts.\n",
      "spectrum = Spectrum()\n",
      "central_moments = CentralMoments()\n",
      "\n",
      "frame_size = 0.100 * sample_rate   # 100ms\n",
      "file_index = 0 # to process a single file, this will be your file loop iteration\n",
      "\n",
      "current_frame = audio[0 : frame_size]\n",
      "features_snare[file_index, 0] = zcr(current_frame)\n",
      "spectral_magnitude = spectrum(hamming_window(current_frame))\n",
      "\n",
      "# Normalize the spectral magnitude\n",
      "norm_spectral_mag = spectral_magnitude / np.sum(spectral_magnitude);\n",
      "# Make index vector\n",
      "indices = np.arange(spectral_magnitude.shape[0])\n",
      "\n",
      "# Centroid, in spectral coefficient indices.\n",
      "centroid = np.sum(norm_spectral_mag * indices)\n",
      "centroid"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 23,
       "text": [
        "485.13728477961195"
       ]
      }
     ],
     "prompt_number": 23
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Now we use the magnitude of the spectrum to compute a 5 element array, corresponding to:\n",
      "# zeroth central moment (1.0)\n",
      "# first central moment (Not the centroid, which is the first moment!)\n",
      "# bandwidth\n",
      "# skew\n",
      "# kurtosis.\n",
      "spectral_moments = central_moments(spectral_magnitude)\n",
      "spectral_moments"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 32,
       "text": [
        "array([ 1.        ,  0.        ,  0.04079479,  0.00723291,  0.00497805], dtype=float32)"
       ]
      }
     ],
     "prompt_number": 32
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Remember the zeroth column of features_snare is the ZCR.\n",
      "features_snare[file_index, 1] = centroid\n",
      "features_snare[file_index, 2:5] = spectral_moments[2:5]\n",
      "features_snare[file_index]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 34,
       "text": [
        "array([  1.29251704e-01,   4.85137285e+02,   4.07947935e-02,\n",
        "         7.23291235e-03,   4.97804629e-03])"
       ]
      }
     ],
     "prompt_number": 34
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "4.  First, extract all of the feature data for the kick drums and store it in a feature array.  (For my example, above, I'd put it in \"features_kick\")\n",
      "\n",
      "5.  Next, extract all of the feature data for the snares, storing them in a different array. \n",
      "Again, the kick and snare features should be separated in two different arrays!\n",
      " \n",
      "OK, no more help.  The rest is up to you!"
     ]
    }
   ],
   "metadata": {}
  }
 ]
}
{'fraction_non_alphanumeric': 0.1696414004633406, 'fraction_numerical': 0.1755147322739583, 'mean_word_length': 2.889340101522843, 'pattern_counts': {'":': 206, '<': 5, '<?xml version=': 0, '>': 16, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4102555', 'n_tokens_mistral': 16794, 'n_tokens_neox': 13922, 'n_words': 1897}