<commit_before>{
 "metadata": {
  "name": "",
  "signature": "sha256:d133ba0fcdad4ff8df0e4a92d5adc801b4acce5895d609546f4f0d805d6affb8"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The Scientific Python Ecosystem\n",
      "===\n",
      "\n",
      " - [IPython](http://ipython.org/):\n",
      "   a must-have whether or not you're doing scientific programming.\n",
      " - [NumPy](http://numpy.scipy.org/):\n",
      "  provides the numerical array, the \"standard interface\" of scientific Python.\n",
      " - [SciPy](http://docs.scipy.org/doc/reference/):\n",
      "   a collection of numerical algorithms and domain-specific modules, from statistics to signal processing to ODE integration.\n",
      " - [matplotlib](http://matplotlib.org/):\n",
      "   plotting functions that mimic Matlab's interface and integrate well with IPython.\n",
      " - [pandas](http://pandas.pydata.org/):\n",
      "   provides the `DataFrame`, the emerging second standard interface of scientific Python.\n",
      " - [SymPy](http://sympy.org/):\n",
      "   symbolic Python.\n",
      " - [Statsmodels](http://statsmodels.sourceforge.net/):\n",
      "   because some statistics functions and models are missing from SciPy.\n",
      " - [scikit-learn](http://scikit-learn.org/stable/):\n",
      "   a popular machine-learning library.\n",
      " - [rpy2](http://rpy.sourceforge.net/):\n",
      "   call R from Python, if your test or model is not in SciPy, statsmodels, or scikit-learn.\n",
      " - [nose](https://nose.readthedocs.org/en/latest/):\n",
      "   an easy-to-use Python testing framework.\n",
      "   \n",
      "When you ran `conda install -n python3 --file conda-packages.txt` back on the first week,\n",
      "all these packages were installed in your `python3` environment,\n",
      "with two exceptions:\n",
      "\n",
      " - I forgot `nose`.\n",
      "   It's on `conda` so `conda install -n python3 nose` will get it.\n",
      " - `rpy2` is not on `conda` for Python 3.\n",
      "   If `pip` can't install it, there are Windows binaries [here](http://www.lfd.uci.edu/~gohlke/pythonlibs/#rpy2) although it might be tricky to get working (need to make sure you're using the same R version referenced on that page).\n",
      "   On OSX it should compile although there may be some dependencies you need to install.\n",
      "   Check the documentation and search the [issues page](https://bitbucket.org/lgautier/rpy2/issues?status=new&status=open) if you have trouble.\n",
      "   There are probably precompiled OSX binaries available but I was not able to find any with a quick search.\n",
      "   (really everyone should use Linux and then this would all just work)\n",
      "   \n",
      "   \n",
      "###Credit where credit is due\n",
      "\n",
      "Again, I am stealing someone else's tutorial, this time thanks are due to <NAME> Continuum, at the PyData conference (2013 I think).\n",
      "I will put the video below.\n",
      "Slides are available [here](http://www.slideshare.net/PyData/introduction-to-numpy)."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from IPython.display import VimeoVideo\n",
      "\n",
      "VimeoVideo(63253083)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "\n",
        "        <iframe\n",
        "            width=\"400\"\n",
        "            height=300\"\n",
        "            src=\"https://player.vimeo.com/video/63253083\"\n",
        "            frameborder=\"0\"\n",
        "            allowfullscreen\n",
        "        ></iframe>\n",
        "        "
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 101,
       "text": [
        "<IPython.lib.display.VimeoVideo at 0x7ff106c06d68>"
       ]
      }
     ],
     "prompt_number": 101
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Array programming\n",
      "\n",
      "Array programming is a computing technique that generalizes scalar operations to apply transparently to vectors, matrices, and high-dimensional arrays.\n",
      "\n",
      "The fundamental data type for array programming in Python is Numpy's `array`."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import random\n",
      "from math import sqrt\n",
      "\n",
      "x_position = [random.random() for _ in range(100000)]\n",
      "y_position = [random.random() for _ in range(100000)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 8
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%time\n",
      "distance = [sqrt(x**2 + y**2) for x, y in zip(x_position, y_position)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "CPU times: user 42.7 ms, sys: 3.89 ms, total: 46.6 ms\n",
        "Wall time: 45.8 ms\n"
       ]
      }
     ],
     "prompt_number": 9
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import numpy as np  # Conventional.\n",
      "\n",
      "x_array = np.array(x_position)\n",
      "y_array = np.array(y_position)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 10
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%time\n",
      "distance = np.sqrt(x_array**2 + y_array**2)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "CPU times: user 3.8 ms, sys: 1.05 ms, total: 4.86 ms\n",
        "Wall time: 4.24 ms\n"
       ]
      }
     ],
     "prompt_number": 32
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Array programming has two advantages:\n",
      "\n",
      " 1. It allows us to specify algorithms succintly in a form that more closely corresponds to the mathematical notation.\n",
      "    In other words, it meets our preference for *high-level* programming.\n",
      " 2. It allows for under-the-hood optimization to speed up computations.\n",
      "\n",
      "I'll now remind you of the rule of thumb: *don't program something that someone else has already coded.*\n",
      "The distance function is an example of this, it's called the *vector norm* in linear algebra and already exists in NumPy.\n",
      "First we have to combine our two 1D arrays into a 2D array."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "position = np.transpose([x_array, y_array])\n",
      "position.shape"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 27,
       "text": [
        "(100000, 2)"
       ]
      }
     ],
     "prompt_number": 27
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%time\n",
      "distance = np.linalg.norm(position, axis=1)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "CPU times: user 1.57 ms, sys: 241 \u00b5s, total: 1.82 ms\n",
        "Wall time: 1.95 ms\n"
       ]
      }
     ],
     "prompt_number": 35
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "distance.shape"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 97,
       "text": [
        "(100000,)"
       ]
      }
     ],
     "prompt_number": 97
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In any case, the defining feature of array programming is operating on entire arrays at a time, without using loops or something like `map`."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x = x_array[:5]\n",
      "print(x)\n",
      "print(10 * x)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[ 0.85295194  0.80037857  0.14238221  0.25253324  0.37767315]\n",
        "[ 8.52951942  8.00378573  1.4238221   2.52533241  3.77673152]\n"
       ]
      }
     ],
     "prompt_number": 100
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## `ndarray`\n",
      "\n",
      "The `array` (an alias for `ndarray`) is a homogeneous, multidimensional data type.\n",
      "It consists of a table of elements, all of the same type, indexed by a tuple of positive integers.\n",
      "The dimensions are sometimes referred to as *axes*, and the number axes is the array's *rank*.\n",
      "\n",
      "`numpy` is a C extension library, which is to say it is very efficient.\n",
      "The entire scientific Python ecosystem is built on top of the `array` so even if you aren't using arrays directly, you're probably using them indirectly."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.array([1, 2, 3, 4])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 37,
       "text": [
        "array([1, 2, 3, 4])"
       ]
      }
     ],
     "prompt_number": 37
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Note that we pass the constructor a list of elements rather than the elements directly.\n",
      "\n",
      "You should think of multi-dimensional arrays as (and indeed, create them with) lists of lists."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "square = np.array([[1, 2], [3, 4]])\n",
      "print(square)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[[1 2]\n",
        " [3 4]]\n"
       ]
      }
     ],
     "prompt_number": 41
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The most important attributes of an `array` are:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "attributes = (\n",
      "    'ndim',\n",
      "    'shape',\n",
      "    'size',\n",
      "    'dtype',\n",
      ")\n",
      "\n",
      "for attribute in attributes:\n",
      "    print('square.{} = {}'.format(\n",
      "        attribute, getattr(square, attribute)))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "square.ndim = 2\n",
        "square.shape = (2, 2)\n",
        "square.size = 4\n",
        "square.dtype = int64\n"
       ]
      }
     ],
     "prompt_number": 102
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "`dtype` is inferred automatically:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "square = np.array([[1, 2], [3, 4.5]])\n",
      "print(square)\n",
      "print('dtype:', square.dtype)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[[ 1.   2. ]\n",
        " [ 3.   4.5]]\n",
        "dtype: float64\n"
       ]
      }
     ],
     "prompt_number": 104
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "However, you can't change a `dtype` later, so if you are starting with integers but don't want to restrict yourself from having floats later, either set the `dtype` explicitly, or use a decimal point:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "square = np.array([[1., 2.], [3., 4.]])\n",
      "print(square)\n",
      "print('dtype:', square.dtype)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[[ 1.  2.]\n",
        " [ 3.  4.]]\n",
        "dtype: float64\n"
       ]
      }
     ],
     "prompt_number": 105
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We can index into arrays using standard Python slice notation:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "rectangle = np.arange(15).reshape((3, 5))\n",
      "print(rectangle)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[[ 0  1  2  3  4]\n",
        " [ 5  6  7  8  9]\n",
        " [10 11 12 13 14]]\n"
       ]
      }
     ],
     "prompt_number": 48
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "rectangle[0]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 49,
       "text": [
        "array([0, 1, 2, 3, 4])"
       ]
      }
     ],
     "prompt_number": 49
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "rectangle[0, :]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 50,
       "text": [
        "array([0, 1, 2, 3, 4])"
       ]
      }
     ],
     "prompt_number": 50
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "rectangle[:, :]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 51,
       "text": [
        "array([[ 0,  1,  2,  3,  4],\n",
        "       [ 5,  6,  7,  8,  9],\n",
        "       [10, 11, 12, 13, 14]])"
       ]
      }
     ],
     "prompt_number": 51
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "rectangle[:, 0]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 52,
       "text": [
        "array([ 0,  5, 10])"
       ]
      }
     ],
     "prompt_number": 52
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "rectangle[1:, :-1]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 123,
       "text": [
        "array([[ 5,  6,  7,  8],\n",
        "       [10, 11, 12, 13]])"
       ]
      }
     ],
     "prompt_number": 123
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "rectangle[:, ::2]  # Every other column (implicit start and stop)."
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 126,
       "text": [
        "array([[ 0,  2,  4],\n",
        "       [ 5,  7,  9],\n",
        "       [10, 12, 14]])"
       ]
      }
     ],
     "prompt_number": 126
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "rectangle[(0, 2), :]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 61,
       "text": [
        "array([[ 0,  1,  2,  3,  4],\n",
        "       [10, 11, 12, 13, 14]])"
       ]
      }
     ],
     "prompt_number": 61
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "**But Henry, isn't it not very Pythonic to use integers?**\n",
      "\n",
      "Yes!\n",
      "Down the line, we'll use `pandas` to create indices with semantic meaning (e.g. timestamps).\n",
      "Patience..."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This should be familiar if you have worked with Matlab or R.\n",
      "There is one key difference, though (from Matlab at least--not sure about R):\n",
      "in Matlab there isn't really any such thing as 1D arrays.\n",
      "All arrays have at least two dimensions in Matlab, even if one of them may only be of length 1.\n",
      "In Numpy there is a difference between a 1D array and a 2D array with one axis having length 1."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "vector = np.array([1, 2, 3])\n",
      "print(vector)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[1 2 3]\n"
       ]
      }
     ],
     "prompt_number": 80
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "vector.shape"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 69,
       "text": [
        "(3,)"
       ]
      }
     ],
     "prompt_number": 69
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "vector.ndim"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 72,
       "text": [
        "1"
       ]
      }
     ],
     "prompt_number": 72
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print(vector.transpose())"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[1 2 3]\n"
       ]
      }
     ],
     "prompt_number": 81
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "vector == vector.transpose()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 83,
       "text": [
        "array([ True,  True,  True], dtype=bool)"
       ]
      }
     ],
     "prompt_number": 83
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.all(vector == vector.transpose())"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 86,
       "text": [
        "True"
       ]
      }
     ],
     "prompt_number": 86
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "(vector == vector.transpose()).all()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 85,
       "text": [
        "True"
       ]
      }
     ],
     "prompt_number": 85
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "vector = np.array([[1, 2, 3]])\n",
      "print(vector)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[[1 2 3]]\n"
       ]
      }
     ],
     "prompt_number": 87
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "vector.shape"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 88,
       "text": [
        "(1, 3)"
       ]
      }
     ],
     "prompt_number": 88
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "vector.ndim"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 89,
       "text": [
        "2"
       ]
      }
     ],
     "prompt_number": 89
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "vector.transpose()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 90,
       "text": [
        "array([[1],\n",
        "       [2],\n",
        "       [3]])"
       ]
      }
     ],
     "prompt_number": 90
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.all(vector == vector.transpose())"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 92,
       "text": [
        "False"
       ]
      }
     ],
     "prompt_number": 92
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "`len` always returns the size of the first dimension; just like you would expect with a list of lists."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "len(rectangle)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 94,
       "text": [
        "3"
       ]
      }
     ],
     "prompt_number": 94
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "len([[1, 2, 3], [4, 5, 6], [7, 8, 9]])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 96,
       "text": [
        "3"
       ]
      }
     ],
     "prompt_number": 96
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Creating `array`s"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.arange(10)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 106,
       "text": [
        "array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])"
       ]
      }
     ],
     "prompt_number": 106
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.arange(3., 10, 2)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 110,
       "text": [
        "array([ 3.,  5.,  7.,  9.])"
       ]
      }
     ],
     "prompt_number": 110
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.arange('2014-06-01', '2014-06-17', dtype='datetime64')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 112,
       "text": [
        "array(['2014-06-01', '2014-06-02', '2014-06-03', '2014-06-04',\n",
        "       '2014-06-05', '2014-06-06', '2014-06-07', '2014-06-08',\n",
        "       '2014-06-09', '2014-06-10', '2014-06-11', '2014-06-12',\n",
        "       '2014-06-13', '2014-06-14', '2014-06-15', '2014-06-16'], dtype='datetime64[D]')"
       ]
      }
     ],
     "prompt_number": 112
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.ones(10)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 113,
       "text": [
        "array([ 1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.])"
       ]
      }
     ],
     "prompt_number": 113
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.ones((10, 2))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 114,
       "text": [
        "array([[ 1.,  1.],\n",
        "       [ 1.,  1.],\n",
        "       [ 1.,  1.],\n",
        "       [ 1.,  1.],\n",
        "       [ 1.,  1.],\n",
        "       [ 1.,  1.],\n",
        "       [ 1.,  1.],\n",
        "       [ 1.,  1.],\n",
        "       [ 1.,  1.],\n",
        "       [ 1.,  1.]])"
       ]
      }
     ],
     "prompt_number": 114
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.ones((3, 3, 3))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 115,
       "text": [
        "array([[[ 1.,  1.,  1.],\n",
        "        [ 1.,  1.,  1.],\n",
        "        [ 1.,  1.,  1.]],\n",
        "\n",
        "       [[ 1.,  1.,  1.],\n",
        "        [ 1.,  1.,  1.],\n",
        "        [ 1.,  1.,  1.]],\n",
        "\n",
        "       [[ 1.,  1.,  1.],\n",
        "        [ 1.,  1.,  1.],\n",
        "        [ 1.,  1.,  1.]]])"
       ]
      }
     ],
     "prompt_number": 115
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.zeros((2, 7))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 119,
       "text": [
        "array([[ 0.,  0.,  0.,  0.,  0.,  0.,  0.],\n",
        "       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.]])"
       ]
      }
     ],
     "prompt_number": 119
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.empty((3, 3))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 120,
       "text": [
        "array([[  6.95020652e-310,   6.95020652e-310,   4.24356710e-246],\n",
        "       [  6.95020669e-310,   6.95020134e-310,  -2.74518236e+008],\n",
        "       [  6.95020539e-310,   5.11289170e-317,   3.95252517e-322]])"
       ]
      }
     ],
     "prompt_number": 120
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.linspace(0, 10, 5)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 118,
       "text": [
        "array([  0. ,   2.5,   5. ,   7.5,  10. ])"
       ]
      }
     ],
     "prompt_number": 118
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.eye(3)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 121,
       "text": [
        "array([[ 1.,  0.,  0.],\n",
        "       [ 0.,  1.,  0.],\n",
        "       [ 0.,  0.,  1.]])"
       ]
      }
     ],
     "prompt_number": 121
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.diag([1, 2, 3, 4])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 122,
       "text": [
        "array([[1, 0, 0, 0],\n",
        "       [0, 2, 0, 0],\n",
        "       [0, 0, 3, 0],\n",
        "       [0, 0, 0, 4]])"
       ]
      }
     ],
     "prompt_number": 122
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## `array` views\n",
      "\n",
      "When we index an `array`, is this expensive?\n",
      "Are we copying values?\n",
      "\n",
      "Whenever possible, `numpy` gives us *views* into the array."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "rectangle"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 128,
       "text": [
        "array([[ 0,  1,  2,  3,  4],\n",
        "       [ 5,  6,  7,  8,  9],\n",
        "       [10, 11, 12, 13, 14]])"
       ]
      }
     ],
     "prompt_number": 128
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "square = rectangle[:, :3]\n",
      "print(square)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[[ 0  1  2]\n",
        " [ 5  6  7]\n",
        " [10 11 12]]\n"
       ]
      }
     ],
     "prompt_number": 130
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "square[0, 0] = 100\n",
      "print(square)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[[100   1   2]\n",
        " [  5   6   7]\n",
        " [ 10  11  12]]\n"
       ]
      }
     ],
     "prompt_number": 132
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print(rectangle)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[[100   1   2   3   4]\n",
        " [  5   6   7   8   9]\n",
        " [ 10  11  12  13  14]]\n"
       ]
      }
     ],
     "prompt_number": 133
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You can double-check if you have a view by checking the `ndarray.flags.owndata` attribute:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "square.flags.owndata"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 134,
       "text": [
        "False"
       ]
      }
     ],
     "prompt_number": 134
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": []
    }
   ],
   "metadata": {}
  }
 ]
}<commit_msg>cleanup array notebook<commit_after>{
 "metadata": {
  "name": "",
  "signature": "sha256:81eb6550b1bc247e4ab16755c8733692d24769217f7ac4b397dd61f015fa2715"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The Scientific Python Ecosystem\n",
      "===\n",
      "\n",
      " - [IPython](http://ipython.org/):\n",
      "   a must-have whether or not you're doing scientific programming.\n",
      " - [NumPy](http://numpy.scipy.org/):\n",
      "  provides the numerical array, the \"standard interface\" of scientific Python.\n",
      " - [SciPy](http://docs.scipy.org/doc/reference/):\n",
      "   a collection of numerical algorithms and domain-specific modules, from statistics to signal processing to ODE integration.\n",
      " - [matplotlib](http://matplotlib.org/):\n",
      "   plotting functions that mimic Matlab's interface and integrate well with IPython.\n",
      " - [pandas](http://pandas.pydata.org/):\n",
      "   provides the `DataFrame`, the emerging second standard interface of scientific Python.\n",
      " - [SymPy](http://sympy.org/):\n",
      "   symbolic Python.\n",
      " - [Statsmodels](http://statsmodels.sourceforge.net/):\n",
      "   because some statistics functions and models are missing from SciPy.\n",
      " - [scikit-learn](http://scikit-learn.org/stable/):\n",
      "   a popular machine-learning library.\n",
      " - [rpy2](http://rpy.sourceforge.net/):\n",
      "   call R from Python, if your test or model is not in SciPy, statsmodels, or scikit-learn.\n",
      " - [nose](https://nose.readthedocs.org/en/latest/):\n",
      "   an easy-to-use Python testing framework.\n",
      "   \n",
      "When you ran `conda install -n python3 --file conda-packages.txt` back on the first week,\n",
      "all these packages were installed in your `python3` environment,\n",
      "with two exceptions:\n",
      "\n",
      " - I forgot `nose`.\n",
      "   It's on `conda` so `conda install -n python3 nose` will get it.\n",
      " - `rpy2` is not on `conda` for Python 3.\n",
      "   If `pip` can't install it, there are Windows binaries [here](http://www.lfd.uci.edu/~gohlke/pythonlibs/#rpy2) although it might be tricky to get working (need to make sure you're using the same R version referenced on that page).\n",
      "   On OSX it should compile although there may be some dependencies you need to install.\n",
      "   Check the documentation and search the [issues page](https://bitbucket.org/lgautier/rpy2/issues?status=new&status=open) if you have trouble.\n",
      "   There are probably precompiled OSX binaries available but I was not able to find any with a quick search.\n",
      "   (really everyone should use Linux and then this would all just work)\n",
      "   \n",
      "   \n",
      "###Credit where credit is due\n",
      "\n",
      "Again, I am stealing someone else's tutorial, this time thanks are due to <NAME> V<NAME> Continuum, at the PyData conference (2013 I think).\n",
      "I will put the video below.\n",
      "Slides are available [here](http://www.slideshare.net/PyData/introduction-to-numpy)."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from IPython.display import VimeoVideo\n",
      "\n",
      "VimeoVideo(63253083)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Array programming\n",
      "\n",
      "Array programming is a computing technique that generalizes scalar operations to apply transparently to vectors, matrices, and high-dimensional arrays.\n",
      "\n",
      "The fundamental data type for array programming in Python is Numpy's `array`."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import random\n",
      "from math import sqrt\n",
      "\n",
      "x_position = [random.random() for _ in range(100000)]\n",
      "y_position = [random.random() for _ in range(100000)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%time\n",
      "distance = [sqrt(x**2 + y**2) for x, y in zip(x_position, y_position)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import numpy as np  # Conventional.\n",
      "\n",
      "x_array = np.array(x_position)\n",
      "y_array = np.array(y_position)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%time\n",
      "distance = np.sqrt(x_array**2 + y_array**2)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Array programming has two advantages:\n",
      "\n",
      " 1. It allows us to specify algorithms succintly in a form that more closely corresponds to the mathematical notation.\n",
      "    In other words, it meets our preference for *high-level* programming.\n",
      " 2. It allows for under-the-hood optimization to speed up computations.\n",
      "\n",
      "I'll now remind you of the rule of thumb: *don't program something that someone else has already coded.*\n",
      "The distance function is an example of this, it's called the *vector norm* in linear algebra and already exists in NumPy.\n",
      "First we have to combine our two 1D arrays into a 2D array."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "position = np.transpose([x_array, y_array])\n",
      "position.shape"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%time\n",
      "distance = np.linalg.norm(position, axis=1)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "distance.shape"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In any case, the defining feature of array programming is operating on entire arrays at a time, without using loops or something like `map`."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x = x_array[:5]\n",
      "print(x)\n",
      "print(10 * x)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## `ndarray`\n",
      "\n",
      "The `array` (an alias for `ndarray`) is a homogeneous, multidimensional data type.\n",
      "It consists of a table of elements, all of the same type, indexed by a tuple of positive integers.\n",
      "The dimensions are sometimes referred to as *axes*, and the number axes is the array's *rank*.\n",
      "\n",
      "`numpy` is a C extension library, which is to say it is very efficient.\n",
      "The entire scientific Python ecosystem is built on top of the `array` so even if you aren't using arrays directly, you're probably using them indirectly."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.array([1, 2, 3, 4])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Note that we pass the constructor a list of elements rather than the elements directly.\n",
      "\n",
      "You should think of multi-dimensional arrays as (and indeed, create them with) lists of lists."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "square = np.array([[1, 2], [3, 4]])\n",
      "print(square)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The most important attributes of an `array` are:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "attributes = (\n",
      "    'ndim',\n",
      "    'shape',\n",
      "    'size',\n",
      "    'dtype',\n",
      ")\n",
      "\n",
      "for attribute in attributes:\n",
      "    print('square.{} = {}'.format(\n",
      "        attribute, getattr(square, attribute)))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "`dtype` is inferred automatically:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "square = np.array([[1, 2], [3, 4.5]])\n",
      "print(square)\n",
      "print('dtype:', square.dtype)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "However, you can't change a `dtype` later, so if you are starting with integers but don't want to restrict yourself from having floats later, either set the `dtype` explicitly, or use a decimal point:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "square = np.array([[1., 2.], [3., 4.]])\n",
      "print(square)\n",
      "print('dtype:', square.dtype)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We can index into arrays using standard Python slice notation:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "rectangle = np.arange(15).reshape((3, 5))\n",
      "print(rectangle)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "rectangle[0]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "rectangle[0, :]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "rectangle[:, :]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "rectangle[:, 0]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "rectangle[1:, :-1]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "rectangle[:, ::2]  # Every other column (implicit start and stop)."
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "rectangle[(0, 2), :]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "**But Henry, isn't it not very Pythonic to use integers?**\n",
      "\n",
      "Yes!\n",
      "Down the line, we'll use `pandas` to create indices with semantic meaning (e.g. timestamps).\n",
      "Patience..."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This should be familiar if you have worked with Matlab or R.\n",
      "There is one key difference, though (from Matlab at least--not sure about R):\n",
      "in Matlab there isn't really any such thing as 1D arrays.\n",
      "All arrays have at least two dimensions in Matlab, even if one of them may only be of length 1.\n",
      "In Numpy there is a difference between a 1D array and a 2D array with one axis having length 1."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "vector = np.array([1, 2, 3])\n",
      "print(vector)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "vector.shape"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "vector.ndim"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print(vector.transpose())"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "vector == vector.transpose()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.all(vector == vector.transpose())"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "(vector == vector.transpose()).all()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "vector = np.array([[1, 2, 3]])\n",
      "print(vector)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "vector.shape"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "vector.ndim"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "vector.transpose()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.all(vector == vector.transpose())"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "`len` always returns the size of the first dimension; just like you would expect with a list of lists."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "len(rectangle)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "len([[1, 2, 3], [4, 5, 6], [7, 8, 9]])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Creating `array`s"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.arange(10)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.arange(3., 10, 2)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.arange('2014-06-01', '2014-06-17', dtype='datetime64')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.ones(10)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.ones((10, 2))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.ones((3, 3, 3))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.zeros((2, 7))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.empty((3, 3))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.linspace(0, 10, 5)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.eye(3)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "np.diag([1, 2, 3, 4])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## `array` views\n",
      "\n",
      "When we index an `array`, is this expensive?\n",
      "Are we copying values?\n",
      "\n",
      "Whenever possible, `numpy` gives us *views* into the array."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "rectangle"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "square = rectangle[:, :3]\n",
      "print(square)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "square[0, 0] = 100\n",
      "print(square)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print(rectangle)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You can double-check if you have a view by checking the `ndarray.flags.owndata` attribute:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "square.flags.owndata"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Views are nice because minimizing copying speeds up our algorithms.\n",
      "This means there is very little incremental cost to creating new names for different selections into an array.\n",
      "\n",
      "In some languages, the best way to speed up numerical code is to minimize assignment operations.\n",
      "This leads to confusing code.\n",
      "With NumPy, in Python we can create whatever sub-arrays we need to make our arrays readable."
     ]
    }
   ],
   "metadata": {}
  }
 ]
}
{'fraction_non_alphanumeric': 0.19468973747016707, 'fraction_numerical': 0.02611376292760541, 'mean_word_length': 1.7566337719298246, 'pattern_counts': {'":': 983, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 5, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4719972', 'n_tokens_mistral': 18123, 'n_tokens_neox': 17002, 'n_words': 3926}