<commit_before>{
 "metadata": {
  "name": "",
  "signature": "sha256:092e02aa583f0ba7c6d6e7c7004f9d952dcf770144fc244350f4d52185cfb9d4"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# SOM Retinotopy\n",
      "\n",
      "This IPython notebook defines and explores the Kohonen SOM (self-organizing map) model of retinotopy described in pages 53-59 of:\n",
      "<blockquote>\n",
      "  Miikkulainen, Bednar, Choe, and Sirosh (2005),\n",
      "  <a href=\"http://computationalmaps.org\">Computational Maps in the Visual Cortex</a>, Springer.\n",
      "</blockquote>\n",
      "\n",
      "If you can double-click on this text and edit it, you are in a live [IPython Notebook](http://ipython.org/notebook) environment where you can run the code and explore the model. Otherwise, you are viewing a static (e.g. HTML) copy of the notebook, which allows you to see the precomputed results only.  To switch to the live notebook, see the [notebook installation instructions](https://github.com/ioam/topographica).\n",
      "\n",
      "This IPython notebook constructs the definition of the SOM retinotopy model in the [Topographica](http://www.topographica.org) simulator, and shows how it organizes.\n",
      "\n",
      "A static version of this notebook may be viewed [online](http://ioam.github.io/media/som_retinotopy.html). To run the live notebook and explore the model interactively, you will need both IPython Notebook and Topographica.\n",
      "\n",
      "If you prefer the older Tk GUI interface or a command line, you may use the som_retinotopy.ty script distributed with Topographica, without IPython Notebook, as follows:\n",
      "\n",
      "```bash\n",
      "./topographica -g examples/som_retinotopy.ty\n",
      "```\n",
      "\n",
      "To run this Notebook version, you can run all cells in order automatically:\n",
      "* Selecting ``Kernel -> Restart`` from the menu above.\n",
      "* Selecting ``Cell -> Run All``. \n",
      "\n",
      "Alternatively, you may run each cell in sequence, starting at the top of the notebook and pressing ``Shift + Enter``."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%reload_ext topo.misc.ipython\n",
      "%opts AxisLayout [tick_format=\"%.1f\" size=70]\n",
      "%timer start"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Model definition\n",
      "The next three cells define the SOM model in its entirety (copied from examples/som_retinotopy.ty).  First, we import required libraries and we declare various parameters to allow the modeller to control the behavior:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "\"\"\"\n",
      "Basic example of a fully connected SOM retinotopic map with ConnectionFields.\n",
      "\n",
      "Contains a Retina (2D Gaussian generator) fully connected to a V1\n",
      "(SOM) sheet, with no initial ordering for topography.\n",
      "\n",
      "Constructed to match the retinotopic simulation from page 53-59 of\n",
      "Miikkulainen, Bednar, Choe, and Sirosh (2005), Computational Maps in\n",
      "the Visual Cortex, Springer.  Known differences include:\n",
      "\n",
      " - The cortex_density and retina_density are smaller for\n",
      "   speed (compared to 40 and 24 in the book).\n",
      " - The original simulation used a radius_0 of 13.3/40, which does work\n",
      "   for some random seeds, but a much larger radius is used here so that\n",
      "   it converges more reliably.\n",
      "\"\"\"\n",
      "\n",
      "import topo\n",
      "import imagen\n",
      "\n",
      "from math import exp, sqrt\n",
      "\n",
      "import param\n",
      "\n",
      "from topo import learningfn,numbergen,transferfn,pattern,projection,responsefn,sheet\n",
      "\n",
      "import topo.learningfn.projfn\n",
      "import topo.pattern.random\n",
      "import topo.responsefn.optimized\n",
      "import topo.transferfn.misc\n",
      "\n",
      "# Disable the measurement progress bar for this notebook\n",
      "from topo.analysis.featureresponses import pattern_response\n",
      "pattern_response.progress_bar = False\n",
      "\n",
      "# Parameters that can be passed on the command line using -p\n",
      "from topo.misc.commandline import global_params as p\n",
      "p.add(\n",
      "\n",
      "    retina_density=param.Number(default=10.0,bounds=(0,None),\n",
      "        inclusive_bounds=(False,True),doc=\"\"\"\n",
      "        The nominal_density to use for the retina.\"\"\"),\n",
      "\n",
      "    cortex_density=param.Number(default=10.0,bounds=(0,None),\n",
      "        inclusive_bounds=(False,True),doc=\"\"\"\n",
      "        The nominal_density to use for V1.\"\"\"),\n",
      "\n",
      "    input_seed=param.Number(default=0,bounds=(0,None),doc=\"\"\"\n",
      "        Seed for the pseudorandom number generator controlling the\n",
      "        input patterns.\"\"\"),\n",
      "\n",
      "    weight_seed=param.Number(default=0,bounds=(0,None),doc=\"\"\"\n",
      "        Seed for the pseudorandom number generator controlling the\n",
      "        initial weight values.\"\"\"),\n",
      "\n",
      "    radius_0=param.Number(default=1.0,bounds=(0,None),doc=\"\"\"\n",
      "        Starting radius for the neighborhood function.\"\"\"),\n",
      "\n",
      "    alpha_0=param.Number(default=0.42,bounds=(\n",
      "\n",
      "\n",
      "0,None),doc=\"\"\"\n",
      "        Starting value for the learning rate.\"\"\"))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "#### Customizing the model parameters\n",
      "\n",
      "The cell below may be used to modify any of the parameters defined above, allowing you to explore the parameter space of the model. To illustrate, the default retina and cortex densities are set to 10 (their default values), but you may change the value of any parameter in this way:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "p.retina_density=10\n",
      "p.cortex_density=10"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The <a href=\"http://ioam.github.io/media/som_retinotopy.html\">online version of this notebook</a> normally uses the full retinal and cortical densities of 24 and 40, respectively, but densities of 10 are the default because they should be sufficient for most purposes.  Using the parameters and definitions above, the following code defines the SOM model of retinotopy, by defining the input patterns, the Sheets, and the connections between Sheets:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Using time dependent random streams.\n",
      "# Corresponding numbergen objects should be given suitable names.\n",
      "param.Dynamic.time_dependent=True\n",
      "numbergen.RandomDistribution.time_dependent = True\n",
      "\n",
      "# input pattern\n",
      "sheet.GeneratorSheet.period = 1.0\n",
      "sheet.GeneratorSheet.phase = 0.05\n",
      "sheet.GeneratorSheet.nominal_density = p.retina_density\n",
      "\n",
      "input_pattern = pattern.Gaussian(scale=1.0,size=2*sqrt(2.0*0.1*24.0)/24.0,\n",
      "    aspect_ratio=1.0,orientation=0,\n",
      "    x=numbergen.UniformRandom(name='xgen', lbound=-0.5,ubound=0.5, seed=p.input_seed+12),\n",
      "    y=numbergen.UniformRandom(name='ygen', lbound=-0.5,ubound=0.5, seed=p.input_seed+56))\n",
      "\n",
      "\n",
      "# Local variable to allow weights to be controlled easily\n",
      "param.random_seed = p.weight_seed + 86\n",
      "\n",
      "topo.sim['Retina'] = sheet.GeneratorSheet(input_generator=input_pattern)\n",
      "\n",
      "topo.sim['V1'] = sheet.CFSheet(\n",
      "    nominal_density = p.cortex_density,\n",
      "    # Original CMVC simulation used an initial radius of 13.3/40.0 (~0.33)\n",
      "    output_fns=[transferfn.misc.KernelMax(density=p.cortex_density,\n",
      "        kernel_radius=numbergen.BoundedNumber(\n",
      "            bounds=(0.5/40,None),\n",
      "            generator=numbergen.ExponentialDecay(\n",
      "                starting_value=p.radius_0,\n",
      "                time_constant=40000/5.0)))])\n",
      "\n",
      "topo.sim.connect('Retina','V1',name='Afferent',delay=0.05,\n",
      "    connection_type=projection.CFProjection,\n",
      "    weights_generator = pattern.random.UniformRandom(),\n",
      "    nominal_bounds_template=sheet.BoundingBox(radius=1.0), # fully connected network.\n",
      "    learning_rate=numbergen.ExponentialDecay(\n",
      "        starting_value = p.alpha_0,\n",
      "        time_constant=40000/6.0),\n",
      "    response_fn = responsefn.optimized.CFPRF_EuclideanDistance_opt(),\n",
      "    learning_fn = learningfn.projfn.CFPLF_EuclideanHebbian())\n",
      "\n",
      "'Loaded the self-organizing map model (som_retinotopy)'"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Exploring the model"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import numpy as np\n",
      "from holoviews import Dimension, Matrix\n",
      "from holoviews.operation import toRGB"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now the model has been defined, we can explore it.  The structure of the loaded model is shown in this screenshot taken from the Tk GUI's Model Editor:\n",
      "\n",
      "### **Model structure**\n",
      "\n",
      "<center>\n",
      "<img src='http://topographica.org/_images/som_network_diagram.png'/>\n",
      "</center>\n",
      "\n",
      "\n",
      "The large circle indicates that the Retina Sheet is fully connected to the V1 Sheet.\n",
      "\n",
      "### **Initial weights**\n",
      "\n",
      "The plot below shows the initial set of weights from a 10x10 subset of the V1 neurons (i.e., every neuron with the reduced cortex_density, or every fourth neuron for cortex_density=40):"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "topo.sim.V1.Afferent.grid()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "As we can see, the initial weights are uniform random. Each neuron receives a full set of connections from all input units, and thus each has a 24x24 or 10x10 array of weights (depending on the retina_density).\n",
      "\n",
      "### **Initial Center-of-Gravity (CoG) plots**"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We can visualize the center of gravity (CoG) of the V1 Afferent weights using the following measurement command:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from topo.command.analysis import measure_cog\n",
      "cog_data = measure_cog()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The center of gravity (a.k.a. centroid or center of mass) is computed for each neuron using its set of incoming weights. The plot below shows each neuron's CoG represented by a point, with a line segment drawn from each neuron to each of its four immediate neighbors so that neighborhood relationships (if any) will be visible."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "cog_data.CoG.Afferent"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "From this plot is is clear that all of the neurons have a CoG near the center of the retina, which is to be expected because the weights are fully connected and evenly distributed (and thus all have an average (X,Y) value near the center of the retina). The same data is also visualized below:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "xcog=cog_data.XCoG.Afferent.last\n",
      "ycog=cog_data.YCoG.Afferent.last\n",
      "xcog + ycog + xcog*ycog"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The V1 X CoG plot shows the X location preferred by each neuron in V1, and the V1 Y CoG plot shows the preferred Y locations. The monochrome values are scaled so that a preference of -0.5 is black and a preference of +0.5 is white, and since they all start out around 0.0 they are all shades of medium gray.  The actual preferences can be accessed directly as a Numpy array if you prefer:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "xcog.data.min(),xcog.data.max(),ycog.data.min(),ycog.data.max()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The colorful XYCoG plot above right shows a false-color visualization of the CoG values in both dimensions at once, where the amount of red in the plot is proportional to the X CoG, and the amount of green in the plot is proportional to the Y CoG. Where both X and Y are low, the plot is black or very dark, and where both are high the plot is yellow (because red and green light together appears yellow). All of the units should be an intermediate color of yellow/gray at this stage, indicating preference for 0.5 in both X and Y. As you will see later, once neurons have developed strong preferences for one location over the others, some neurons will show up as bright black (X and Y both low), red (X high, Y low), red (X low, Y high), or yellow (X and Y both high) in this plot."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Training stimuli\n",
      "\n",
      "We can have a look at what the training patterns that will be used to train the model without having to run it. In the cell labelled ``In [4]`` (in the model definition), we see where the training patterns are defined in a variable called ``input_pattern``. We see that the circular Gaussian stimulus has ``x`` and ``y`` values that are drawn from a random distribution by two independent ``numbergen.UniformRandom`` objects. We can now view what 100 frames of training patterns will look like:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "input_pattern.anim(30, offset=0.05)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Collecting data for animated plots\n",
      "\n",
      "At the end of the notebook, we will generate a set of nice animations showing the plots we have already shown evolve over development. We now create a ``Collector`` object that collects all information needed for plotting and animation. We will collect the information we have just examined and advance the simulation one iteration:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from topo.analysis import Collector\n",
      "from topo.command.analysis import measure_cog\n",
      "\n",
      "c = Collector()\n",
      "c.collect(measure_cog)\n",
      "\n",
      "c.Activity.Retina     = c.collect(topo.sim.Retina)\n",
      "c.Activity.V1         = c.collect(topo.sim.V1)\n",
      "c.Activity.V1Afferent = c.collect(topo.sim.V1.Afferent)\n",
      "c.CFs.Afferent        = c.collect(topo.sim.V1.Afferent, grid=True, rows=10, cols=10)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## **Activity after a single training iteration**\n",
      "\n",
      "The initial activities would be blank, but after running the model a single iteration, the sheet activities now look as follows:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data = c(times=[1])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.Activity.Retina + data.Activity.V1"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In the Retina plot, each photoreceptor is represented as a pixel whose shade of grey codes the response level, increasing from black to white. As expected, this particular example matches the first frame we visualized in the training stimulus animation. The V1 plot shows the response to that input, which for a SOM is initially a large Gaussian-shaped blob centered around the maximally responding unit."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## **Projection Activity**\n",
      "\n",
      "To see what the responses were before SOM\u2019s neighborhood function forced them into a Gaussian shape, you can look at the Projection Activity plot, which shows the feedforward activity in V1:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.Activity.V1Afferent"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Here these responses are best thought of as Euclidean proximity, not distance. This formulation of the SOM response function actually subtracts the distances from the max distance, to ensure that the response will be larger for smaller Euclidean distances (as one intuitively expects for a neural response). The V1 feedforward activity appears random here because the Euclidean distance from the input vector to the initial random weight vector is random.\n",
      "\n",
      "\n",
      "## **Learning after a few iterations**\n",
      "\n",
      "If you look at the weights now we have run a single training iteration, you'll see that most of the neurons have learned new weight patterns based on this input:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.CFs.Afferent"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Some of the weights to each neuron have now changed due to learning. In the SOM algorithm, the unit with the maximum response (i.e., the minimum Euclidean distance between its weight vector and the input pattern) is chosen, and the weights of units within a circular area defined by a Gaussian-shaped neighborhood function around this neuron are updated.\n",
      "\n",
      "\n",
      "This effect is visible in this plot \u2013 a few neurons around the winning unit at the top right have changed their weights. Let us run a few more iterations before having another look:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "topo.sim.run(4)\n",
      "topo.sim.V1.Afferent.grid()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The weights have been updated again - it is clear after these five iterations that the input patterns are becoming represented in the weight patterns, though not very cleanly yet. We also see that the projection activity patterns are becoming smoother, since the weight vectors are now similar between neighboring neurons:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "topo.sim.V1.Afferent.projection_view()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We will now advance to simulation time 250 because we will want to make animations regularly sampled every 250 steps:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "topo.sim.run(245)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## At 5000 iterations\n",
      "\n",
      "Let us use our ``Collector`` object ``c`` to collect more measurements. We will start collecting measurements every 250 steps until we complete 5000 iterations, which should take a few seconds at the default densities, and may be a minute or two at the higher densities:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "times = [topo.sim.time()*i for i in range(1,21)]\n",
      "print(\"Running %d measurements between iteration %s and iteration %s\" % \n",
      "      (len(times), min(times), max(times)))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data = c(data, times=times)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We see that the topographic grid plot evolves over this period as follows:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.CoG.Afferent"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The X and Y CoG plots are now smooth, but not yet the axis-aligned gradients (e.g. left to right and bottom to top) that an optimal topographic mapping would have:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%opts Matrix {+groupwise}\n",
      "(data.XCoG.Afferent + data.YCoG.Afferent + \n",
      " data.XCoG.Afferent * data.YCoG.Afferent).select(Time=5000)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The weight patterns are still quite broad, not very selective for typical input patterns, and not very distinct from one another:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.CFs.Afferent.last"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<center><h3 class='alert-success'>In the live notebook you can remove the ``.last`` to view an animation to 5000 iterations</h3></center>  "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## At 10000 iterations\n",
      "\n",
      "Additional training up to 10000 iterations (which becomes faster due to a smaller neighborhood radius) leads to a nearly flat, square map:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "times = [5000+(250*i) for i in range(1,21)]\n",
      "print(\"Running %d measurements between iteration %s and iteration %s\" % \n",
      "      (len(times), min(times), max(times)))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data = c(data, times=times)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.CoG.Afferent.last"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "<center><h4 class='alert-success'>Note: In the live notebook you can remove ``.last`` to view an animation to 10000 iterations</h4></center>  "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Yet the weight patterns are still quite broad and not very selective for typical input patterns, because the neighborhood radius (initially 1.0 in Sheet coordinates, i.e. larger than the entire V1) is still large enough to force most cells to respond to the same patterns:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "topo.sim.V1.output_fns[0].kernel_radius"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.CFs.Afferent.last"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## At 30000 iterations"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "By 30000 iterations the map has good coverage of the available portion of the input space:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "times = [10000+(500*i) for i in range(1,41)]\n",
      "print(\"Running %d measurements between iteration %s and iteration %s\\n\\n\" % \n",
      "      (len(times), min(times), max(times)))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data = c(data, times=times)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.CoG.Afferent.last"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The final projection plot at 30000 now shows that each neuron has developed weights concentrated in a small part of the input space, matching a prototypical input at one location, and that neurons each have distinct preferences:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "topo.sim.V1.output_fns[0].kernel_radius"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.CFs.Afferent.last"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The final topographic mapping may have any one of a number of possible arrangements, flipped or rotated by 90 degrees along any axis or both axes.  E.g. a vertically flipped map will have blobs in each CF that are at the bottom for the top row of neurons, but the top for the bottom row of neurons. Nothing in the network reliably drives it to have any particular overall orientation or mapping apart from aligning to the square shape, and each of these possible organizations is equivalent functionally."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Animations over 30000 iterations\n",
      "\n",
      "We can watch how the topographic mapping unfolds over all 30000 iterations we have run:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%opts Gravity_Contours [rescale_individually=False]\n",
      "data.CoG.Afferent"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "And how the XCoG and YCoG components unfold:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.XCoG.Afferent + data.YCoG.Afferent + data.XCoG.Afferent * data.YCoG.Afferent"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In the false color CoG plot, a perfectly organized mapping will have black in one corner and yellow in the opposite, with green and red in the two other corners, and smooth colors between each corner.   Next, the weights (though by default this animation is disabled to save time, so you would need to remove \".last\" and re-run this cell, which may take several minutes):"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.CFs.Afferent.last"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Snapshots of the retinal and V1 activity over development. Notice how the activity in V1 becomes more focused over time as the kernel radius decreases:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.Activity.Retina + data.Activity.V1"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Exploring the parameters of the SOM \n",
      "\n",
      "Now, you can re-run the basic simulation by restarting the IPython Notebook Kernel (<code>Kernel -> Restart</code>) or using the keyboard shortcut ``Ctrl+M .`` (see <code>Help -> Keyboard Shortcuts</code> for the full list). Then you can change one of the parameter values, either by editing the model definition above before running those cells, or (usually clearer) by setting the parameters in the cell labelled ``In [3]``.\n",
      "\n",
      "For instance, the starting value of the neighborhood radius (from which all future values are calculated according to exponential decay) is 1.0. You can change this value as you see fit, e.g. to 0.1, by setting ``p.radius_0=0.1``. With such a small learning radius, global ordering is unlikely to happen, and one can expect the topographic grid not to flatten out (despite local order in patches).\n",
      "\n",
      "Similarly, consider changing the initial learning rate from 0.42 to e.g. 1.0 (e.g. by setting ``p.alpha_0=1.0`` in cell 3). The retina and V1 densities cannot be changed after the simulation has started, but again, they can be changed by providing their values above and restarting the notebook.\n",
      "\n",
      "You can also try changing the input_seed (``p.input_seed=XX``), to get a different stream of inputs, or weight_seed (``p.weight_seed=XX``), to get a different set of initial weights. \n",
      "\n",
      "Note that with some of these values, you may encounter cases where the SOM fails to converge even though it seems to be working properly otherwise. For instance, some seed values result in topological defects like a kink:\n",
      "<br>\n",
      "<br>\n",
      "<center><img src=\"https://ioam.github.io/topographica/_images/som_grid_kink.png\"></img></center>"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%timer"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    }
   ],
   "metadata": {}
  }
 ]
}<commit_msg>Fixed input and weight seed handling<commit_after>{
 "metadata": {
  "name": "",
  "signature": "sha256:63b08d3d9b67960a911e9637f0c913e56801282e1f7ed3735cc125ff8757e7de"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# SOM Retinotopy\n",
      "\n",
      "This IPython notebook defines and explores the Kohonen SOM (self-organizing map) model of retinotopy described in pages 53-59 of:\n",
      "<blockquote>\n",
      "  Miikkulainen, Bednar, Choe, and Sirosh (2005),\n",
      "  <a href=\"http://computationalmaps.org\">Computational Maps in the Visual Cortex</a>, Springer.\n",
      "</blockquote>\n",
      "\n",
      "If you can double-click on this text and edit it, you are in a live [IPython Notebook](http://ipython.org/notebook) environment where you can run the code and explore the model. Otherwise, you are viewing a static (e.g. HTML) copy of the notebook, which allows you to see the precomputed results only.  To switch to the live notebook, see the [notebook installation instructions](https://github.com/ioam/topographica).\n",
      "\n",
      "This IPython notebook constructs the definition of the SOM retinotopy model in the [Topographica](http://www.topographica.org) simulator, and shows how it organizes.\n",
      "\n",
      "A static version of this notebook may be viewed [online](http://ioam.github.io/media/som_retinotopy.html). To run the live notebook and explore the model interactively, you will need both IPython Notebook and Topographica.\n",
      "\n",
      "If you prefer the older Tk GUI interface or a command line, you may use the som_retinotopy.ty script distributed with Topographica, without IPython Notebook, as follows:\n",
      "\n",
      "```bash\n",
      "./topographica -g examples/som_retinotopy.ty\n",
      "```\n",
      "\n",
      "To run this Notebook version, you can run all cells in order automatically:\n",
      "* Selecting ``Kernel -> Restart`` from the menu above.\n",
      "* Selecting ``Cell -> Run All``. \n",
      "\n",
      "Alternatively, you may run each cell in sequence, starting at the top of the notebook and pressing ``Shift + Enter``."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%reload_ext topo.misc.ipython\n",
      "%opts AxisLayout [tick_format=\"%.1f\" size=70]\n",
      "%timer start"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Model definition\n",
      "The next three cells define the SOM model in its entirety (copied from examples/som_retinotopy.ty).  First, we import required libraries and we declare various parameters to allow the modeller to control the behavior:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "\"\"\"\n",
      "Basic example of a fully connected SOM retinotopic map with ConnectionFields.\n",
      "\n",
      "Contains a Retina (2D Gaussian generator) fully connected to a V1\n",
      "(SOM) sheet, with no initial ordering for topography.\n",
      "\n",
      "Constructed to match the retinotopic simulation from page 53-59 of\n",
      "Miikkulainen, Bednar, Choe, and Sirosh (2005), Computational Maps in\n",
      "the Visual Cortex, Springer.  Known differences include:\n",
      "\n",
      " - The cortex_density and retina_density are smaller for\n",
      "   speed (compared to 40 and 24 in the book).\n",
      " - The original simulation used a radius_0 of 13.3/40, which does work\n",
      "   for some random seeds, but a much larger radius is used here so that\n",
      "   it converges more reliably.\n",
      "\"\"\"\n",
      "import numpy as np\n",
      "import numpy.random\n",
      "\n",
      "import topo\n",
      "import imagen\n",
      "\n",
      "from math import exp, sqrt\n",
      "\n",
      "import param\n",
      "\n",
      "from topo import learningfn,numbergen,transferfn,pattern,projection,responsefn,sheet\n",
      "\n",
      "import topo.learningfn.projfn\n",
      "import topo.pattern.random\n",
      "import topo.responsefn.optimized\n",
      "import topo.transferfn.misc\n",
      "\n",
      "# Disable the measurement progress bar for this notebook\n",
      "from topo.analysis.featureresponses import pattern_response\n",
      "pattern_response.progress_bar = False\n",
      "\n",
      "# Parameters that can be passed on the command line using -p\n",
      "from topo.misc.commandline import global_params as p\n",
      "p.add(\n",
      "\n",
      "    retina_density=param.Number(default=10.0,bounds=(0,None),\n",
      "        inclusive_bounds=(False,True),doc=\"\"\"\n",
      "        The nominal_density to use for the retina.\"\"\"),\n",
      "\n",
      "    cortex_density=param.Number(default=10.0,bounds=(0,None),\n",
      "        inclusive_bounds=(False,True),doc=\"\"\"\n",
      "        The nominal_density to use for V1.\"\"\"),\n",
      "\n",
      "    input_seed=param.Number(default=0,bounds=(0,None),doc=\"\"\"\n",
      "        Seed for the pseudorandom number generator controlling the\n",
      "        input patterns.\"\"\"),\n",
      "\n",
      "    weight_seed=param.Number(default=0,bounds=(0,None),doc=\"\"\"\n",
      "        Seed for the pseudorandom number generator controlling the\n",
      "        initial weight values.\"\"\"),\n",
      "\n",
      "    radius_0=param.Number(default=1.0,bounds=(0,None),doc=\"\"\"\n",
      "        Starting radius for the neighborhood function.\"\"\"),\n",
      "\n",
      "    alpha_0=param.Number(default=0.42,bounds=(0,None),doc=\"\"\"\n",
      "        Starting value for the learning rate.\"\"\"))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "#### Customizing the model parameters\n",
      "\n",
      "The cell below may be used to modify any of the parameters defined above, allowing you to explore the parameter space of the model. To illustrate, the default retina and cortex densities are set to 10 (their default values), but you may change the value of any parameter in this way:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "p.retina_density=10\n",
      "p.cortex_density=10"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The <a href=\"http://ioam.github.io/media/som_retinotopy.html\">online version of this notebook</a> normally uses the full retinal and cortical densities of 24 and 40, respectively, but densities of 10 are the default because they should be sufficient for most purposes.  Using the parameters and definitions above, the following code defines the SOM model of retinotopy, by defining the input patterns, the Sheets, and the connections between Sheets:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Using time dependent random streams.\n",
      "# Corresponding numbergen objects should be given suitable names.\n",
      "param.Dynamic.time_dependent=True\n",
      "numbergen.RandomDistribution.time_dependent = True\n",
      "\n",
      "# input pattern\n",
      "sheet.GeneratorSheet.period = 1.0\n",
      "sheet.GeneratorSheet.phase = 0.05\n",
      "sheet.GeneratorSheet.nominal_density = p.retina_density\n",
      "\n",
      "input_pattern = pattern.Gaussian(scale=1.0,size=2*sqrt(2.0*0.1*24.0)/24.0,\n",
      "    aspect_ratio=1.0,orientation=0,\n",
      "    x=numbergen.UniformRandom(name='xgen'+str(p.input_seed), lbound=-0.5,ubound=0.5),\n",
      "    y=numbergen.UniformRandom(name='ygen'+str(p.input_seed), lbound=-0.5,ubound=0.5))\n",
      "\n",
      "topo.sim['Retina'] = sheet.GeneratorSheet(input_generator=input_pattern)\n",
      "\n",
      "topo.sim['V1'] = sheet.CFSheet(\n",
      "    nominal_density = p.cortex_density,\n",
      "    # Original CMVC simulation used an initial radius of 13.3/40.0 (~0.33)\n",
      "    output_fns=[transferfn.misc.KernelMax(density=p.cortex_density,\n",
      "        kernel_radius=numbergen.BoundedNumber(\n",
      "            bounds=(0.5/40,None),\n",
      "            generator=numbergen.ExponentialDecay(\n",
      "                starting_value=p.radius_0,\n",
      "                time_constant=40000/5.0)))])\n",
      "\n",
      "topo.sim.connect('Retina','V1',name='Afferent',delay=0.05,\n",
      "    connection_type=projection.CFProjection,\n",
      "    weights_generator = pattern.random.UniformRandom(random_generator=np.random.RandomState(seed=p.weight_seed)),\n",
      "    nominal_bounds_template=sheet.BoundingBox(radius=1.0), # fully connected network.\n",
      "    learning_rate=numbergen.ExponentialDecay(\n",
      "        starting_value = p.alpha_0,\n",
      "        time_constant=40000/6.0),\n",
      "    response_fn = responsefn.optimized.CFPRF_EuclideanDistance_opt(),\n",
      "    learning_fn = learningfn.projfn.CFPLF_EuclideanHebbian())\n",
      "\n",
      "'Loaded the self-organizing map model (som_retinotopy)'"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# Exploring the model"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import numpy as np\n",
      "from holoviews import Dimension, Matrix\n",
      "from holoviews.operation import toRGB"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now the model has been defined, we can explore it.  The structure of the loaded model is shown in this screenshot taken from the Tk GUI's Model Editor:\n",
      "\n",
      "### **Model structure**\n",
      "\n",
      "<center>\n",
      "<img src='http://topographica.org/_images/som_network_diagram.png'/>\n",
      "</center>\n",
      "\n",
      "\n",
      "The large circle indicates that the Retina Sheet is fully connected to the V1 Sheet.\n",
      "\n",
      "### **Initial weights**\n",
      "\n",
      "The plot below shows the initial set of weights from a 10x10 subset of the V1 neurons (i.e., every neuron with the reduced cortex_density, or every fourth neuron for cortex_density=40):"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "topo.sim.V1.Afferent.grid()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "As we can see, the initial weights are uniform random. Each neuron receives a full set of connections from all input units, and thus each has a 24x24 or 10x10 array of weights (depending on the retina_density).\n",
      "\n",
      "### **Initial Center-of-Gravity (CoG) plots**"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We can visualize the center of gravity (CoG) of the V1 Afferent weights using the following measurement command:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from topo.command.analysis import measure_cog\n",
      "cog_data = measure_cog()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The center of gravity (a.k.a. centroid or center of mass) is computed for each neuron using its set of incoming weights. The plot below shows each neuron's CoG represented by a point, with a line segment drawn from each neuron to each of its four immediate neighbors so that neighborhood relationships (if any) will be visible."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "cog_data.CoG.Afferent"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "From this plot is is clear that all of the neurons have a CoG near the center of the retina, which is to be expected because the weights are fully connected and evenly distributed (and thus all have an average (X,Y) value near the center of the retina). The same data is also visualized below:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "xcog=cog_data.XCoG.Afferent.last\n",
      "ycog=cog_data.YCoG.Afferent.last\n",
      "xcog + ycog + xcog*ycog"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The V1 X CoG plot shows the X location preferred by each neuron in V1, and the V1 Y CoG plot shows the preferred Y locations. The monochrome values are scaled so that a preference of -0.5 is black and a preference of +0.5 is white, and since they all start out around 0.0 they are all shades of medium gray.  The actual preferences can be accessed directly as a Numpy array if you prefer:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "xcog.data.min(),xcog.data.max(),ycog.data.min(),ycog.data.max()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The colorful XYCoG plot above right shows a false-color visualization of the CoG values in both dimensions at once, where the amount of red in the plot is proportional to the X CoG, and the amount of green in the plot is proportional to the Y CoG. Where both X and Y are low, the plot is black or very dark, and where both are high the plot is yellow (because red and green light together appears yellow). All of the units should be an intermediate color of yellow/gray at this stage, indicating preference for 0.5 in both X and Y. As you will see later, once neurons have developed strong preferences for one location over the others, some neurons will show up as bright black (X and Y both low), red (X high, Y low), red (X low, Y high), or yellow (X and Y both high) in this plot."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Training stimuli\n",
      "\n",
      "We can have a look at what the training patterns that will be used to train the model without having to run it. In the cell labelled ``In [4]`` (in the model definition), we see where the training patterns are defined in a variable called ``input_pattern``. We see that the circular Gaussian stimulus has ``x`` and ``y`` values that are drawn from a random distribution by two independent ``numbergen.UniformRandom`` objects. We can now view what 100 frames of training patterns will look like:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "input_pattern.anim(30, offset=0.05)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Collecting data for animated plots\n",
      "\n",
      "At the end of the notebook, we will generate a set of nice animations showing the plots we have already shown evolve over development. We now create a ``Collector`` object that collects all information needed for plotting and animation. We will collect the information we have just examined and advance the simulation one iteration:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from topo.analysis import Collector\n",
      "from topo.command.analysis import measure_cog\n",
      "\n",
      "c = Collector()\n",
      "c.collect(measure_cog)\n",
      "\n",
      "c.Activity.Retina     = c.collect(topo.sim.Retina)\n",
      "c.Activity.V1         = c.collect(topo.sim.V1)\n",
      "c.Activity.V1Afferent = c.collect(topo.sim.V1.Afferent)\n",
      "c.CFs.Afferent        = c.collect(topo.sim.V1.Afferent, grid=True, rows=10, cols=10)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## **Activity after a single training iteration**\n",
      "\n",
      "The initial activities would be blank, but after running the model a single iteration, the sheet activities now look as follows:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data = c(times=[1])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.Activity.Retina + data.Activity.V1"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In the Retina plot, each photoreceptor is represented as a pixel whose shade of grey codes the response level, increasing from black to white. As expected, this particular example matches the first frame we visualized in the training stimulus animation. The V1 plot shows the response to that input, which for a SOM is initially a large Gaussian-shaped blob centered around the maximally responding unit."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## **Projection Activity**\n",
      "\n",
      "To see what the responses were before SOM\u2019s neighborhood function forced them into a Gaussian shape, you can look at the Projection Activity plot, which shows the feedforward activity in V1:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.Activity.V1Afferent"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Here these responses are best thought of as Euclidean proximity, not distance. This formulation of the SOM response function actually subtracts the distances from the max distance, to ensure that the response will be larger for smaller Euclidean distances (as one intuitively expects for a neural response). The V1 feedforward activity appears random here because the Euclidean distance from the input vector to the initial random weight vector is random.\n",
      "\n",
      "\n",
      "## **Learning after a few iterations**\n",
      "\n",
      "If you look at the weights now we have run a single training iteration, you'll see that most of the neurons have learned new weight patterns based on this input:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.CFs.Afferent"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Some of the weights to each neuron have now changed due to learning. In the SOM algorithm, the unit with the maximum response (i.e., the minimum Euclidean distance between its weight vector and the input pattern) is chosen, and the weights of units within a circular area defined by a Gaussian-shaped neighborhood function around this neuron are updated.\n",
      "\n",
      "\n",
      "This effect is visible in this plot \u2013 a few neurons around the winning unit at the top right have changed their weights. Let us run a few more iterations before having another look:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "topo.sim.run(4)\n",
      "topo.sim.V1.Afferent.grid()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The weights have been updated again - it is clear after these five iterations that the input patterns are becoming represented in the weight patterns, though not very cleanly yet. We also see that the projection activity patterns are becoming smoother, since the weight vectors are now similar between neighboring neurons:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "topo.sim.V1.Afferent.projection_view()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We will now advance to simulation time 250 because we will want to make animations regularly sampled every 250 steps:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "topo.sim.run(245)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## At 5000 iterations\n",
      "\n",
      "Let us use our ``Collector`` object ``c`` to collect more measurements. We will start collecting measurements every 250 steps until we complete 5000 iterations, which should take a few seconds at the default densities, and may be a minute or two at the higher densities:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "times = [topo.sim.time()*i for i in range(1,21)]\n",
      "print(\"Running %d measurements between iteration %s and iteration %s\" % \n",
      "      (len(times), min(times), max(times)))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data = c(data, times=times)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We see that the topographic grid plot evolves over this period as follows:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.CoG.Afferent"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The X and Y CoG plots are now smooth, but not yet the axis-aligned gradients (e.g. left to right and bottom to top) that an optimal topographic mapping would have:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%opts Matrix {+groupwise}\n",
      "(data.XCoG.Afferent + data.YCoG.Afferent + \n",
      " data.XCoG.Afferent * data.YCoG.Afferent).select(Time=5000)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The weight patterns are still quite broad, not very selective for typical input patterns, and not very distinct from one another:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.CFs.Afferent.last"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<center><h3 class='alert-success'>In the live notebook you can remove the ``.last`` to view an animation to 5000 iterations</h3></center>  "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## At 10000 iterations\n",
      "\n",
      "Additional training up to 10000 iterations (which becomes faster due to a smaller neighborhood radius) leads to a nearly flat, square map:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "times = [5000+(250*i) for i in range(1,21)]\n",
      "print(\"Running %d measurements between iteration %s and iteration %s\" % \n",
      "      (len(times), min(times), max(times)))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data = c(data, times=times)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.CoG.Afferent.last"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "<center><h4 class='alert-success'>Note: In the live notebook you can remove ``.last`` to view an animation to 10000 iterations</h4></center>  "
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Yet the weight patterns are still quite broad and not very selective for typical input patterns, because the neighborhood radius (initially 1.0 in Sheet coordinates, i.e. larger than the entire V1) is still large enough to force most cells to respond to the same patterns:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "topo.sim.V1.output_fns[0].kernel_radius"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.CFs.Afferent.last"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## At 30000 iterations"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "By 30000 iterations the map has good coverage of the available portion of the input space:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "times = [10000+(500*i) for i in range(1,41)]\n",
      "print(\"Running %d measurements between iteration %s and iteration %s\\n\\n\" % \n",
      "      (len(times), min(times), max(times)))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data = c(data, times=times)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.CoG.Afferent.last"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The final projection plot at 30000 now shows that each neuron has developed weights concentrated in a small part of the input space, matching a prototypical input at one location, and that neurons each have distinct preferences:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "topo.sim.V1.output_fns[0].kernel_radius"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.CFs.Afferent.last"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The final topographic mapping may have any one of a number of possible arrangements, flipped or rotated by 90 degrees along any axis or both axes.  E.g. a vertically flipped map will have blobs in each CF that are at the bottom for the top row of neurons, but the top for the bottom row of neurons. Nothing in the network reliably drives it to have any particular overall orientation or mapping apart from aligning to the square shape, and each of these possible organizations is equivalent functionally."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Animations over 30000 iterations\n",
      "\n",
      "We can watch how the topographic mapping unfolds over all 30000 iterations we have run:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%%opts Gravity_Contours [rescale_individually=False]\n",
      "data.CoG.Afferent"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "And how the XCoG and YCoG components unfold:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.XCoG.Afferent + data.YCoG.Afferent + data.XCoG.Afferent * data.YCoG.Afferent"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In the false color CoG plot, a perfectly organized mapping will have black in one corner and yellow in the opposite, with green and red in the two other corners, and smooth colors between each corner.   Next, the weights (though by default this animation is disabled to save time, so you would need to remove \".last\" and re-run this cell, which may take several minutes):"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.CFs.Afferent.last"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Snapshots of the retinal and V1 activity over development. Notice how the activity in V1 becomes more focused over time as the kernel radius decreases:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data.Activity.Retina + data.Activity.V1"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Exploring the parameters of the SOM \n",
      "\n",
      "Now, you can re-run the basic simulation by restarting the IPython Notebook Kernel (<code>Kernel -> Restart</code>) or using the keyboard shortcut ``Ctrl+M .`` (see <code>Help -> Keyboard Shortcuts</code> for the full list). Then you can change one of the parameter values, either by editing the model definition above before running those cells, or (usually clearer) by setting the parameters in the cell labelled ``In [3]``.\n",
      "\n",
      "For instance, the starting value of the neighborhood radius (from which all future values are calculated according to exponential decay) is 1.0. You can change this value as you see fit, e.g. to 0.1, by setting ``p.radius_0=0.1``. With such a small learning radius, global ordering is unlikely to happen, and one can expect the topographic grid not to flatten out (despite local order in patches).\n",
      "\n",
      "Similarly, consider changing the initial learning rate from 0.42 to e.g. 1.0 (e.g. by setting ``p.alpha_0=1.0`` in cell 3). The retina and V1 densities cannot be changed after the simulation has started, but again, they can be changed by providing their values above and restarting the notebook.\n",
      "\n",
      "You can also try changing the input_seed (``p.input_seed=XX``), to get a different stream of inputs, or weight_seed (``p.weight_seed=XX``), to get a different set of initial weights. \n",
      "\n",
      "Note that with some of these values, you may encounter cases where the SOM fails to converge even though it seems to be working properly otherwise. For instance, some seed values result in topological defects like a kink:\n",
      "<br>\n",
      "<br>\n",
      "<center><img src=\"https://ioam.github.io/topographica/_images/som_grid_kink.png\"></img></center>"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%timer"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    }
   ],
   "metadata": {}
  }
 ]
}
{'fraction_non_alphanumeric': 0.1443750292115226, 'fraction_numerical': 0.01157555268200726, 'mean_word_length': 2.5212024795655275, 'pattern_counts': {'":': 714, '<': 57, '<?xml version=': 0, '>': 65, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1345756', 'n_tokens_mistral': 20166, 'n_tokens_neox': 19282, 'n_words': 6312}