<commit_before>{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The [Containers](Containers.ipynb) Tutorial introduced the [HoloMap](Containers.ipynb#HoloMap), a core HoloViews data structure that allows easy exploration of parameter spaces. The essence of a HoloMap is that it contains a collection of [Elements](Elements.ipynb) (e.g. Images and Curves) that you can easily select and visualize.\n",
    "\n",
    "HoloMaps hold Elements at sampled points in a multidimensional space. Although this property makes them useful for exploring high-dimensional parameter spaces, HoloMaps can very quickly consume huge amounts of memory for such spaces. For instance, a hundred samples along four orthogonal dimensions would need a HoloMap containing a hundred *million* Elements, each of which could be a substantial object. Thus ``HoloMaps`` have some clear limitations:\n",
    "\n",
    "* HoloMaps may require the generation of millions of elements before they are fully defined.\n",
    "* HoloMaps can easily exhaust all the memory available to Python.\n",
    "* HoloMaps can simultaneously exhaust all the memory in the browser when displayed.\n",
    "* Static export of a notebook containing HoloMaps can result in impractically large HTML files.\n",
    "\n",
    "The ``DynamicMap`` addresses these issues by computing and displaying elements dynamically, allowing exploration of much larger datasets:\n",
    "\n",
    "* DynamicMaps generate elements on the fly allowing the process of exploration to begin immediately.\n",
    "* DynamicMaps do not require fixed sampling, allowing exploration of parameters with arbitrary resolution.\n",
    "* DynamicMaps are lazy in the sense they only compute only as much data as the user wishes to explore.\n",
    "\n",
    "The limitations of ``DynamicMaps`` are:\n",
    "\n",
    "* DynamicMaps require a live notebook server and cannot be directly exported to static HTML.\n",
    "* DynamicMaps do not hold data which reduces the utility of a pickling DynamicMap.\n",
    "* DynamicMaps are often stateful as are their element caches.\n",
    "\n",
    "Now we have enumerated the pros and cons of DynamicMaps, let's create one!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center><div class=\"alert alert-info\" role=\"alert\">To use visualize and use a <b>DynamicMap</b> you need to be running a live Jupyter server.<br>This tutorial assumes that it will be run in a live notebook environment.<br>\n",
    "When viewed statically, DynamicMaps will only show the first available Element.</div></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ``DynamicMap``  <a id='DynamicMap'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start by importing HoloViews and loading the extension:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import holoviews as hv\n",
    "import numpy as np\n",
    "hv.notebook_extension()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will now create the ``DynamicMap`` equivalent of the ``HoloMap`` introduced in the [Containers Tutorial](Containers.ipynb#HoloMap). The ``HoloMap`` in that tutorial consisted of ``Image`` elements containing sine ring arrays as defined by the ``sine_array`` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x,y = np.mgrid[-50:51, -50:51] * 0.1\n",
    "\n",
    "def sine_array(phase, freq):\n",
    "    return np.sin(phase + (freq*x**2+freq*y**2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This function returns NumPy arrays when called:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "sine_array(0,1).shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For a ``DynamicMap`` we will need a function that returns HoloViews elements. This is easy as a trivial extension of the ``sine_array`` function allows it to output [Image](Elements.ipynb#Image) elements:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def sine_image(phase, freq):\n",
    "    return hv.Image(np.sin(phase + (freq*x**2+freq*y**2)))\n",
    "\n",
    "sine_image(0,1) + sine_image(0.5,2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can demonstrate the first type of exploration enabled by a ``DynamicMap`` we call 'bounded' mode."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bounded mode"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A 'bounded' mode ``DynamicMap`` is simply one where all the key dimensions have finite bounds. Bounded mode has the following properties:\n",
    "\n",
    "* The limits of the space and/or the allowable values must be declared for all the key dimensions (unless [sampled mode](#SampledMode) is enabled).\n",
    "* You can explore within the declared bounds at any resolution.\n",
    "* The ``DynamicMap`` is defined using a callable that *must* be a function of its arguments (i.e., the output is strictly determined by the input arguments).\n",
    "\n",
    "We can now create a DynamicMap by simply declaring the ranges of the two dimensions and passing the ``sine_image`` function as the ``.data``:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap = hv.DynamicMap(sine_image, kdims=[hv.Dimension('phase',range=(0, np.pi)),\n",
    "                                 hv.Dimension('frequency', range=(0.01,np.pi))])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This object is created instantly as no data has been generated yet. We can now look at the ``repr`` of this object: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "repr(dmap)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All ``DynamicMaps`` will look similar, only differing in the listed dimensions. Now let's see how this dynamic map visualizes itself:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can combine ``DynamicMaps`` with each other (and any other HoloViews Element) to create a ``Layout`` using the ``+`` operator:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap + hv.DynamicMap(sine_image, kdims=[hv.Dimension('phase',range=(0, np.pi)),\n",
    "                                 hv.Dimension('frequency', range=(0.01,np.pi))])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As both elements are ``DynamicMaps`` with the same dimension ranges, the continuous sliders are retained. If one or more ``HoloMaps`` is used with a ``DynamicMap``, the sliders will snap to the samples available in any ``HoloMap`` in the layout. For bounded ``DynamicMaps`` that do not require ranges to be declared, see [sampled mode](#SampledMode)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you are running this tutorial in a live notebook, you should now see something that looks like the ``HoloMap`` in the [Containers Tutorial](Containers.ipynb#HoloMap). ``DynamicMap`` is in fact a subclass of ``HoloMap`` with some crucial differences:\n",
    "\n",
    "* You can now pick any value of **phase** or **frequency** up to the precision allowed by the slider.\n",
    "* What you see in the cell above will not be exported in any HTML snapshot of the notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Using your own callable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can use any callable to define a ``DynamicMap`` in closed mode. A valid ``DynamicMap`` is defined by the following criteria:\n",
    "\n",
    "* There must be as many positional arguments in the callable signature as key dimensions.\n",
    "* The argument order in the callable signature must match the order of the declared key dimensions.\n",
    "* All key dimensions are defined with a bounded ``range`` or ``values`` parameter (for categorical dimensions).\n",
    "\n",
    "Here is another example of a bounded ``DynamicMap``:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def shapes(N, radius=0.5): # Positional keyword arguments are fine\n",
    "    paths = [hv.Path([[(radius*np.sin(a), radius*np.cos(a)) \n",
    "                        for a in np.linspace(-np.pi, np.pi, n+2)]], \n",
    "                     extents=(-1,-1,1,1)) \n",
    "             for n in range(N,N+3)]\n",
    "    return hv.Overlay(paths)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%opts Path (linewidth=1.5)\n",
    "dmap = hv.DynamicMap(shapes, kdims=[hv.Dimension('N', range=(2,20)), hv.Dimension('radius', range=(0.5,1))])\n",
    "dmap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, you can return ``Overlays`` from ``DynamicMaps`` and ``DynamicMaps`` can be styled in exactly the same way as ``HoloMaps``. Note that currently, ``Overlay`` objects should be returned from the callable and the ``*`` operator is not yet supported at the ``DynamicMap`` level. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%opts Path (linewidth=1.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### The ``DynamicMap`` cache"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Above we mentioned that ``DynamicMap`` is an instance of ``HoloMap``. Does this means it has a ``.data`` attribute?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap.data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is exactly the same sort of ``.data`` as the equivalent ``HoloMap``, except that this value will vary according to how much you explored the parameter space of ``dmap`` using the sliders above. In a ``HoloMap``, ``.data`` contains a defined sampling along the different dimensions, whereas in a ``DynamicMap``, the ``.data`` is simply the *cache*.\n",
    "\n",
    "The cache serves two purposes:\n",
    "\n",
    "* Avoids recomputation of an element should we revisit a particular point in the parameter space. This works well for categorical data, but doesn't help much when using continuous sliders.\n",
    "* Records the space that has been explored with the ``DynamicMap`` when converting to a ``HoloMap``.\n",
    "* Ensures a finite history of generator output when using [open mode](#OpenMode) together with infinite generators.\n",
    "\n",
    "We can always convert *any* ``DynamicMap`` directly to a ``HoloMap`` as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "hv.HoloMap(dmap)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is in fact equivalent to declaring a HoloMap with the same parameters using ``dmap.data`` as input.\n",
    "\n",
    "Although creating a HoloMap this way is easy, the result is poorly controlled as the keys in the HoloMap are defined by how you moved the sliders around. This can be easily rectified using the same key selection semantics of ``HoloMap`` to define exactly which elements are sampled in the cache:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap[{(2,0.5), (2,1.0), (3,0.5), (3,1.0)}] # Returns a *new* DynamicMap with the specified keys in it's cache"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "This object behaves the same way as before it was sampled but now this ``DynamicMap`` can now be exported to static HTML with the allowed slider positions as specified in the cache (without even having to cast to a ``HoloMap``). Nonetheless, casting to a ``HoloMap`` makes the intention to generate a statically exportable, sampled visualization obvious.\n",
    "\n",
    "As the key selection above defines a cartesian product which is one of the most common way to sample across dimensions, there is another convenient syntax which also works with ``HoloMaps``. Here is an equivalent way of defining the same set of keys:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap[{2,3},{0.5,1.0}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap.data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that you can index a ``DynamicMap`` with a literal key in exactly the same way as a ``HoloMap``. If the key exists in the cache, it is returned otherwise a suitable element will be generated. Here is an example of how you can access the last key in the cache as well as create a new element:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap[dmap.keys()[-1]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The default cache size is the reasonably high value of 500 Elements. You can set the size using the ``cache_size`` parameter if you find you are running into issues with memory consumption. A bounded ``DynamicMap`` with ``cache_size=1`` requires the least memory, but will recompute a new Element every time the sliders are moved."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "####  Slicing bounded ``DynamicMaps``"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The declared dimension ranges define the absolute limits allowed for exploration in a bounded ``DynamicMap``. That said, you can use the ``soft_range`` parameter to view subregions within that range. This setting of the ``soft_range`` parameter on dimensions defines the semantics of slicing on bounded ``DynamicMaps``:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "sliced = dmap[4:8, :]\n",
    "sliced"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Open slices are used to release any ``soft_range`` values which resets the limits back to those defined by range:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "sliced[:, 0.8:1.0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The ``[:]`` slice leaves the soft_range values alone and can be used as a convenient way to clone a ``DynamicMap``. Note that you are not allowed to mix slices when any other object type. In other words, once you use a single slice, you have to only uses slices in that indexing operation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sampling DynamicMaps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have now seen one way of sampling a ``DynamicMap``, which is to populate the cache with a set of keys. This approach is designed to make conversion of a ``DynamicMap`` into a ``HoloMap`` easy. One disadvantage of this type of sampling is that populating the cache consumes memory, resulting in many of the same limitations as ``HoloMap``. To avoid this, there are two other ways of sampling a bounded ``DynamicMap``:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Dimension values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "If you want a fixed sampling instead of continuous sliders, yet still wish to retain the online generation of elements as the sliders are moved, you can simply set the dimension values. The result appears to the user just like a ``HoloMap``, and has the same data as the pre-cached version above, but now generates the data dynamically:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "hv.DynamicMap(shapes, kdims=[hv.Dimension('N', values=[2,3,4,5]), \n",
    "                             hv.Dimension('radius', values=[0.7,0.8,0.9,1])])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Sampled mode  <a id='SampledMode'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A bounded ``DynamicMap`` in sampled mode is the least restricted type of ``DynamicMap``, as it can be declared without any information about the allowable dimension ranges or values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap = hv.DynamicMap(shapes, kdims=['N', 'radius'], sampled=True)\n",
    "dmap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, this type of ``DynamicMap`` cannot be visualized in isolation. As before, you can sample the cache and thereby  cast it to a HoloMap as necessary:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap[{2,3},{0.5,1.0}]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The important usage of a sampled ``DynamicMap`` is how it interacts with ``HoloMaps`` in a Layout. As a sampled ``DynamicMap`` doesn't have explicitly declared dimension ranges, it can always adopt the set of sample values from ``HoloMaps`` in the layout."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap + hv.HoloMap({(N,r):shapes(N, r) for N in [3,4,5] for r in [0.5,0.75]},  kdims=['N', 'radius'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The additional convenience of sampled ``DynamicMaps`` is subject to three particular restrictions:\n",
    "\n",
    "* Sampled ``DynamicMaps`` do not visualize themselves in isolation (as we have already seen).\n",
    "* You cannot build a layout consisting of ``DynamicMaps`` only as at least one HoloMap is needed to define the samples.\n",
    "* There cannot be more dimensions declared in the sampled ``DynamicMap`` than across the rest of the layout. We hope to relax this restriction in future."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using groupby to discretize a DynamicMap\n",
    "\n",
    "A DynamicMap also makes it easy to partially or completely discretize a function to evaluate in a complex plot. By grouping over specific dimensions dimensions which define a fixed sampling via the Dimension values parameter can be viewed as a ``GridSpace``, ``NdLayout`` or ``NdOverlay``. If a dimension specifies only a continuous range it can't be grouped over but it may still be explored using the widgets. This means we can plot partial or completely discretized views of a parameter space easily.\n",
    "\n",
    "#### Partially discretize\n",
    "\n",
    "Fundamentally all the groupby operations use the ``groupby`` method, however there are three convenience functions to group dimensions into an ``NdOverlay`` (``.overlay``), ``GridSpace`` (``.grid``) and ``NdLayout`` (``.layout``).\n",
    "\n",
    "Here we will evaluate a simple sine function with three dimensions, the phase, frequency and amplitude. We assign the frequency and amplitude discrete samples and define a continuous range for the phase."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "xs = np.linspace(0, 2*np.pi)\n",
    "\n",
    "def sin(ph, f, amp):\n",
    "    return hv.Curve((xs, np.sin(xs*f+ph)*amp))\n",
    "\n",
    "kdims=[hv.Dimension('phase', range=(0, np.pi)),\n",
    "       hv.Dimension('frequency', values=[0.1, 1, 2, 5, 10]),\n",
    "       hv.Dimension('amplitude', values=[0.5, 5, 10])]\n",
    "\n",
    "sine_dmap = hv.DynamicMap(sin, kdims=kdims)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we define the amplitude dimension to be overlaid and the amplitude dimension to be gridded."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%opts GridSpace [show_legend=True]\n",
    "sine_dmap.overlay('amplitude').grid('frequency')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Fully discretize\n",
    "\n",
    "Now we define a function to evaluate over a space, instead of sampling it manually we can evaluate it using the groupby method. The function is a spiral with a frequency and a first and second order phase term. Then we define the dimension values for all the parameters and declare the DynamicMap."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%opts Path (linewidth=1 color=Palette('Blues'))\n",
    "\n",
    "def spiral_equation(f, ph, ph2):\n",
    "    r = np.arange(0, 1, 0.005)\n",
    "    xs, ys = (r * fn(f*np.pi*np.sin(r+ph)+ph2) for fn in (np.cos, np.sin))\n",
    "    return hv.Path((xs, ys))\n",
    "\n",
    "kdims=[hv.Dimension('f', values=list(np.linspace(1, 10, 10))),\n",
    "       hv.Dimension('ph', values=list(np.linspace(0, np.pi, 10))),\n",
    "       hv.Dimension('ph2', values=list(np.linspace(0, np.pi, 4)))]\n",
    "\n",
    "spiral_dmap = hv.DynamicMap(spiral_equation, kdims=kdims)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can make use of the groupby method directly to group over the frequency and phase dimensions, which we will display as part of a GridSpace by setting the ``container_type``. This leaves second phase variable, which we assign to an NdOverlay by setting the ``group_type``:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%opts GridSpace [xaxis=None yaxis=None] Path [bgcolor='w' xaxis=None yaxis=None]\n",
    "spiral_dmap.groupby(['f', 'ph'], group_type=hv.NdOverlay, container_type=hv.GridSpace)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Open mode <a id='OpenMode'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "``DynamicMap`` also allows unbounded exploration with unbounded dimensions in 'open' mode. There are two key differences of this mode with bounded mode:\n",
    "\n",
    "* Instead of a callable, the input to an open ``DynamicMap`` is a generator. Once created, the generator is only used via ``next()``.\n",
    "* At least one of the declared key dimensions must have an unbounded range. (upper range only? need to check!)\n",
    "* An open mode ``DynamicMap`` can run forever, or until a ``StopIteration`` exception is raised.\n",
    "* Open mode ``DynamicMaps`` can be stateful, with an irreversible direction of time.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Infinite generators"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our first example will be using an infinite generator which plots the histogram for a given number of random samples drawn from a Gaussian distribution:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def gaussian_histogram(samples, scale):\n",
    "    frequencies, edges = np.histogram([np.random.normal(scale=scale) \n",
    "                                       for i in range(samples)], 20)\n",
    "    return hv.Histogram(frequencies, edges).relabel('Gaussian distribution')\n",
    "\n",
    "gaussian_histogram(100,1) + gaussian_histogram(150,1.5) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets now use this in the following generator:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def gaussian_sampler(samples=10, delta=10, scale=1.0):\n",
    "    np.random.seed(1)\n",
    "    while True:\n",
    "        yield gaussian_histogram(samples, scale)\n",
    "        samples+=delta\n",
    "        \n",
    "gaussian_sampler()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Which allows us to define the following infinite ``DynamicMap``:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap = hv.DynamicMap(gaussian_sampler(), kdims=['step'])\n",
    "dmap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that step is shown as an integer. This is the default behavior and corresponds to the call count (i.e the number of times ``next()`` has been called on the generator. If we want to show the actual number of samples properly, we need our generator to return a (key, element) pair:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def gaussian_sampler_kv(samples=10, delta=10, scale=1.0):\n",
    "    np.random.seed(1)\n",
    "    while True:\n",
    "        yield (samples, gaussian_histogram(samples, scale))\n",
    "        samples+=delta\n",
    "        \n",
    "hv.DynamicMap(gaussian_sampler_kv(), kdims=['samples'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that if you pause the ``DynamicMap``, you can scrub back to previous frames in the cache. In other words, you can view a limited history of elements already output by the generator, which does *not* re-execute the generator in any way (as it is indeed impossible to rewind generator state). If you have a stateful generator that, say, depends on the current wind speed in Scotland, this history may be misleading, in which case you can simply set the ``cache_size`` parameter to 1."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Multi-dimensional generators"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In open mode, elements are naturally serialized by a linear sequence of ``next()`` calls but that doesn't mean multiple key dimensions cannot be defined:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def gaussian_sampler_2D(samples=10, scale=1.0, delta=10):\n",
    "    np.random.seed(1)\n",
    "    while True:\n",
    "        yield ((samples, scale), gaussian_histogram(samples, scale))\n",
    "        samples=(samples + delta) if scale==2 else samples\n",
    "        scale = 2 if scale == 1 else 1\n",
    "        \n",
    "dmap = hv.DynamicMap(gaussian_sampler_2D(), kdims=['samples', 'scale'])\n",
    "dmap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we bin the histogram for two different scale values. Above we can visualize this linear sequence of ``next()`` calls but by casting this open map to a ``HoloMap``, we can obtain a multi-dimensional parameter space that we can freely explore:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "hv.HoloMap(dmap)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Finite generators"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Open mode ``DynamicMaps`` are finite and terminate if ``StopIteration`` is raised. This example terminates when two the mean of two sets of gaussian samples fall within a certain distance of each other:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def sample_distributions(samples=10, delta=50, tol=0.04):\n",
    "    np.random.seed(42)\n",
    "    while True:\n",
    "        gauss1 = np.random.normal(size=samples)\n",
    "        gauss2 = np.random.normal(size=samples)\n",
    "        data = (['A']*samples + ['B']*samples, np.hstack([gauss1, gauss2]))\n",
    "        diff = abs(gauss1.mean() - gauss2.mean())\n",
    "        if abs(gauss1.mean() - gauss2.mean()) > tol:\n",
    "            yield ((samples, diff), hv.BoxWhisker(data, kdims=['Group'], vdims=['Value']))\n",
    "        else:\n",
    "            raise StopIteration\n",
    "        samples+=delta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap = hv.DynamicMap(sample_distributions(), kdims=['samples', '$\\delta$'])\n",
    "dmap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now if you are familiar with generators in Python, you might be wondering what happens when a finite generator is exhausted. First we should mention that casting a ``DynamicMap`` to a list is always finite, because  ``__iter__`` returns the cache instead of a potentially infinite generator:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "list(dmap) # The cache"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we know this ``DynamicMap`` is finite, we can make sure it is exhausted it as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "while True:\n",
    "    try:\n",
    "        next(dmap) # Returns Image elements\n",
    "    except StopIteration:\n",
    "        print(\"The dynamic map is exhausted.\")\n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's have a look at the dynamic map:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are given only the repr to reflect that the generator is exhausted. However, as the process of iteration has populated the cache, we can still view the output as a ``HoloMap`` using ``hv.HoloMap(dmap)`` as before."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Counter mode and temporal state"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Open mode is intended to interface live data streams or ongoing simulations with HoloViews. The ``DynamicMap`` will generate live visualizations for as long as new data is requested. Although this works for simple cases, Python generators have problematic limitations that can be resolved using 'counter' mode.\n",
    "\n",
    "In this example, lets say we have a simulation or data recording where time increases in integer steps:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def time_gen(time=1):\n",
    "    while True:\n",
    "        yield time\n",
    "        time += 1\n",
    "        \n",
    "time = time_gen()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we have two generators that return Images that are a function of the simulation time. Here, they have identical output except one of the outputs includes additive noise:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ls = np.linspace(0, 10, 200)\n",
    "xx, yy = np.meshgrid(ls, ls)\n",
    "\n",
    "def cells():\n",
    "    while True:\n",
    "        t = next(time)\n",
    "        arr = np.sin(xx+t)*np.cos(yy+t)\n",
    "        yield hv.Image(arr)\n",
    "\n",
    "def cells_noisy():\n",
    "    while True:\n",
    "        t = next(time)\n",
    "        arr = np.sin(xx+t)*np.cos(yy+t)\n",
    "        yield hv.Image(arr + 0.2*np.random.rand(200,200))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's create a Layout using these two generators:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "hv.DynamicMap(cells(), kdims=['time']) + hv.DynamicMap(cells_noisy(), kdims=['time'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you pause the animation, you'll see that these two outputs are *not* in phase, despite the fact that the generators are defined identically (modulo the additive noise)!\n",
    "\n",
    "The issue is that generators are used via the ``next()`` interface, and so when either generator is called, the simulation time is increased. In other words, the noisy version in subfigure **B** actually corresponds to a later time than in subfigure **A**.\n",
    "\n",
    "This is a fundamental issue, as the ``next`` method does not take arguments. What we want is for all the ``DynamicMaps`` presented in a Layout to share a common simulation time, which is only incremented by interaction with the scrubber widget. This is exactly the sort of situation where you want to use counter mode."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Handling time-dependent state"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To define a ``DynamicMap`` in counter mode:\n",
    "\n",
    "* Leave one or more dimensions *unbounded* (as in open mode)\n",
    "* Supply a callable (as in bounded mode) that accepts *one* argument\n",
    "\n",
    "This callable should act in the same way as the generators of open mode except the output in controlled by the single counter argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ls = np.linspace(0, 10, 200)\n",
    "xx, yy = np.meshgrid(ls, ls)\n",
    "\n",
    "def cells_counter(t):\n",
    "    while True:\n",
    "        arr = np.sin(xx+t)*np.cos(yy+t)\n",
    "        return hv.Image(arr)\n",
    "\n",
    "def cells_noisy_counter(t):\n",
    "    while True:\n",
    "        arr = np.sin(xx+t)*np.cos(yy+t)\n",
    "        return hv.Image(arr + 0.2*np.random.rand(200,200))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now if we supply these functions instead of generators:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "hv.DynamicMap(cells_counter, kdims=['time']) + hv.DynamicMap(cells_noisy_counter, kdims=['time'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now **A** and **B** are correctly in phase.\n",
    "\n",
    "Unfortunately, an integer counter is often too simple to describe simulation time, which is more often a float with real-world units. To address this, we can simply return the actual key values we want along the time dimension, just as was demonstrated in open mode using generators:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "ls = np.linspace(0, 10, 200)\n",
    "xx, yy = np.meshgrid(ls, ls)\n",
    "\n",
    "# Example of a global simulation time\n",
    "# typical in many applications\n",
    "t = 0 \n",
    "        \n",
    "def cells_counter_kv(c):\n",
    "    global t\n",
    "    t = 0.1 * c\n",
    "    while True:\n",
    "        arr = np.sin(xx+t)*np.cos(yy+t)\n",
    "        return (t, hv.Image(arr))\n",
    "\n",
    "def cells_noisy_counter_kv(c):\n",
    "    global t\n",
    "    t = 0.1 * c\n",
    "    while True:\n",
    "        arr = np.sin(xx+t)*np.cos(yy+t)\n",
    "        return (t, hv.Image(arr + 0.2*np.random.rand(200,200)))\n",
    "    \n",
    "hv.DynamicMap(cells_counter_kv, kdims=['time']) + hv.DynamicMap(cells_noisy_counter_kv, kdims=['time'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "print(\"The global simulation time is now t=%f\" % t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ensuring that the HoloViews counter maps to a suitable simulation time is the responsibility of the user. However, once a consistent scheme is configured, the callable in each ``DynamicMap`` can specify the desired simulation time. If the requested simulation time is the same as the current simulation time, nothing needs to happen. Otherwise, the simulator can be run forward by the requested amount."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Slicing in open and counter mode"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Slicing open and counter mode ``DynamicMaps`` has the exact same semantics as normal ``HoloMap`` slicing, except now the ``.data`` attribute corresponds to the cache. For instance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def sine_kv_gen(phase=0, freq=0.5):\n",
    "    while True:\n",
    "        yield (phase, hv.Image(np.sin(phase + (freq*x**2+freq*y**2))))\n",
    "        phase+=0.2\n",
    "        \n",
    "dmap = hv.DynamicMap(sine_kv_gen(), kdims=['phase'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's fill the cache with some elements:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "for i in range(21):\n",
    "    dmap.next()\n",
    "    \n",
    "print(\"Min key value in cache:%s\\nMax key value in cache:%s\" % (min(dmap.keys()), max(dmap.keys())))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "sliced = dmap[1:3.1]\n",
    "print(\"Min key value in cache:%s\\nMax key value in cache:%s\" % (min(sliced.keys()), max(sliced.keys())))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## DynamicMaps and normalization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By default, a ``HoloMap`` normalizes the display of elements according the minimum and maximum values found across the ``HoloMap``. This automatic behavior is not possible in a ``DynamicMap`` where arbitrary new elements are being generated on the fly. Consider the following examples where the arrays contained within the returned ``Image`` objects are scaled with time:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%opts Image {+axiswise}\n",
    "ls = np.linspace(0, 10, 200)\n",
    "xx, yy = np.meshgrid(ls, ls)\n",
    "\n",
    "def cells(vrange=False):\n",
    "    \"The range is set on the value dimension when vrange is True \"\n",
    "    time = time_gen()\n",
    "    while True:\n",
    "        t = next(time)\n",
    "        arr = t*np.sin(xx+t)*np.cos(yy+t)\n",
    "        vdims=[hv.Dimension('Intensity', range=(0,10))] if vrange else ['Intensity']\n",
    "        yield hv.Image(arr, vdims=vdims)\n",
    "\n",
    "hv.DynamicMap(cells(vrange=False), kdims=['time']) + hv.DynamicMap(cells(vrange=True), kdims=['time'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we use ``+axiswise`` to see the behavior of the two cases independently. We see in **A** that when ``vrange=False`` and no range is set on the value dimension, no automatic normalization occurs (as it would for a ``HoloMap``). In **B** we see that normalization is applied but only when the value dimension range has been specified. \n",
    "\n",
    "In other words ``DynamicMaps`` do not support automatic normalization across their elements but do support the same explicit normalization behavior as ``HoloMaps``. Values that are generated outside this range are simply clipped according the usual semantics of explicit value dimension ranges. \n",
    "\n",
    "Note that we can always have the option of casting a ``DynamicMap`` to a ``HoloMap`` in order to automatically normalize across the cached values without needing explicit value dimension ranges."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using DynamicMaps in your code\n",
    "\n",
    "As you can see, ``DynamicMaps`` let you use HoloViews with a very wide range of dynamic data formats and sources, making it simple to visualize ongoing processes or very large data spaces."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3+"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}<commit_msg>Cleaned up and clarified Dynamic_Map tutorial<commit_after>{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The [Containers](Containers.ipynb) Tutorial introduced the [HoloMap](Containers.ipynb#HoloMap), a core HoloViews data structure that allows easy exploration of parameter spaces. The essence of a HoloMap is that it contains a collection of [Elements](Elements.ipynb) (e.g. Images and Curves) that you can easily select and visualize.\n",
    "\n",
    "HoloMaps hold Elements at specific sampled points in a multidimensional space. Although this property makes them useful for exploring high-dimensional parameter spaces, HoloMaps can very quickly consume huge amounts of memory for such spaces. For instance, a hundred samples along four orthogonal dimensions would need a HoloMap containing a hundred *million* Elements, each of which could be a substantial object that takes time to create and costs memory to store. Thus ``HoloMaps`` have some clear limitations:\n",
    "\n",
    "* HoloMaps may require the generation of millions of elements before they are fully defined.\n",
    "* HoloMaps can easily exhaust all the memory available to Python.\n",
    "* HoloMaps can simultaneously exhaust all the memory in the browser when displayed.\n",
    "* Static export of a notebook containing HoloMaps can result in impractically large HTML files.\n",
    "\n",
    "The ``DynamicMap`` addresses these issues by computing and displaying elements dynamically, allowing exploration of much larger datasets:\n",
    "\n",
    "* DynamicMaps generate elements on the fly, allowing the process of exploration to begin immediately.\n",
    "* DynamicMaps do not require fixed sampling, allowing exploration of parameters with arbitrary resolution.\n",
    "* DynamicMaps are lazy in the sense they only compute only as much data as the user wishes to explore.\n",
    "\n",
    "The limitations of ``DynamicMaps`` are:\n",
    "\n",
    "* DynamicMaps require a live notebook server and cannot be fully exported to static HTML.\n",
    "* DynamicMaps do not store copies of the underlying data, which reduces the utility of pickling a DynamicMap.\n",
    "* DynamicMaps are often stateful, as are their element caches, which can make them more difficult to reason about.\n",
    "\n",
    "In order to handle the various situations in which one might want to use a ``DynamicMap``, ``DynamicMaps`` can be used in various \"modes\" that will be each described separately below:\n",
    "\n",
    "* [**Bounded mode:**](#BoundedMode) All dimension ranges specified, allowing exploration within the ranges.\n",
    "* [**Sampled mode:**](#SampledMode) Some dimension ranges left unspecified, making the ``DynamicMap`` not viewable directly, but useful in combination with other objects or for later specifying samples or ranges.\n",
    "* [**Open mode:**](#OpenMode) Dimension ranges not specified, with new elements created on request using a generator.\n",
    "* [**Counter mode:**](#CounterMode) Dimension ranges not specified, with new elements created based on a shared counter state, e.g. from a simulation.\n",
    "\n",
    "All this will make much more sense once we've tried out some ``DynamicMaps`` and showed how they work, so let's create one!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center><div class=\"alert alert-info\" role=\"alert\">To use visualize and use a <b>DynamicMap</b> you need to be running a live Jupyter server.<br>This tutorial assumes that it will be run in a live notebook environment.<br>\n",
    "When viewed statically, DynamicMaps will only show the first available Element,<br> and will thus not have any sliders, making it difficult to follow the descriptions below.<br><br>\n",
    "It's also best to run this notebook one cell at a time, not via \"Run All\",<br> so that subsequent cells can reflect your dynamic interaction with previous cells.</div></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ``DynamicMap``  <a id='DynamicMap'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start by importing HoloViews and loading the extension:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import holoviews as hv\n",
    "import numpy as np\n",
    "hv.notebook_extension()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will now create the ``DynamicMap`` equivalent of the ``HoloMap`` introduced in the [Containers Tutorial](Containers.ipynb#HoloMap). The ``HoloMap`` in that tutorial consisted of ``Image`` elements containing sine ring arrays as defined by the ``sine_array`` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x,y = np.mgrid[-50:51, -50:51] * 0.1\n",
    "\n",
    "def sine_array(phase, freq):\n",
    "    return np.sin(phase + (freq*x**2+freq*y**2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This function returns NumPy arrays when called:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "sine_array(0,1).shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For a ``DynamicMap`` we will need a function that returns HoloViews elements. It is easy to modify the ``sine_array`` function so that it returns [Image](Elements.ipynb#Image) elements instead of raw arrays:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def sine_image(phase, freq):\n",
    "    return hv.Image(np.sin(phase + (freq*x**2+freq*y**2)))\n",
    "\n",
    "sine_image(0,1) + sine_image(0.5,2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can demonstrate the first type of exploration enabled by a ``DynamicMap``, called 'bounded' mode."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bounded mode <a id='BoundedMode'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A 'bounded' mode ``DynamicMap`` is simply one where all the key dimensions have finite bounds. Bounded mode has the following properties:\n",
    "\n",
    "* The limits of the space and/or the allowable values must be declared for all the key dimensions (unless [sampled mode](#SampledMode) is also enabled).\n",
    "* You can explore within the declared bounds at any resolution.\n",
    "* The data for the ``DynamicMap`` is defined using a callable that *must* be a function of its arguments (i.e., the output is strictly determined by the input arguments).\n",
    "\n",
    "We can now create a DynamicMap by simply declaring the ranges of the two dimensions and passing the ``sine_image`` function as the ``.data``:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap = hv.DynamicMap(sine_image, kdims=[hv.Dimension('phase',range=(0, np.pi)),\n",
    "                                 hv.Dimension('frequency', range=(0.01,np.pi))])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This object is created instantly, because it doesn't generate any `hv.Image` objects initially. We can now look at the ``repr`` of this object: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "repr(dmap)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All ``DynamicMaps`` will look similar, only differing in the listed dimensions. Now let's see how this dynamic map visualizes itself:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here each `hv.Image` object visualizing a particular sine ring pattern with the given parameters is created dynamically, whenever the slider is set to that value.  Any value in the allowable range can be requested, with a rough step size supported by sliding, and more precise values available by using the left and right arrow keys to change the value in small steps. In each case the new image is dynamically generated based on whatever the slider's values are.\n",
    "\n",
    "As for any HoloViews Element, you can combine ``DynamicMaps`` to create a ``Layout`` using the ``+`` operator:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap + hv.DynamicMap(sine_image, kdims=[hv.Dimension('phase',range=(0, np.pi)),\n",
    "                                 hv.Dimension('frequency', range=(0.01,np.pi))])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As both elements are ``DynamicMaps`` with the same dimension ranges, the continuous sliders are retained. If one or more ``HoloMaps`` is used with a ``DynamicMap``, the sliders will snap to the samples available in any ``HoloMap`` in the layout. For bounded ``DynamicMaps`` that do not require ranges to be declared, see [sampled mode](#SampledMode) below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you are running this tutorial in a live notebook, the above cell should look like the ``HoloMap`` in the [Containers Tutorial](Containers.ipynb#HoloMap). ``DynamicMap`` is in fact a subclass of ``HoloMap`` with some crucial differences:\n",
    "\n",
    "* You can now pick any value of **phase** or **frequency** up to the precision allowed by the slider.\n",
    "* What you see in the cell above will not be exported in any HTML snapshot of the notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Using your own callable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can use any callable to define a ``DynamicMap`` in closed mode. A valid ``DynamicMap`` is defined by the following criteria:\n",
    "\n",
    "* There must be as many positional arguments in the callable signature as key dimensions.\n",
    "* The argument order in the callable signature must match the order of the declared key dimensions.\n",
    "* All key dimensions are defined with a bounded ``range`` or ``values`` parameter (for categorical dimensions).\n",
    "\n",
    "Here is another example of a bounded ``DynamicMap``:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def shapes(N, radius=0.5): # Positional keyword arguments are fine\n",
    "    paths = [hv.Path([[(radius*np.sin(a), radius*np.cos(a)) \n",
    "                        for a in np.linspace(-np.pi, np.pi, n+2)]], \n",
    "                     extents=(-1,-1,1,1)) \n",
    "             for n in range(N,N+3)]\n",
    "    return hv.Overlay(paths)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%opts Path (linewidth=1.5)\n",
    "dmap = hv.DynamicMap(shapes, kdims=[hv.Dimension('N', range=(2,20)), hv.Dimension('radius', range=(0.5,1))])\n",
    "dmap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, you can return ``Overlays`` from ``DynamicMaps``, and ``DynamicMaps`` can be styled in exactly the same way as ``HoloMaps``. Note that currently, ``Overlay`` objects should be returned from the callable itself; the ``*`` operator is not yet supported at the ``DynamicMap`` level (i.e., between a ``DynamicMap`` and other Elements). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%opts Path (linewidth=1.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### The ``DynamicMap`` cache"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Above we mentioned that ``DynamicMap`` is an instance of ``HoloMap``. Does this mean it has a ``.data`` attribute?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap.data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is exactly the same sort of ``.data`` as the equivalent ``HoloMap``, except that this value will vary according to how much you explored the parameter space of ``dmap`` using the sliders above. In a ``HoloMap``, ``.data`` contains a defined sampling along the different dimensions, whereas in a ``DynamicMap``, the ``.data`` is simply the *cache*.\n",
    "\n",
    "The cache serves two purposes:\n",
    "\n",
    "* Avoids recomputation of an element should we revisit a particular point in the parameter space. This works well for categorical or integer dimensions, but doesn't help much when using continuous sliders for real-valued dimensions.\n",
    "* Records the space that has been explored with the ``DynamicMap`` for any later conversion to a ``HoloMap``.\n",
    "* Ensures that we store only a finite history of generator output when using [open mode](#OpenMode) together with infinite generators.\n",
    "\n",
    "We can always convert *any* ``DynamicMap`` directly to a ``HoloMap`` as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "hv.HoloMap(dmap)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is in fact equivalent to declaring a HoloMap with the same parameters (dimensions, etc.) using ``dmap.data`` as input, but is more convenient.\n",
    "\n",
    "Although creating a HoloMap this way is easy, the result is poorly controlled, as the keys in the HoloMap are defined by how you moved the sliders around. For instance, if you run this tutorial straight through using `Run All`, the above cell won't have any sliders at all, because only a single element will be in the dmap's cache until the `DynamicMap`s sliders are dragged.\n",
    "\n",
    "If you instead want to specify a specific set of samples, you can easily do so by using the same key-selection semantics as for a ``HoloMap`` to define exactly which elements are initially sampled in the cache:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap[{(2,0.5), (2,1.0), (3,0.5), (3,1.0)}] # Returns a *new* DynamicMap with the specified keys in its cache"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "This object behaves the same way as before it was sampled, but now this ``DynamicMap`` can now be exported to static HTML with the allowed slider positions as specified in the cache, without even having to cast to a ``HoloMap``.  Of course, if the intent is primarily to have something statically exportable, then it's still a good idea to explicitly cast it to a ``HoloMap`` so that it will clearly contain only a finite set of Elements.\n",
    "\n",
    "The key selection above happens to define a Cartesian product, which is one of the most common way to sample across dimensions. Because the list of such dimension values can quickly get very large when enumerated as above, we provide a way to specify a Cartesian product directly, which also works with ``HoloMaps``. Here is an equivalent way of defining the same set of four points in that two-dimensional space:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap[{2,3},{0.5,1.0}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap.data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that you can index a ``DynamicMap`` with a literal key in exactly the same way as a ``HoloMap``. If the key exists in the cache, it is returned directly, otherwise a suitable element will be generated. Here is an example of how you can access the last key in the cache, creating such an element if it didn't already exist:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap[dmap.keys()[-1]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The default cache size of 500 Elements is relatively high so that interactive exploration will work smoothly, but you can reduce it using the ``cache_size`` parameter if you find you are running into issues with memory consumption. A bounded ``DynamicMap`` with ``cache_size=1`` requires the least memory, but will recompute a new Element every time the sliders are moved, making it less responsive."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "####  Slicing bounded ``DynamicMaps``"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The declared dimension ranges define the absolute limits allowed for exploration in a bounded ``DynamicMap``. That said, you can use the ``soft_range`` parameter to view subregions within that range. Setting the ``soft_range`` parameter on dimensions can be done conveniently using slicing on bounded ``DynamicMaps``:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "sliced = dmap[4:8, :]\n",
    "sliced"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that N is now restricted to the range 4:8.  Open slices are used to release any ``soft_range`` values, which resets the limits back to those defined by the full range:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "sliced[:, 0.8:1.0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The ``[:]`` slice leaves the soft_range values alone and can be used as a convenient way to clone a ``DynamicMap``. Note that mixing slices with any other object type is not supported. In other words, once you use a single slice, you can only use slices in that indexing operation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sampling DynamicMaps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have now seen one way of sampling a ``DynamicMap``, which is to populate the cache with a set of keys. This approach is designed to make conversion of a ``DynamicMap`` into a ``HoloMap`` easy. One disadvantage of this type of sampling is that populating the cache consumes memory, resulting in many of the same limitations as ``HoloMap``. To avoid this, there are two other ways of sampling a bounded ``DynamicMap``:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Dimension values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "If you want a fixed sampling instead of continuous sliders, yet still wish to retain the online generation of elements as the sliders are moved, you can simply declare the dimension to have a fixed list of values. The result appears to the user just like a ``HoloMap``, and will show the same data as the pre-cached version above, but now generates the data dynamically to reduce memory requirements and speed up the initial display:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "hv.DynamicMap(shapes, kdims=[hv.Dimension('N', values=[2,3,4,5]), \n",
    "                             hv.Dimension('radius', values=[0.7,0.8,0.9,1])])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Sampled mode  <a id='SampledMode'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A bounded ``DynamicMap`` in `sampled` mode is the least restricted type of ``DynamicMap``, as it can be declared without any information about the allowable dimension ranges or values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap = hv.DynamicMap(shapes, kdims=['N', 'radius'], sampled=True)\n",
    "dmap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, this type of ``DynamicMap`` cannot be visualized in isolation, because there are no values or ranges specified for the dimensions. You could view it by explicitly specifying such values to get cached samples (and can then cast it to a HoloMap if desired):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap[{2,3},{0.5,1.0}]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Still, on its own, a sampled-mode ``DynamicMap`` may not seem very useful.  When they become very convenient is when combined with one or more ``HoloMaps`` in a ``Layout``. Because a sampled ``DynamicMap`` doesn't have explicitly declared dimension ranges, it can always adopt the set of sample values from ``HoloMaps`` in the layout."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap + hv.HoloMap({(N,r):shapes(N, r) for N in [3,4,5] for r in [0.5,0.75]},  kdims=['N', 'radius'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this way, you only need to worry about choosing specific values or ranges for your dimensions for a single (``HoloMap``) object, with the others left open-ended as sampled-mode ``DynamicMaps``.  This convenience is subject to three particular restrictions:\n",
    "\n",
    "* Sampled ``DynamicMaps`` do not visualize themselves in isolation (as we have already seen).\n",
    "* You cannot build a layout consisting of sampled-mode ``DynamicMaps`` only, because at least one HoloMap is needed to define the samples.\n",
    "* There currently cannot be more dimensions declared in the sampled ``DynamicMap`` than across the rest of the layout. We hope to relax this restriction in future."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using groupby to discretize a DynamicMap\n",
    "\n",
    "A DynamicMap also makes it easy to partially or completely discretize a function to evaluate in a complex plot. By grouping over specific dimensions that define a fixed sampling via the Dimension values parameter, the DynamicMap can be viewed as a ``GridSpace``, ``NdLayout``, or ``NdOverlay``. If a dimension specifies only a continuous range it can't be grouped over, but it may still be explored using the widgets. This means we can plot partial or completely discretized views of a parameter space easily.\n",
    "\n",
    "#### Partially discretize\n",
    "\n",
    "The implementation for all the groupby operations uses the ``.groupby`` method internally, but we also provide three higher-level convenience methods to group dimensions into an ``NdOverlay`` (``.overlay``), ``GridSpace`` (``.grid``), or ``NdLayout`` (``.layout``).\n",
    "\n",
    "Here we will evaluate a simple sine function with three dimensions, the phase, frequency, and amplitude. We assign the frequency and amplitude discrete samples, while defining a continuous range for the phase:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "xs = np.linspace(0, 2*np.pi)\n",
    "\n",
    "def sin(ph, f, amp):\n",
    "    return hv.Curve((xs, np.sin(xs*f+ph)*amp))\n",
    "\n",
    "kdims=[hv.Dimension('phase', range=(0, np.pi)),\n",
    "       hv.Dimension('frequency', values=[0.1, 1, 2, 5, 10]),\n",
    "       hv.Dimension('amplitude', values=[0.5, 5, 10])]\n",
    "\n",
    "sine_dmap = hv.DynamicMap(sin, kdims=kdims)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we define the amplitude dimension to be overlaid and the frequency dimension to be gridded:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%opts GridSpace [show_legend=True fig_size=200]\n",
    "sine_dmap.overlay('amplitude').grid('frequency')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, instead of having three sliders (one per dimension), we've now laid out the frequency dimension as a discrete set of values in a grid, and the amplitude dimension as a discrete set of values in an overlay, leaving one slider for the remaining dimension (phase).  This approach can help you visualize a large, multi-dimensional space efficiently, with full control over how each dimension is made visible.\n",
    "\n",
    "\n",
    "#### Fully discretize\n",
    "\n",
    "Given a continuous function defined over a space, we could sample it manually, but here we'll look at an example of evaluating it using the groupby method. Let's look at a spiral function with a frequency and first- and second-order phase terms. Then we define the dimension values for all the parameters and declare the DynamicMap:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%opts Path (linewidth=1 color=Palette('Blues'))\n",
    "\n",
    "def spiral_equation(f, ph, ph2):\n",
    "    r = np.arange(0, 1, 0.005)\n",
    "    xs, ys = (r * fn(f*np.pi*np.sin(r+ph)+ph2) for fn in (np.cos, np.sin))\n",
    "    return hv.Path((xs, ys))\n",
    "\n",
    "kdims=[hv.Dimension('f', values=list(np.linspace(1, 10, 10))),\n",
    "       hv.Dimension('ph', values=list(np.linspace(0, np.pi, 10))),\n",
    "       hv.Dimension('ph2', values=list(np.linspace(0, np.pi, 4)))]\n",
    "\n",
    "spiral_dmap = hv.DynamicMap(spiral_equation, kdims=kdims)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can make use of the ``.groupby`` method to group over the frequency and phase dimensions, which we will display as part of a GridSpace by setting the ``container_type``. This leaves the second phase variable, which we assign to an NdOverlay by setting the ``group_type``:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%opts GridSpace [xaxis=None yaxis=None] Path [bgcolor='w' xaxis=None yaxis=None]\n",
    "spiral_dmap.groupby(['f', 'ph'], group_type=hv.NdOverlay, container_type=hv.GridSpace)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This grid shows a range of frequencies `f` on the x axis, a range of the first phase variable `ph` on the `y` axis, and a range of different `ph2` phases as overlays within each location in the grid.  As you can see, these techniques can help you visualize multidimensional parameter spaces compactly and conveniently.\n",
    "\n",
    "### Open mode <a id='OpenMode'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "``DynamicMap`` also allows unconstrained exploration over unbounded dimensions in 'open' mode. There are two key differences between open mode and bounded mode:\n",
    "\n",
    "* Instead of a callable, the input to an open ``DynamicMap`` is a generator. Once created, the generator is only used via ``next()``.\n",
    "* At least one of the declared key dimensions must have an unbounded range (i.e., with an upper or lower bound not specified).\n",
    "* An open mode ``DynamicMap`` can run forever, or until a ``StopIteration`` exception is raised.\n",
    "* Open mode ``DynamicMaps`` can be stateful, with an irreversible direction of time.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Infinite generators"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our first example will be using an infinite generator which plots the histogram for a given number of random samples drawn from a Gaussian distribution:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def gaussian_histogram(samples, scale):\n",
    "    frequencies, edges = np.histogram([np.random.normal(scale=scale) \n",
    "                                       for i in range(samples)], 20)\n",
    "    return hv.Histogram(frequencies, edges).relabel('Gaussian distribution')\n",
    "\n",
    "gaussian_histogram(100,1) + gaussian_histogram(150,1.5) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets now use this in the following generator:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def gaussian_sampler(samples=10, delta=10, scale=1.0):\n",
    "    np.random.seed(1)\n",
    "    while True:\n",
    "        yield gaussian_histogram(samples, scale)\n",
    "        samples+=delta\n",
    "        \n",
    "gaussian_sampler()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Which allows us to define the following infinite ``DynamicMap``:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap = hv.DynamicMap(gaussian_sampler(), kdims=['step'])\n",
    "dmap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that step is shown as an integer. This is the default behavior and corresponds to the call count (i.e the number of times ``next()`` has been called on the generator. If we want to show the actual number of samples properly, we need our generator to return a (key, element) pair:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def gaussian_sampler_kv(samples=10, delta=10, scale=1.0):\n",
    "    np.random.seed(1)\n",
    "    while True:\n",
    "        yield (samples, gaussian_histogram(samples, scale))\n",
    "        samples+=delta\n",
    "        \n",
    "hv.DynamicMap(gaussian_sampler_kv(), kdims=['samples'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that if you pause the ``DynamicMap``, you can scrub back to previous frames in the cache. In other words, you can view a limited history of elements already output by the generator, which does *not* re-execute the generator in any way (as it is indeed impossible to rewind generator state). If you have a stateful generator that, say, depends on the current wind speed in Scotland, this history may be misleading, in which case you can simply set the ``cache_size`` parameter to 1."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Multi-dimensional generators"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In open mode, elements are naturally serialized by a linear sequence of ``next()`` calls, yet multiple key dimensions can still be defined:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def gaussian_sampler_2D(samples=10, scale=1.0, delta=10):\n",
    "    np.random.seed(1)\n",
    "    while True:\n",
    "        yield ((samples, scale), gaussian_histogram(samples, scale))\n",
    "        samples=(samples + delta) if scale==2 else samples\n",
    "        scale = 2 if scale == 1 else 1\n",
    "        \n",
    "dmap = hv.DynamicMap(gaussian_sampler_2D(), kdims=['samples', 'scale'])\n",
    "dmap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we bin the histogram for two different scale values. Above we can visualize this linear sequence of ``next()`` calls, but by casting this open map to a ``HoloMap``, we can obtain a multi-dimensional parameter space that we can freely explore:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "hv.HoloMap(dmap)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that if you ran this notebook using `Run All`, only a single frame will be available in the above cell, with no sliders, but if you ran it interactively and viewed a range of values in the previous cell, you'll have multiple sliders in this cell allowing you to explore whatever range of frames is in the cache from the previous cell.\n",
    "\n",
    "#### Finite generators"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Open mode ``DynamicMaps`` are finite and terminate if ``StopIteration`` is raised. This example terminates when the means of two sets of gaussian samples fall within a certain distance of each other:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def sample_distributions(samples=10, delta=50, tol=0.04):\n",
    "    np.random.seed(42)\n",
    "    while True:\n",
    "        gauss1 = np.random.normal(size=samples)\n",
    "        gauss2 = np.random.normal(size=samples)\n",
    "        data = (['A']*samples + ['B']*samples, np.hstack([gauss1, gauss2]))\n",
    "        diff = abs(gauss1.mean() - gauss2.mean())\n",
    "        if abs(gauss1.mean() - gauss2.mean()) > tol:\n",
    "            yield ((samples, diff), hv.BoxWhisker(data, kdims=['Group'], vdims=['Value']))\n",
    "        else:\n",
    "            raise StopIteration\n",
    "        samples+=delta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap = hv.DynamicMap(sample_distributions(), kdims=['samples', '$\\delta$'])\n",
    "dmap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now if you are familiar with generators in Python, you might be wondering what happens when a finite generator is exhausted. First we should mention that casting a ``DynamicMap`` to a list is always finite, because  ``__iter__`` returns the cache instead of a potentially infinite generator:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "list(dmap) # The cache"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we know this ``DynamicMap`` is finite, we can make sure it is exhausted as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "while True:\n",
    "    try:\n",
    "        next(dmap) # Returns Image elements\n",
    "    except StopIteration:\n",
    "        print(\"The dynamic map is exhausted.\")\n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's have a look at the dynamic map:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "dmap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, we are given only the text-based repr, to indicate that the generator is exhausted. However, as the process of iteration has populated the cache, we can still view the output as a ``HoloMap`` using ``hv.HoloMap(dmap)`` as before."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Counter mode and temporal state  <a id='CounterMode'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Open mode is intended to use live data streams or ongoing simulations with HoloViews. The ``DynamicMap`` will generate live visualizations for as long as new data is requested. Although this works for simple cases, Python generators have problematic limitations that can be resolved using 'counter' mode.\n",
    "\n",
    "In this example, let's say we have a simulation or data recording where time increases in integer steps:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def time_gen(time=1):\n",
    "    while True:\n",
    "        yield time\n",
    "        time += 1\n",
    "        \n",
    "time = time_gen()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's create two generators that return Images that are a function of the simulation time. Here, they have identical output except one of the outputs includes additive noise:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ls = np.linspace(0, 10, 200)\n",
    "xx, yy = np.meshgrid(ls, ls)\n",
    "\n",
    "def cells():\n",
    "    while True:\n",
    "        t = next(time)\n",
    "        arr = np.sin(xx+t)*np.cos(yy+t)\n",
    "        yield hv.Image(arr)\n",
    "\n",
    "def cells_noisy():\n",
    "    while True:\n",
    "        t = next(time)\n",
    "        arr = np.sin(xx+t)*np.cos(yy+t)\n",
    "        yield hv.Image(arr + 0.2*np.random.rand(200,200))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's create a Layout using these two generators:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "hv.DynamicMap(cells(), kdims=['time']) + hv.DynamicMap(cells_noisy(), kdims=['time'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you pause the animation, you'll see that these two outputs are *not* in phase, despite the fact that the generators are defined identically (modulo the additive noise)!\n",
    "\n",
    "The issue is that generators are used via the ``next()`` interface, and so when either generator is called, the simulation time is increased. In other words, the noisy version in subfigure **B** actually corresponds to a later time than in subfigure **A**.\n",
    "\n",
    "This is a fundamental issue, as the ``next`` method does not take arguments. What we want is for all the ``DynamicMaps`` presented in a Layout to share a common simulation time, which is only incremented by interaction with the scrubber widget. This is exactly the sort of situation where you want to use counter mode."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Handling time-dependent state"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To define a ``DynamicMap`` in counter mode:\n",
    "\n",
    "* Leave one or more dimensions *unbounded* (as in open mode)\n",
    "* Supply a callable (as in bounded mode) that accepts *one* argument\n",
    "\n",
    "This callable should act in the same way as the generators of open mode, except the output is controlled by the single counter argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ls = np.linspace(0, 10, 200)\n",
    "xx, yy = np.meshgrid(ls, ls)\n",
    "\n",
    "def cells_counter(t):\n",
    "    while True:\n",
    "        arr = np.sin(xx+t)*np.cos(yy+t)\n",
    "        return hv.Image(arr)\n",
    "\n",
    "def cells_noisy_counter(t):\n",
    "    while True:\n",
    "        arr = np.sin(xx+t)*np.cos(yy+t)\n",
    "        return hv.Image(arr + 0.2*np.random.rand(200,200))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now if we supply these functions instead of generators, **A** and **B** will correctly be in phase:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "hv.DynamicMap(cells_counter, kdims=['time']) + hv.DynamicMap(cells_noisy_counter, kdims=['time'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unfortunately, an integer counter is often too simple to describe simulation time, which may be a float with real-world units. To address this, we can simply return the actual key values we want along the time dimension, just as was demonstrated in open mode using generators:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "ls = np.linspace(0, 10, 200)\n",
    "xx, yy = np.meshgrid(ls, ls)\n",
    "\n",
    "# Example of a global simulation time\n",
    "# typical in many applications\n",
    "t = 0 \n",
    "        \n",
    "def cells_counter_kv(c):\n",
    "    global t\n",
    "    t = 0.1 * c\n",
    "    while True:\n",
    "        arr = np.sin(xx+t)*np.cos(yy+t)\n",
    "        return (t, hv.Image(arr))\n",
    "\n",
    "def cells_noisy_counter_kv(c):\n",
    "    global t\n",
    "    t = 0.1 * c\n",
    "    while True:\n",
    "        arr = np.sin(xx+t)*np.cos(yy+t)\n",
    "        return (t, hv.Image(arr + 0.2*np.random.rand(200,200)))\n",
    "    \n",
    "hv.DynamicMap(cells_counter_kv, kdims=['time']) + hv.DynamicMap(cells_noisy_counter_kv, kdims=['time'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "print(\"The global simulation time is now t=%f\" % t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ensuring that the HoloViews counter maps to a suitable simulation time is the responsibility of the user. However, once a consistent scheme is configured, the callable in each ``DynamicMap`` can specify the desired simulation time. If the requested simulation time is the same as the current simulation time, nothing needs to happen. Otherwise, the simulator can be run forward by the requested amount.  In this way, HoloViews can provide a rich graphical interface for controlling and visualizing an external simulator, with very little code required."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Slicing in open and counter mode"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Slicing open and counter mode ``DynamicMaps`` has the exact same semantics as normal ``HoloMap`` slicing, except now the ``.data`` attribute corresponds to the cache. For instance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def sine_kv_gen(phase=0, freq=0.5):\n",
    "    while True:\n",
    "        yield (phase, hv.Image(np.sin(phase + (freq*x**2+freq*y**2))))\n",
    "        phase+=0.2\n",
    "        \n",
    "dmap = hv.DynamicMap(sine_kv_gen(), kdims=['phase'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's fill the cache with some elements:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "for i in range(21):\n",
    "    dmap.next()\n",
    "    \n",
    "print(\"Min key value in cache:%s\\nMax key value in cache:%s\" % (min(dmap.keys()), max(dmap.keys())))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "sliced = dmap[1:3.1]\n",
    "print(\"Min key value in cache:%s\\nMax key value in cache:%s\" % (min(sliced.keys()), max(sliced.keys())))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## DynamicMaps and normalization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By default, a ``HoloMap`` normalizes the display of elements according the minimum and maximum values found across the ``HoloMap``. This automatic behavior is not possible in a ``DynamicMap``, where arbitrary new elements are being generated on the fly. Consider the following examples where the arrays contained within the returned ``Image`` objects are scaled with time:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%opts Image {+axiswise}\n",
    "ls = np.linspace(0, 10, 200)\n",
    "xx, yy = np.meshgrid(ls, ls)\n",
    "\n",
    "def cells(vrange=False):\n",
    "    \"The range is set on the value dimension when vrange is True \"\n",
    "    time = time_gen()\n",
    "    while True:\n",
    "        t = next(time)\n",
    "        arr = t*np.sin(xx+t)*np.cos(yy+t)\n",
    "        vdims=[hv.Dimension('Intensity', range=(0,10))] if vrange else ['Intensity']\n",
    "        yield hv.Image(arr, vdims=vdims)\n",
    "\n",
    "hv.DynamicMap(cells(vrange=False), kdims=['time']) + hv.DynamicMap(cells(vrange=True), kdims=['time'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we use ``+axiswise`` to see the behavior of the two cases independently. We see in **A** that when ``vrange=False`` and no range is set on the value dimension, no automatic normalization occurs (unlike a ``HoloMap``). In **B** we see that normalization is applied, but only when the value dimension range has been specified. \n",
    "\n",
    "In other words, ``DynamicMaps`` cannot support automatic normalization across their elements, but do support the same explicit normalization behavior as ``HoloMaps``. Values that are generated outside this range are simply clipped according the usual semantics of explicit value dimension ranges. \n",
    "\n",
    "Note that we can always have the option of casting a ``DynamicMap`` to a ``HoloMap`` in order to automatically normalize across the cached values, without needing explicit value dimension ranges."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using DynamicMaps in your code\n",
    "\n",
    "As you can see, ``DynamicMaps`` let you use HoloViews with a very wide range of dynamic data formats and sources, making it simple to visualize ongoing processes or very large data spaces."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
{'fraction_non_alphanumeric': 0.15643864275284938, 'fraction_numerical': 0.00598208642970776, 'mean_word_length': 3.032647860093154, 'pattern_counts': {'":': 1082, '<': 39, '<?xml version=': 0, '>': 41, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7126216', 'n_tokens_mistral': 29980, 'n_tokens_neox': 29215, 'n_words': 9753}