<jupyter_start><jupyter_text>Making Animations of UMAP Hyper-parametersSometimes one of the best ways to see the effects of hyperparameters is simply to visualise what happens as they change. We can do that in practice with UMAP by simply creating an animation that transitions between embeddings generated with variations of hyperparameters. To do this we'll make use of matplotlib and its animation capabilities. <NAME> has [a great tutorial](https://jakevdp.github.io/blog/2012/08/18/matplotlib-animation-tutorial/) if you want to know more about creating animations with matplotlib. **Note:**This is a self contained example of how to use UMAP and the impact of individual hyper-parameters. To make sure everything works correctly please use `conda`.For install and usage details see [here](https://docs.conda.io/en/latest/miniconda.html)To create animations we need `ffmpeg`. It can be installed with `conda`.If you already have `ffmpeg` installed on your machine and you know what you are doing you do not need conda. It is only used to install `ffmpeg`.=> Remove the next two cells if you are not using `conda`.<jupyter_code>!conda --version
!conda install -c conda-forge ffmpeg -y
!python --version<jupyter_output>Python 3.7.3<jupyter_text>To start we'll need some basic libraries. First ``numpy`` will be needed for basic array manipulation. Since we will be visualising the results we will need ``matplotlib`` and ``seaborn``. Finally we will need ``umap`` for doing the dimension reduction itself.<jupyter_code>!pip install numpy matplotlib seaborn umap-learn<jupyter_output>Requirement already satisfied: numpy in c:\progams\miniconda\envs\tf\lib\site-packages (1.17.1)
Requirement already satisfied: matplotlib in c:\progams\miniconda\envs\tf\lib\site-packages (3.1.1)
Requirement already satisfied: seaborn in c:\progams\miniconda\envs\tf\lib\site-packages (0.9.0)
Requirement already satisfied: umap-learn in c:\progams\miniconda\envs\tf\lib\site-packages (0.3.10)
Requirement already satisfied: kiwisolver>=1.0.1 in c:\progams\miniconda\envs\tf\lib\site-packages (from matplotlib) (1.1.0)
Requirement already satisfied: cycler>=0.10 in c:\progams\miniconda\envs\tf\lib\site-packages (from matplotlib) (0.10.0)
Requirement already satisfied: python-dateutil>=2.1 in c:\progams\miniconda\envs\tf\lib\site-packages (from matplotlib) (2.8.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in c:\progams\miniconda\envs\tf\lib\site-packages (from matplotlib) (2.4.2)
Requirement already satisfied: pandas>=0.15.2 in c:\progams\miniconda\envs\tf\lib\si[...]<jupyter_text>To start let's load everything we'll need<jupyter_code>%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.axes_grid1 import make_axes_locatable
from matplotlib import animation
from IPython.display import HTML
import seaborn as sns
import itertools
sns.set(style='white', rc={'figure.figsize':(14, 12), 'animation.html': 'html5'})
# Ignore UserWarnings
import warnings
warnings.simplefilter('ignore', UserWarning)
from sklearn.datasets import load_digits
from umap import UMAP<jupyter_output><empty_output><jupyter_text>To try this out we'll needs a reasonably small dataset (so embedding runs don't take *too* long since we'll be doing a lot of them). For ease of reproducibility for everyone else I'll use the digits dataset from sklearn. If you want to try other datasets just drop them in here -- COIL20 might be interesting, or you might have your own data.<jupyter_code>digits = load_digits()
data = digits.data
data<jupyter_output><empty_output><jupyter_text>We need to move the points in between the embeddings given by different parameter values. There are potentially fancy ways to do this (Something using rotation and reflection to get an initial alignment might be interesting), but we'll use straighforward linear interpolation between the two embeddings. To do this we'll need a simple function that can turn out intermediate embeddings for the in-between frames of the animation.<jupyter_code>def tween(e1, e2, n_frames=20):
    for i in range(5):
        yield e1
    for i in range(n_frames):
        alpha = i / float(n_frames - 1)
        yield (1 - alpha) * e1 + alpha * e2
    for i in range(5):
        yield(e2)
    return<jupyter_output><empty_output><jupyter_text>Now that we can fill in intermediate frame we just need to generate all the embeddings. We'll create a function that can take an argument and set of parameter values and then generate all the embeddings including the in-between frames.<jupyter_code>def generate_frame_data(data, arg_name='n_neighbors', arg_list=[]):
    result = []
    es = []
    for arg in arg_list:
        kwargs = {arg_name:arg}
        if len(es) > 0:
            es.append(UMAP(init=es[-1], negative_sample_rate=3, **kwargs).fit_transform(data))
        else:
            es.append(UMAP(negative_sample_rate=3, **kwargs).fit_transform(data))
        
    for e1, e2 in zip(es[:-1], es[1:]):
        result.extend(list(tween(e1, e2)))
        
    return result<jupyter_output><empty_output><jupyter_text>Next we just need to create a function to actually generate the animation given a list of embeddings (one for each frame). This is really just a matter of workign through the details of how matplotlib generates animations -- I would refer you again to Jake's tutorial if you are interested in the detailed mechanics of this.<jupyter_code>def create_animation(frame_data, arg_name='n_neighbors', arg_list=[]):
    fig, ax = plt.subplots()
    all_data = np.vstack(frame_data)
    frame_bounds = (all_data[:, 0].min() * 1.1, 
                    all_data[:, 0].max() * 1.1,
                    all_data[:, 1].min() * 1.1, 
                    all_data[:, 1].max() * 1.1)
    ax.set_xlim(frame_bounds[0], frame_bounds[1])
    ax.set_ylim(frame_bounds[2], frame_bounds[3])
    points = ax.scatter(frame_data[0][:, 0], frame_data[0][:, 1], 
                        s=5, c=digits.target, cmap='Spectral', animated=True)
    title = ax.set_title('', fontsize=24)
    ax.set_xticks([])
    ax.set_yticks([])

    cbar = fig.colorbar(
        points,
        cax=make_axes_locatable(ax).append_axes("right", size="5%", pad=0.05),
        orientation="vertical",
        values=np.arange(10),
        boundaries=np.arange(11)-0.5,
        ticks=np.arange(10),
        drawedges=True,
    )
    cbar.ax.yaxis.set_ticklabels(np.arange(10), fontsize=18)

    def init():
        points.set_offsets(frame_data[0])
        arg = arg_list[0]
        arg_str = f'{arg:.3f}' if isinstance(arg, float) else f'{arg}'
        title.set_text(f'UMAP with {arg_name}={arg_str}')
        return (points,)

    def animate(i):
        points.set_offsets(frame_data[i])
        if (i + 15) % 30 == 0:
            arg = arg_list[(i + 15) // 30]
            arg_str = f'{arg:.3f}' if isinstance(arg, float) else f'{arg}'
            title.set_text(f'UMAP with {arg_name}={arg_str}')
        return (points,)

    anim = animation.FuncAnimation(fig, animate, init_func=init, frames=len(frame_data), interval=20, blit=True)
    plt.close()
    return anim<jupyter_output><empty_output><jupyter_text>Finally a little bit of glue to make it all go together.<jupyter_code>def animate_param(data, arg_name='n_neighbors', arg_list=[]):
    frame_data = generate_frame_data(data, arg_name, arg_list)
    return create_animation(frame_data, arg_name, arg_list)<jupyter_output><empty_output><jupyter_text>Now we can create an animation. It will be embedded as an HTML5 video into this notebook.<jupyter_code>animate_param(data, 'n_neighbors', [3, 4, 5, 7, 10, 15, 25, 50, 100, 200])
animate_param(data, 'min_dist', [0.0, 0.01, 0.1, 0.2, 0.4, 0.6, 0.9])
animate_param(data, 'local_connectivity', [0.1, 0.2, 0.5, 1, 2, 5, 10])
animate_param(data, 'set_op_mix_ratio', np.linspace(0.0, 1.0, 10))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1123780565057646, 'fraction_numerical': 0.02495882427467376, 'mean_word_length': 4.162851536952257, 'pattern_counts': {'":': 0, '<': 36, '<?xml version=': 0, '>': 43, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '636518', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 2692, 'n_tokens_neox': 2542, 'n_words': 790, 'path': 'notebooks/AnimatingUMAP.ipynb', 'repo_name': 'stjordanis/umap', 'size': '7902'}