We introduce a nearest-neighbor-field (NNF) based diversity measure. The NNF is computed by searching for each \((3,3,3)\)  spatio-temporal patch in the generated video, its nearest neighbour patch in the real video.
Simple generated videos (e.g. a translated or shuffled version of the input), will have a rather constant NNF, while more complex generated videos will have complex NNFs.
An NNF of a single frame can be viewed as a color map by converting the spatio-temporal direction to the nearest neighbour using a color wheel [1]}.
In fig:nnfplot we show how such an NNF color map looks in a frame generated by VGPNN [2]}, compared to a frame generated by our framework. The former is very simple, and corresponds to copying large chunks from a single input frame, whereas ours is more complex, which corresponds to a diverse generation from the input video (see full videos in the project page).
