It is known that many issues remain unsolved for optical flow estimation in a scene due to effects such as ambient disturbance, illumination change, and occlusion. A perfect algorithm that can solve all of these problems does not exist, and errors are inevitable in certain cases. For example, it is difficult for optical flow algorithms to predict the pixels' movements near the motion boundary due to occlusion. Fig. REF  shows a person and a horse in the foreground moving to the right (Supplementary Video 1). Since the camera is moving along with the person and the horse, the foreground is roughly static, but the background is moving to the left. The bottom row shows the optical flow results obtained using DeepFlow [1]}. The black regions in the bottom-middle indicate that the estimation near the boundary of the person and the horse has very low confidence. This is because these regions in the background are occluded by the person and the horse in the next frame, and the pixels in these regions do not have corresponding pixels in the next frame. Clearly, this situation worsens if the speed of motion is higher or the frame rate (frames per second) is lower.
<FIGURE>