1. A computer implemented method for estimating sub-pixel optical flow of a pixel of an input image, the method comprising: receiving the input image and a reference image of the input image, wherein the reference image is one of a subsequent image or a previous image of the input image; for each pixel of the input image: estimating a plurality of candidate pixel-level optical flows of the pixel using the reference image of the input image, wherein the pixel-level optical flow of the pixel represents relative movement of the pixel between the input image and the reference image; selecting a pixel-level optical flow from a plurality of candidate pixel-level optical flows of the pixel associated with the pixel; obtaining a plurality of neighboring pixel-level optical flow vectors of the selected pixel-level optical flow; and estimating the sub-pixel optical flow of the pixel based on the selected pixel-level optical flow and its neighboring pixel-level optical flow vectors by approximating an underlying continuous function for sub-pixel optical flow estimation, wherein approximating the underlying continuous function comprises: calculating a horizontal shift of the pixel-level optical flow using the selected pixel-level optical flow, its horizontal pixel-level optical flow vectors and associated matching scores; and calculating a vertical shift of the pixel-level optical flow using the selected pixel-level optical flow, its two vertical pixel-level optical flow vectors and associated matching scores; wherein the sub-pixel optical flow of the pixel in horizontal direction is sum of estimated pixel-level optical flow in horizontal direction and the calculated horizontal shift, and the sub-pixel optical flow of the pixel in vertical direction is sum of estimated pixel-level optical flow in vertical direction and the calculated vertical shift.