1. A method of fine-grained motion compensated prediction for boundary pixels in a video coding system, the method comprising: determining one or more neighboring coding units (CUs) associated with a current coding unit (CU), wherein each of said one or more neighboring CUs is associated with a neighboring motion vector (MV); performing motion-compensated prediction using the neighboring MV for each of said one or more neighboring CUs to derive pre-generated predictors, wherein the pre-generated predictors correspond to one or more boundary lines or columns in a boundary region of the current CU; storing the pre-generated predictors; receiving input data associated with the current CU having a current MV; generating a first predictor for a current boundary pixel in the boundary region by applying motion compensation based on the current MV, wherein pixels at boundaries of the current CU utilize the current MV and at least one MV from at least one of: an upper side MV and a left side MV to form a weighted sum of motion prediction when performing motion compensation; generating a current boundary pixel predictor for the current boundary pixel using a weighted sum of the first predictor and one or more corresponding pre-generated predictors according to weighting factors; and applying encoding or decoding to the current CU using prediction data including the current boundary pixel predictor, wherein said pre-generated predictors are at a bottom side or a right side of each of said one or more neighboring CUs on a smallest CU (SCU) basis, and wherein said pre-generated predictors are stored on a SCU basis.