1. A method of fine-grained motion compensated prediction for boundary pixels in a video coding system, the method comprising: determining one or more neighboring coding units (CUs) associated with a current coding unit (CU), wherein each of said one or more neighboring CUs is associated with a neighboring motion vector (MV); performing motion-compensated prediction using the neighboring MV for each of said one or more neighboring CUs to derive pre-generated predictors, wherein the pre-generated predictors correspond to one or more boundary lines or columns in a boundary region of the current CU; storing the pre-generated predictors associated with said one or more neighboring CUs; receiving input data associated with the current CU having a current MV; generating a first predictor for a current boundary pixel in the boundary region by applying motion compensation based on the current MV; generating a current boundary pixel predictor for the current boundary pixel using a weighted sum of the first predictor and one or more corresponding pre-generated predictors according to weighting factors; and applying encoding or decoding to the current CU using prediction data including the current boundary pixel predictor.