1. A video encoding apparatus comprising: a first motion prediction unit generating a first prediction residue image of an input image on a basis of a first motion prediction result of the input image; an image information detection unit setting a reference color component among color components of an R-G-B image, determining whether the input image is a Y-Cb-Cr image or the R-G-B image, and determining whether a color component of the input image is the reference color component; and a second motion prediction unit performing motion prediction of the first prediction residue image and generating a second prediction residue image on a basis of the reference color component if the input image is the R-G-B image and if the color component of the input image is not the reference color component.