1. An image processing apparatus, comprising: an image separator to separate a first frame image and a second frame image from an encoded image; a mode selector to select an output mode; a 2D output unit to output a 2D image by decoding the first frame image when the selected output mode is a 2D output mode; a stereo output unit to output a stereo image by decoding the second frame image when the selected output mode is a stereo output mode; and a multi-view output unit to output a multi-view image by decoding the first frame image and the second frame image and by generating at least one new viewpoint image when the selected output mode is a multi-view output mode, wherein the first frame image is configured using an original central image and the second frame image is configured using a processed left image and a processed right image obtained by processing an original left image and an original right image.