1. An image processing apparatus comprising: a calculating unit configured to calculate correlation values of captured images of a plurality of pairs of viewpoints, using captured images of the viewpoints; and a generating unit configured to generate parallax images of different hierarchical levels of a reference viewpoint based on the correlation values calculated by the calculating unit, the parallax images being a viewpoint parallax image formed with a disparity value of the reference viewpoint and an occlusion parallax image formed with a disparity value of an occlusion region, the reference viewpoint being one of the viewpoints.