1. An image processing apparatus, comprising: a first calculator configured to calculate a view transform vector of a color camera between a first frame and a second frame by comparing corresponding points between a color image of the first frame and a color image of the second frame; a second calculator configured to calculate a view transform vector of a depth camera between the first frame and the second frame by comparing corresponding points between a depth image of the first frame and a depth image of the second frame; and a third calculator configured to calculate a view transform vector between the color camera and the depth camera using the view transform vector of the color camera and the view transform vector of the depth camera, wherein the first calculator is configured to extract a plurality of feature points from each of the color image of the first frame and the color image of the second frame, determine the corresponding points between the color image of the first frame and the color image of the second frame among the extracted plurality of feature points, and calculate the view transform vector of the color camera between the first frame and the second frame using a displacement between the corresponding points, and wherein the second calculator is configured to extract a plurality of feature points from each of the depth image of the first frame and the depth image of the second image, determine the corresponding points between the depth image of the first frame and the depth image of the second frame among the extracted plurality of feature points, and calculate the view transform vector of the depth camera between the first frame and the second frame using a displacement between the corresponding points.