1. A multi-view image display apparatus comprising: a tracker configured to track a position of a user's face according to a movement of the user; and a processor configured to compensate for crosstalk with respect to multi-view image comprising a plurality of views using a crosstalk matrix defining the crosstalk that occurs between different views, wherein the processor determines at least two views from among the plurality of views to be provided to the user's left and right eyes, based on the tracked position of the user's face and compensates for the crosstalk by applying a weight matrix that applies a predetermined weight to the at least two views to an inverse matrix of the crosstalk matrix, and wherein the predetermined weight applied to the at least two views is higher than a weight which is applied to at least one remaining view among the plurality of views; and a display configured to display the compensated multi-view image.