1. A system for recognizing a user's gesture for carrying out an operation of a vehicle, comprising: a camera, which is installed within the vehicle, and is configured to generate an object image; and a gesture recognizer configured to detect a user's hand from the object image, set a region of interest (ROI) with reference to the user's hand, and recognize the user's gesture by tracking the user's hand only within the ROI, wherein the ROI is varied depending on a moving direction and a moving speed of the user's hand.