1. A system for recognizing a user's gesture for carrying out an operation of a vehicle, comprising: a camera, which is installed within the vehicle, and is configured to generate an object image; and a gesture recognizer configured to detect a user's hand from the object image, set a region of interest (ROI) with reference to the user's hand, and recognize the user's gesture by tracking the user's hand only within the ROI, wherein the ROI is varied depending on a moving direction and a moving speed of the user's hand wherein the gesture recognizer comprises an ROI setting unit configured set the ROI with reference to the user's hand, wherein the ROI setting unit sets the ROI as a first region, which is formed with reference to the user's hand, when the moving speed of the user's hand is less than a predetermined speed, wherein the ROI setting unit resets the ROI as a second region when the moving speed of the user's hand is greater than or equal to the predetermined speed after the ROI is set as the first region, a size of the second region is smaller than a size of the first region, and the second region is expanded in the moving direction of the user's hand and reduced in a direction that is perpendicular to the moving direction of the user's hand based on the area of the first region.