1. An image recognition device comprising: an image pickup device that picks up an image through a lens; a focus detector that detects a focusing state of a plurality of focus detection areas that are set in an imagefield; a first determination device that determines a first area satisfying a predetermined condition from the plurality of focus detection areas based on the focusing states of the plurality of focus detection areas; a recognizer that recognizes a main object in an image picked up by the image pickup device by using a reference image; a second determination device that determines a second area corresponding to a position of the main object based on a result of recognition by the recognizer; a judgment device that judges whether the first area is out of the second area or not; and a controller that executes at least one of resetting of the reference image and retrying recognition of the position corresponding to the reference image when the judgment device judges that the first area is out of the second area.