1. A focusing apparatus comprising: a detection unit configured to detect an object from a captured image; a setting unit configured to set an auto-focus area in the captured image at an area corresponding to the detected object; a focus detection unit configured to acquire an auto-focus evaluation value, which indicates a state of focus of an imaging optical system, based on an output signal from the auto-focus area; a determination unit configured to determine a direction to move the imaging optical system to focus on the detected object based on the auto-focus evaluation value; and a focusing unit configured to, when a plurality of objects is detected by the detection unit, combine auto-focus evaluation values obtained from a plurality of auto-focus areas except for an auto-focus area where the auto-focus evaluation value is increased by moving the imaging optical system in a direction different from a direction determined by the determination unit, and to execute focusing by moving the imaging optical system based on the combined auto-focus evaluation values.