1. A focus adjustment apparatus comprising: a face detection unit that detects a position at which at least a face of a person exists from an image signal obtained by photoelectrically converting an object image formed by a photographing optical system; a focus detection unit that detects a plurality of in-focus positions of the photographing optical system in a plurality of focus detection areas in which it is estimated that a body of the person, which corresponds to the face of the person detected by said face detection unit, is present; and a control unit that controls movement of the photographing optical system based on at least one of the in-focus positions obtained in the focus detection area, wherein said control unit controls to move the photographing optical system to the in-focus position corresponding to a farther distance in the plurality of in-focus positions detected by said focus detection unit.