1. A control apparatus for controlling a driver driving an optical element of an image pickup device to adjust a focus position of the image pickup device in the capturing of an image of a subject, the control apparatus comprising: local peak comparing means for comparing a local peak of an assessment value, as a parameter for adjusting the focus position, with a local peak of an integrated luminance value, the assessment value assessing an out-of-focus level of the captured image of the subject on a per field image basis, and the integrated luminance value being obtained by integrating a luminance value of the captured image of the subject on a per field image basis; and mode selecting means for selecting, from among a plurality of prepared modes, a mode searching for a peak over the entire range of the assessment value based on the comparison result provided by the local peak comparing means.