1. An image processing apparatus comprising: a moving image input unit configured to input a moving image; a storage unit configured to store object likelihood information in association with a corresponding position in a frame in the moving image; a setting unit configured to sequentially set a plurality of collation patterns in a frame with respective positions based on the object likelihood information stored in the storage unit; and a determination unit configured to derive object reliability information for each of the collation patterns, determine whether each of the collation patterns represents an object based on the derived object reliability information, and update the object likelihood information stored in the storage unit in association with a position of each of the collation patterns.