1. An image processing apparatus for processing moving images each divisible into a plurality of shots, said image processing apparatus comprising: a holding unit configured to hold discrimination models acquired by learning beforehand a first rule and a second rule from a moving image formed by a plurality of known shot groups which are made up of at least one shot each and from which a highlight is to be extracted in accordance with said discrimination models, said first rule governing relevance between said shots, said second rule governing relevance between frames within each of said shots; and an extraction unit configured to extract from a newly input moving image a shot group recognized as said highlight in accordance with said discrimination models held in said holding unit.