1. A moving object recognition apparatus for recognizing and tracking a moving object based on a moving image photographed by a camera, the moving object recognition apparatus comprising: a feature image extraction unit that extracts feature image portions for enabling the moving object to be determined from the moving image; a position detection unit that detects position information of the moving object in time series from the moving image, the position information based on which the feature image portion is extracted; a determination unit that determines identity of the feature image portions extracted from the moving object; and a track information retention unit that retains the time-series position information linked to the moving object having the feature image portion having the identity based on the determination.