1. A method for providing trajectory information about a video scene, the method comprising: obtaining video scene metadata that includes information regarding trajectories for a plurality of objects moving through the video scene; normalizing each of the trajectories into a unified length; clustering each of the plurality of the normalized trajectories into trajectory groups according to relative distances from each other; extracting statistical features about each trajectory group; classifying each trajectory in the plurality of trajectories as a normal trajectory or an abnormal trajectory based at least in part on the clustering; extracting semantic data about each trajectory group or each normal trajectory; and annotating each trajectory with the semantic data; wherein the clustering comprises a plurality of clustering iterations using a plurality of different distance measures, wherein the plurality of different distance measures determine at least two different distance metrics using different mathematical models.