1. A method for providing trajectory information about a video scene, the method comprising: obtaining video scene metadata that includes information regarding trajectories for a plurality of objects moving through the video scene; clustering each of the plurality of trajectories into trajectory groups according to their relative distance from each other; extracting statistical features about each trajectory group; classifying each trajectory in the plurality of trajectories as a normal trajectory or an abnormal trajectory based at least in part on the clustering; extracting semantic data about each trajectory group or each normal trajectory; and annotating each trajectory with the semantic data.