1. A computer-implemented method for analyzing a scene depicted in an input stream of video frames captured by a video camera of a video surveillance system, the method comprising: retrieving a first sequence and a second sequence, each providing an ordered string of labels, wherein each label corresponds to a cluster in an adaptive resonance theory (ART) network, wherein the strings of labels are generated by mapping kinematic data vectors generated for a first foreground object and a second foreground object detected in the input stream of video frames, respectively, to nodes of a self-organizing map (SOM) and clustering the nodes of the SOM using the ART network, and wherein the first sequence and the second sequence correspond to an observed interaction between the first foreground object and the second foreground object; identifying one or more segments in each of the first and second sequences, wherein each segment includes a subsequence of the ordered string of labels in the first and second sequences; determining a probability of observing the interaction between the first foreground object and the second foreground object, relative to a probability distribution generated from an ngram trie, wherein the ngram trie is generated from a plurality of previously observed sequences, each storing an ordered string of labels assigned to clusters in the ART network for objects detected in the input stream of video frames; and upon determining the probability the observed interaction between the first foreground object and the second foreground object falls below a specified threshold, issuing an alert to a user of the video surveillance system.