1. A system for processing data describing a scene depicted in a sequence of video frames, the system comprising: a processor; and a memory communicatively coupled to the processor, the memory comprising computer-readable instructions that, when executed by the processor, cause the system to: identify one or more objects detected in the scene; receive input data associated with the one or more identified objects; evaluate the received input data to identify one or more primitive events, wherein for a first one of the primitive events, a semantic value is provided describing a behavior engaged in by a first one of the objects depicted in the sequence of video frames and wherein the first one of the primitive events has an assigned primitive event symbol; generate, for the first object, a primitive event symbol stream which includes the primitive event symbol corresponding to the first one of the primitive events identified for the first object; and output the primitive event symbol stream.