1. A network construction system having a layered architecture and utilizing surveillance video stored in files on a computer, where the surveillance video is from one or more image sensors, comprising: a first layer that captures image data in the files and identifies actors and events in the image data and provides identifications for the identified actors; a second layer that indicates which actors in the image data are involved in which events in the image data; and a third layer that: infers links between the actors in the image data from the second layer; and assigns a weighted frequency to at least some of the links between the actors in the image data, where the weighted frequency is dependent on the events in the image data in which the actors in the image data are involved; and constructs a network of linked actors in the image data based on the inferred links between actors in the image data; and wherein: a first identification and a second identification are provided for identification of one or more of the identified actors in the image data; and the weighted frequency indicates a strength of a link between the actors in the image data.