On-Line, Off-Line Compatible Image Retrieval for Surveillance Video Based on the Trajectory Information of Moving Objects
We propose an image processing system which searches the moving human and vehicles from the long-term surveillance video. Image retrieval is done by comparing a series of queries like passing a certain area, moving direction, duration time and so on. In this kind of systems, not only on-line detection from real-time video by pre-defined query but also quick re-search going back to the past when the user changed the query, and post-retrieval by interactive query change are required. The main subjects are the size of the meta-data and the simpleness of the matching functions. In proposed method, meta-data is described in a small and fixed length however complicated the trajectory of moving object is. Moreover, the matching function is realized by a simple comparator, that enables fast image retrieval however complicated the series of the retrieval queries is. Fig. 1 shows the raw image data and the structure of the meta-data which is composed of the hierarchical configuration of 1st meta-data and 2nd meta-data. 1st meta-data is the frame based region information which describes the gravity point, area size, width and height of the each detected objects. 2nd meta-data is the object based trajectory information which describes the existence of the motion of quantized orientation and stay duration in each divided small block in the image. In most cases, only 2nd meta-data is used for image retrieval. When additional retrieval query, i.e. area size is required, a group of related 1st meta-data is referred through the link data. To test the performance of the system, we used a long-term parking lot monitoring sequence. Fig. 2 shows the image retrieval query for experiment. In this situation, Line Passing and Duration Time are set. Fig. 3 shows the result of on-line, off-line simultaneous image retrieval. About 34 hours after the on-line image processing started when the total number of moving objects reached to 10,000, we changed the retrieval query and restarted matching process and confirmed that the new result is listed up in less than 0.2 seconds.