Deep Learning for Free-Hand Sketch: A Survey <s> Sketch-video retrieval <s> We present a novel Content Based Video Retrieval (CBVR) system, driven by free-hand sketch queries depicting both objects and their movement (via dynamic cues; streak-lines and arrows). Our main contribution is a probabilistic model of video clips (based on Linear Dynamical Systems), leading to an algorithm for matching descriptions of sketched objects to video. We demonstrate our model fitting to clips under static and moving camera conditions, exhibiting linear and oscillatory motion. We evaluate retrieval on two real video data sets, and on a video data set exhibiting controlled variation in shape, color, motion and clutter. <s> BIB001 </s> Deep Learning for Free-Hand Sketch: A Survey <s> Sketch-video retrieval <s> Crowd video retrieval is an important problem in surveillance video management in the era of big data, e.g., video indexing and browsing. In this paper, we address this issue from the motion-level perspective by using hand-drawn sketches as queries. Motion sketch based crowd video retrieval naturally suffers from challenges in motion-level video indexing and sketch representation. We tackle them by leveraging the motion structure coding algorithm to extract robust structure-preserved motion descriptors. For video indexing, we use motion decomposition to separate the sub-motion vector fields with typical patterns from a set of optical flows. Then, the motion-level descriptors of the vector fields are computed and stored in the index database. To represent sketch queries, we propose a sketch vectorization algorithm followed by motion structure coding. In the retrieval stage, given a new query, the retrieval function learned by the Ranking SVM algorithm predicts the ranking score of each motion pattern in the index database. Extensive experiments are conducted on the publicly available crowd datasets, which demonstrate the robustness and effectiveness of the proposed sketch based crowd video retrieval system. <s> BIB002 </s> Deep Learning for Free-Hand Sketch: A Survey <s> Sketch-video retrieval <s> Crowd video retrieval with desired motion flow segmentation is an important problem in surveillance video management, e.g., video indexing and browsing, especially in the age of big data. In this paper, we address this issue from the motion-level perspective by using hand-drawn sketches as queries. Motion sketch based crowd video retrieval naturally suffers from challenges in crowd motion representation and similarity measurement. To tackle them, we propose to (1) leverage the motion structure coding algorithm for motion-level video indexing and hand-drawn sketch representation and (2) exploit distance metric fusion strategy incorporated with Ranking SVM for measuring the relevant degree between a sketch query and crowd videos. Specifically, for video indexing, motion decomposition is utilized to separate sub-motion vector fields with typical patterns from a set of optical flows. Then, the motion-level descriptors of the vector fields are computed and stored in an index database. To represent motion sketches, we propose a mechanism by vectorizing the sketches followed by motion structure coding. In the retrieval stage, we first compute the pairwise distance with different metrics between a new sketch query and crowd videos, and then stack them into a feature vector as the input of the Ranking SVM algorithm. Finally, we use the learned retrieval model to predict the ranking score of each crowd video in the database. Experimental results on the publicly available crowd datasets show the robustness and effectiveness of the proposed sketch based crowd video retrieval system. <s> BIB003
Sketch based video retrieval (SBVR) BIB001 has been studied before the boom of deep learning. SBVR is also an extremely Fig. 16 . Sketch samples randomly selected from sketch based video retrieval dataset TSF BIB001 . The left sketch depicts that two persons are approaching each other. The right one depicts that a person is gliding up the hillside. challenging task due to the huge domain gap between free-hand sketch and video. In SBVR application scenarios, serving as the query, the advantage of sketch is that humans can use lines or arrow vectors to describe the moving or other dynamic scenes, as illustrated in Figure 16 . This is to say that sketch can not only depict static object/scene but also motion information. The task-specific challenge of SBVR is that both of apprearance matching and motion matching across free-hand sketch and video should be considered. The apprearance matching and motion matching need to consider the static and dynamic information, respectively. Furthermore, motion sketch based crowd video retrieval BIB002 , BIB003 has been studied recently, which is useful for the video surveillance analysis.