Fisher vector has been widely used in many multimedia retrieval and visual recognition applications with good performance. However, the computation complexity prevents its usage in real-time video monitoring. In this work, we proposed and implemented GPU-FV, a fast Fisher vector extraction method with the help of modern GPUs. The challenge of implementing Fisher vector on GPUs lies in the data dependency in feature extraction and expensive memory access in Fisher vector computing. To handle these challenges, we carefully designed GPU-FV in a way that utilizes the computing power of GPU as much as possible, and applied optimizations such as loop tiling to boost the performance. GPUFV is about 12 times faster than the CPU version, and 50 faster than a non-optimized GPU implementation. For standard video input (320*240), GPU-FV can process each frame within 34ms on a model GPU. Our experiments show that GPU-FV obtains a similar recognition accuracy as traditional FV on VOC 2007 and Caltech 256 image sets. We also applied GPU-FV for realtime video monitoring tasks and found that GPU-FV outperforms a number of previous works. Especially, when the number of training examples are small, GPU-FV outperforms the recent popular deep CNN features borrowed from ImageNet.
SURF is an optimized robust feature extraction system @cite_16 . Cornelis and Van Gool @cite_0 implemented SURF on the GPU (Graphics Processing Unit) and obtained an order of magnitude speedup compared to a CPU implementation. Extracting SIFT descriptors on GPU has been studied by other researchers @cite_2 @cite_14 . Recent efforts were also made on accelerating Dense SIFT computation @cite_12 . SVM model training, have been independently studied on the GPU before @cite_36 . To address the bottlenecks in accurate visual categorization systems, Sande et.al @cite_30 did a detailed analysis, and proposed two GPU-accelerated algorithms, GPU vector quantization and GPU kernel value precomputation, which results in a substantial acceleration of the complete visual categorization pipeline. However, their method does not involve Fisher vector encoding.
@cite_30 @cite_14 @cite_36 @cite_0 @cite_2 @cite_16 @cite_12
2083822463 1511559872 2057460344 1573085410
This paper describes novel implementations of the KLT feature tracking and SIFT feature extraction algorithms that run on the graphics processing unit (GPU) and is suitable for video analysis in real-time vision systems. While significant acceleration over standard CPU implementations is obtained by exploiting parallelism provided by modern programmable graphics hardware, the CPU is freed up to run other computations in parallel. Our GPU-based KLT implementation tracks about a thousand features in real-time at 30 Hz on 1,024 × 768 resolution video which is a 20 times improvement over the CPU. The GPU-based SIFT implementation extracts about 800 features from 640 × 480 video at 10 Hz which is approximately 10 times faster than an optimized CPU implementation. Many computer vision tasks require interest point detection and description, such as real-time visual navigation. We present a GPU implementation of the recently proposed SpeededUp Robust Feature extractor [1], currently the state of the art for this task. Robust feature descriptors can give vast improvements in the quality and speed of subsequent steps, but require intensive computation up front that is well-suited to inexpensive graphics hardware. We describe the algorithm’s translation to the GPU in detail, with several novel optimizations, including a new method of computing multi-dimensional parallel prefix sums. It operates at over 30 Hz at HD resolutions with thousands of features and in excess of 70 Hz at SD resolutions. Scale Invariance Feature Transform (SIFT) is quite suitable for image matching because of its invariance to image scaling, rotation and slight changes in illumination or viewpoint. However, due to high computation complexity it's technically challenging to deploy SIFT in real time application situations. To address this problem, we propose CLSIFT, an OpenCL based highly speeded up and performance portable SIFT solution. Important optimization techniques employed in CLSIFT such as: (1) For less global memory traffic, independent logical functions are merged into the same kernel to reuse data.(2) loop buffers are introduced in for data and intermediate results reusing.(3)Task queue used to schedule threads in the same branch to remove branch divergences. (4) Data partition is based on the statics patterns for workload balance among workgroups. (5) Overlap of CPU time and better parallel strategies are used too. With all mentioned efforts, CLSIFT processes lena. jpg at 74.2 FPS and 43.4FPS respectively on NVidia and AMD GPUS, much higher than CPU's nearly 10 FPS and the known fastest SIFTGPU's 39.8 FPS and 13FPS. Moreover in a quantitative comparison approach we analyze those successful strategies beating SIFTGPU, a famous existing GPU implementation. Additionally, we observe and conclude that NVidia GPU achieves better occupancy and performance due to some factors. Finally, we summarize some techniques and empirical guiding principles that may be shared by other applications on GPU. With the addition of free programmable components to modern graphics hardware, graphics processing units (GPUs) become increasingly interesting for general purpose computations, especially due to utilizing parallel buffer processing. In this paper we present methods and techniques that take advantage of modern graphics hardware for real-time tracking and recognition of feature-points. The focus lies on the generation of feature vectors from input images in the various stages. For the generation of feature-vectors the Scale Invariant Feature Transform (SIFT) method [Low04a] is used due to its high stability against rotation, scale and lighting condition changes of the processed images. We present results of the various stages for feature vector generation of our GPU implementation and compare it to the CPU version of the SIFT algorithm. The approach works well on Geforce6 series graphics board and above and takes advantage of new hardware features, e.g. dynamic branching and multiple render targets (MRT) in the fragment processor [KF05]. With the presented methods feature-tracking with real time frame rates can be achieved on the GPU and meanwhile the CPU can be used for other tasks.