We introduce modifications to state-of-the-art approaches to aggregating local video descriptors by using attention mechanisms and function approximations. Rather than using ensembles of existing architectures, we provide an insight on creating new architectures. We demonstrate our solutions in the “The 2nd YouTube-8M Video Understanding Challenge”, by using frame-level video and audio descriptors. We obtain testing accuracy similar to the state of the art, while meeting budget constraints, and touch upon strategies to improve the state of the art. Model implementations are available in https: github.com pomonam LearnablePoolingMethods.
What is newly introduced to VLAD is the flattened vector of @math (in this equation @math is the flattening operation), this provides the second order approximation of our hypothetical function, derived from a Taylor expansion @cite_17 . In addition, objective functions are provided to help regularize the learning of weights, so as to ensure a valid function approximation, this is further discussed in Section .
@cite_17
1913833407
The objective of this paper is to design an embedding method mapping local features describing image (e.g. SIFT) to a higher dimensional representation used for image retrieval problem.