On the Eigenvalues of Global Covariance Pooling for Fine-grained Visual Recognition
The Fine-Grained Visual Categorization (FGVC) is challenging because the
subtle inter-class variations are difficult to be captured. One notable
research line uses the Global Covariance Pooling (GCP) layer to learn powerful
representations with second-order statistics, which can effectively model
inter-class differences. In our previous conference paper, we show that
truncating small eigenvalues of the GCP covariance can attain smoother gradient
and improve the performance on large-scale benchmarks. However, on fine-grained
datasets, truncating the small eigenvalues would make the model fail to
converge. This observation contradicts the common assumption that the small
eigenvalues merely correspond to the noisy and unimportant information.
Consequently, ignoring them should have little influence on the performance. To
diagnose this peculiar behavior, we propose two attribution methods whose
visualizations demonstrate that the seemingly unimportant small eigenvalues are
crucial as they are in charge of extracting the discriminative class-specific
features. Inspired by this observation, we propose a network branch dedicated
to magnifying the importance of small eigenvalues. Without introducing any
additional parameters, this branch simply amplifies the small eigenvalues and
achieves state-of-the-art performances of GCP methods on three fine-grained
benchmarks. Furthermore, the performance is also competitive against other FGVC
approaches on larger datasets. Code is available at
\href{https://github.com/KingJamesSong/DifferentiableSVD}{https://github.com/KingJamesSong/DifferentiableSVD}.