Contextual Squeeze-and-Excitation for Efficient Few-Shot Image Classification
Recent years have seen a growth in user-centric applications that require
effective knowledge transfer across tasks in the low-data regime. An example is
personalization, where a pretrained system is adapted by learning on small
amounts of labeled data belonging to a specific user. This setting requires
high accuracy under low computational complexity, therefore the Pareto frontier
of accuracy vs.~adaptation cost plays a crucial role. In this paper we push
this Pareto frontier in the few-shot image classification setting with two key
contributions: (i) a new adaptive block called Contextual
Squeeze-and-Excitation (CaSE) that adjusts a pretrained neural network on a new
task to significantly improve performance with a single forward pass of the
user data (context), and (ii) a hybrid training protocol based on
Coordinate-Descent called UpperCaSE that exploits meta-trained CaSE blocks and
fine-tuning routines for efficient adaptation. UpperCaSE achieves a new
state-of-the-art accuracy relative to meta-learners on the 26 datasets of
VTAB+MD and on a challenging real-world personalization benchmark (ORBIT),
narrowing the gap with leading fine-tuning methods with the benefit of orders
of magnitude lower adaptation cost.