Empirical Bayes Transductive Meta Learning with Synthetic Gradients
We propose a meta learning approach that learns from multiple tasks in a transductive setting, by leveraging unlabeled information in the query set to learn a more powerful meta model. To develop our framework we revisit the empirical Bayes formulation for multi task learning. The evidence lower bound of the marginal log likelihood of empirical Bayes decomposes as a sum of local KL divergences between the variational posterior and the true posterior of each task. We derive a novel amortized variational inference that couples all the variational posteriors into a meta model, which consists of a synthetic gradient network and an initialization network. The combination of local KL divergences and synthetic gradient network allows for backpropagating information from unlabeled data, thereby enabling transduction. Our results on the Mini ImageNet and CIFAR FS benchmarks for episodic few shot classification significantly outperform previous state of the art methods.
ICLR