Recent few-shot learning algorithms have enabled models to quickly adapt to new tasks based on only a few training samples. Previous few-shot learning works have mainly focused on classification and reinforcement learning. In this paper, we propose a few-shot meta-learning system that focuses exclusively on regression tasks. Our model is based on the idea that the degree of freedom of the unknown function can be significantly reduced if it is represented as a linear combination of a set of sparsifying basis functions. This enables a few labeled samples to approximate the function. We design a Basis Function Learner network to encode basis functions for a task distribution, and a Weights Generator network to generate the weight vector for a novel task. We show that our model outperforms the current state of the art meta-learning methods in various regression tasks. Regression deals with the problem of learning a model relating a set of inputs to a set of outputs. The learned model can be thought as function y = F (x) that gives a prediction y ∈ R dy given input x ∈ R dx where d y and d x are dimensions of the output and input respectively. Typically, a regression model is trained on a large number of data points to be able to provide accurate predictions for new inputs. Recently, there have been a surge in popularity on few-shot learning methods (Vinyals et al., 2016; Koch et al., 2015; Gidaris & Komodakis, 2018) . Few-shot learning methods require only a few examples from each task to be able to quickly adapt and perform well on a new task. These few-shot learning methods in essence are learning to learn i.e. the model learns to quickly adapt itself to new tasks rather than just learning to give the correct prediction for a particular input sample. In this work, we propose a few shot learning model that targets few-shot regression tasks. Our model takes inspiration from the idea that the degree of freedom of F (x) can be significantly reduced when it is modeled a linear combination of sparsifying basis functions. Thus, with a few samples, we can estimate F (x). The two primary components of our model are (i) the Basis Function Learner network which encodes the basis functions for the distribution of tasks, and (ii) the Weights Generator network which produces the appropriate weights given a few labelled samples. We evaluate our model on the sinusoidal regression tasks and compare the performance to several meta-learning algorithms. We also evaluate our model on other regression tasks, namely the 1D heat equation tasks modeled by partial differential equations and the 2D Gaussian distribution tasks. Furthermore, we evaluate our model on image completion as a 2D regression problem on the MNIST and CelebA data-sets, using only a small subset of known pixel values. To summarize, our contributions for this paper are: • We propose to address few shot regression by linear combination of a set of sparsifying basis functions. • We propose to learn these (continuous) sparsifying basis functions from data. Traditionally, basis functions are hand-crafted (e.g. Fourier basis). • We perform experiments to evaluate our approach using sinusoidal, heat equation, 2D Gaussian tasks and MNIST/CelebA image completion tasks. An overview of our model as in meta-training. Our system learns the basis functions Φ that can result in sparse representation for any task drawn from a certain task distribution. The basis functions are encoded in the Basis Function Learner network. The system produces predictions for a regression task by generating a weight vector, w for a novel task, using the Weights Generator network. The prediction is obtained by taking a dot-product between the weight vector and learned basis functions.
We propose a method of doing few-shot regression by learning a set of basis functions to represent the function distribution.
{"src": "scitldr-AIC_train", "paper_id": "BJxDNxSFDH"}