The ability to learn new concepts with small amounts of data is a critical aspect of intelligence that has proven challenging for deep learning methods. Meta-learning has emerged as a promising technique for leveraging data from previous tasks to enable efficient learning of new tasks. However, most meta-learning algorithms implicitly require that the meta-training tasks be mutually-exclusive, such that no single model can solve all of the tasks at once. For example, when creating tasks for few-shot image classification, prior work uses a per-task random assignment of image classes to N-way classification labels. If this is not done, the meta-learner can ignore the task training data and learn a single model that performs all of the meta-training tasks zero-shot, but does not adapt effectively to new image classes.   This requirement means that the user must take great care in designing the tasks, for example by shuffling labels or removing task identifying information from the inputs. In some domains, this makes meta-learning entirely inapplicable. In this paper, we address this challenge by designing a meta-regularization objective using information theory that places precedence on data-driven adaptation. This causes the meta-learner to decide what must be learned from the task training data and what should be inferred from the task testing input. By doing so, our algorithm can successfully use data from non-mutually-exclusive tasks to efficiently adapt to novel tasks. We demonstrate its applicability to both contextual and gradient-based meta-learning algorithms, and apply it in practical settings where applying standard meta-learning has been difficult. Our approach substantially outperforms standard meta-learning algorithms in these settings.  The ability to learn new concepts and skills with small amounts of data is a critical aspect of intelligence that many machine learning systems lack. Meta-learning (Schmidhuber, 1987) has emerged as a promising approach for enabling systems to quickly learn new tasks by building upon experience from previous related tasks (Thrun & Pratt, 2012; Koch et al., 2015; Santoro et al., 2016; Ravi & Larochelle, 2016; Finn et al., 2017) . Meta-learning accomplishes this by explicitly optimizing for few-shot generalization across a set of meta-training tasks. The meta-learner is trained such that, after being presented with a small task training set, it can accurately make predictions on test datapoints for that meta-training task. While these methods have shown promising results, current methods require careful design of the meta-training tasks to prevent a subtle form of task overfitting, distinct from standard overfitting in supervised learning. If the task can be accurately inferred from the test input alone, then the task training data can be ignored while still achieving low meta-training loss. In effect, the model will collapse to one that makes zero-shot decisions. This presents an opportunity for overfitting where the meta-learner generalizes on meta-training tasks, but fails to adapt when presented with training data from novel tasks. We call this form of overfitting the memorization problem in meta-learning because the meta-learner memorizes a function that solves all of the meta-training tasks, rather than learning to adapt. Existing meta-learning algorithms implicitly resolve this problem by carefully designing the metatraining tasks such that no single model can solve all tasks zero-shot; we call tasks constructed in this Implementation and examples available here: https://github.com/google-research/ google-research/tree/master/meta_learning_without_memorization. way mutually-exclusive. For example, for N -way classification, each task consists of examples from N randomly sampled classes. The N classes are labeled from 1 to N , and critically, for each task, we randomize the assignment of classes to labels {1, 2, . . . , N } (visualized in Appendix Figure 3 ). This ensures that the task-specific class-to-label assignment cannot be inferred from a test input alone. However, the mutually-exclusive tasks requirement places a substantial burden on the user to cleverly design the meta-training setup (e.g., by shuffling labels or omitting goal information). While shuffling labels provides a reasonable mechanism to force tasks to be mutually-exclusive with standard few-shot image classification datasets such as MiniImageNet (Ravi & Larochelle, 2016) , this solution cannot be applied to all domains where we would like to utilize meta-learning. For example, consider meta-learning a pose predictor that can adapt to different objects: even if N different objects are used for meta-training, a powerful model can simply learn to ignore the training set for each task, and directly learn to predict the pose of each of the N objects. However, such a model would not be able to adapt to new objects at meta-test time. The primary contributions of this work are: 1) to identify and formalize the memorization problem in meta-learning, and 2) to propose an meta-regularizer (MR) using information theory as a general approach for mitigating this problem without placing restrictions on the task distribution. We formally differentiate the meta-learning memorization problem from overfitting problem in conventional supervised learning, and empirically show that naïve applications of standard regularization techniques do not solve the memorization problem in meta-learning. The key insight of our metaregularization approach is that the model acquired when memorizing tasks is more complex than the model that results from task-specific adaptation because the memorization model is a single model that simultaneously performs well on all tasks. It needs to contain all information in its weights needed to do well on test points without looking at training points. Therefore we would expect the information content of the weights of a memorization model to be larger, and hence the model should be more complex. As a result, we propose an objective that regularizes the information complexity of the meta-learned function class (motivated by Alemi et al. (2016) ; Achille & Soatto (2018) ). Furthermore, we show that meta-regularization in MAML can be rigorously motivated by a PAC-Bayes bound on generalization. In a series of experiments on non-mutually-exclusive task distributions entailing both few-shot regression and classification, we find that memorization poses a significant challenge for both gradient-based (Finn et al., 2017) and contextual (Garnelo et al., 2018a ) meta-learning methods, resulting in near random performance on test tasks in some cases. Our meta-regularization approach enables both of these methods to achieve efficient adaptation and generalization, leading to substantial performance gains across the board on non-mutually-exclusive tasks. Meta-learning has achieved remarkable success in few-shot learning problems. However, we identify a pitfall of current algorithms: the need to create task distributions that are mutually exclusive. This requirement restricts the domains that meta-learning can be applied to. We formalize the failure mode, i.e. the memorization problem, that results from training on non-mutually-exclusive tasks and distinguish it as a function-level overfitting problem compared to the the standard label-level overfitting in supervised learning. We illustrate the memorization problem with different meta-learning algorithms on a number of domains. To address the problem, we propose an algorithm-agnostic meta-regularization (MR) approach that leverages an information-theoretic perspective of the problem. The key idea is that by placing a soft restriction on the information flow from meta-parameters in prediction of test set labels, we can encourage the meta-learner to use task training data during meta-training. We achieve this by successfully controlling the complexity of model prior to the task adaptation. The memorization issue is quite broad and is likely to occur in a wide range of real-world applications, for example, personalized speech recognition systems, learning robots that can adapt to different environments (Nagabandi et al., 2018) , and learning goal-conditioned manipulation skills using trial-and-error data. Further, this challenge may also be prevalent in other conditional prediction problems, beyond meta-learning, an interesting direction for future study. By both recognizing the challenge of memorization and developing a general and lightweight approach for solving it, we believe that this work represents an important step towards making meta-learning algorithms applicable to and effective on any problem domain. We present the detailed algorithm for meta-regularization on weights with conditional neural processes (CNP) in Algorithm 1 and with model-agnostic meta-learning (MAML) in Algorithm 2. For CNP, we add the regularization on the weights θ of encoder and leave other weightsθ unrestricted. For MAML, we similarly regularize the weights θ from input to an intermediate hidden layer and leave the weightsθ for adaptation unregularized. In this way, we restrict the complexity of the pre-adaptation model not the post-adaptation model. Algorithm 1: Meta-Regularized CNP input : Task distribution p(T ); Encoder weights distribution q(θ; τ ) = N (θ; τ ) with Gaussian parameters τ = (θ µ , θ σ ); Prior distribution r(θ) and Lagrangian multiplier β;θ that parameterizes feature extractor hθ(·) and decoder Tθ(·). Stepsize α. output: Network parameter τ ,θ. Initialize τ ,θ randomly; while not converged do Sample a mini-batch of {T i } from p(T ); Sample θ ∼ q(θ; τ ) with reparameterization ; Algorithm 2: Meta-Regularized MAML input : Task distribution p(T ); Weights distribution q(θ; τ ) = N (θ; τ ) with Gaussian parameters τ = (θ µ , θ σ ); Prior distribution r(θ) and Lagrangian multiplier β; Stepsize α, α . output: Network parameter τ ,θ. Initialize τ ,θ randomly; while not converged do Sample a mini-batch of {T i } from p(T ); Sample θ ∼ q(θ; τ ) with reparameterization ; Compute task specific parameter φ i =θ + α ∇θ log q(y i |z i ,θ) ; Updateθ ←θ + α∇θ Ti log q(y Algorithm 3: Meta-Regularized Methods in Meta-testing input : Meta-testing task T with training data D = (x, y) and testing input x * , optimized parameters τ,θ. (hθ(z k , y) ) for MR-CNP and We show that I(x * ;ŷ * |D, z * , θ) ≤ I(ŷ * ; D|z * , θ). By Figure 4 , we have that I(ŷ * ; x * |θ, D, z * ) = 0. By the chain rule of mutual information we have A.3 META REGULARIZATION ON WEIGHTS Similar to (Achille & Soatto, 2018) , we use ξ to denote the unknown parameters of the true data generating distribution. This defines a joint distribution The meta-training loss in Eq. 1 is an upper bound for the cross entropy H p,q (y Here the only negative term is the I(y * 1:N , D 1:N ; θ|x * 1:N , ξ), which quantifies the information that the meta-parameters contain about the meta-training data beyond what can be inferred from the data generating parameters (i.e., memorization). Without proper regularization, the cross entropy loss can be minimized by maximizing this term. We can control its value by upper bounding it where the second equality follows because θ and ξ are conditionally independent given M. This gives the regularization in Section 4.2.
We identify and formalize the memorization problem in meta-learning and solve this problem with novel meta-regularization method, which greatly expand the domain that meta-learning can be  applicable to and effective on.
{"src": "scitldr-AIC_train", "paper_id": "BklEFpEYwS"}