We use the same model as in [1]} for our results to be comparableCode and datasets will be available soon.
The model is a BERT transformer encoder coupled with a parameter generator, a 2-layer MLP, that generates the initial point for classification layer for a task conditioned on the support examples.
The model is meta-trained using the MAML algorithm [2]}, with learned per-layer learning rates, on the self-supervised task distributions.
All model hyper-parameters are kept the same so that any change in performance can be attributed to differences in the task distribution. See Supplementary for all hyper-parameters.
