Recent work on few-shot learning for natural language tasks explores the dynamics of scaling up either the number of model parameters [1]} or labeled examples [2]}, while controlling for the other variable by setting it to a constant.
For example, [1]} focus on in-context learning from roughly 32 to 64 examples, a practice that was adopted by fine-tuning approaches as well [4]}, [5]}, [6]}; however, there are many practical few-shot scenarios where hundreds of examples can be collected at a relatively low effort.In SQuAD [7]}, for example, the average annotation pace is around one minute per question, producing 480 examples in a single 8-hour workday.
Other work experiments with single-size models [8]}, [9]}, [2]}, [5]}, even though larger (or smaller) models may exhibit different behavior.
Furthermore, much of the literature focuses on classification tasks [12]}, [5]}, [2]}, leaving it unclear whether their conclusions generalize to tasks with less restricted output spaces.
<FIGURE>