Pre-training: FRN pre-training is crucial for competitive general few-shot performance, especially when compared to pre-trained baselines. However, pre-training alone does not produce a competitive few-shot learner. An FRN trained from scratch outperforms a pre-trained FRN evaluated naively (Table REF , bottom rows). The two-round process of pre-training followed by episodic fine-tuning appears to be crucial. This finding is in line with prior work [1]}, [2]}.
