Our experiments till this point have focused on isolating the impact of various pipeline components on zero-shot generalization under limited computation.
At the same time, recent results on natural language have highlighted the power of scale in effective transfer learning [1]}, [2]}.
