Predictable scaling, and especially the underlying dependency on precise mixtures of data, model size, and training, has implications for the process of model development. It shifts development of this type of model from a process of artisanal trial-and-error to more of a predictable engineering process, where the resources needed to achieve a particular result can be precisely calculated, and the cost of those resources can be compared to the utility of the result. Although very specific behaviors may not be predictable (more on this in Section REF ), the general test loss tends to correlate well on average with many tasks, meaning that larger models typically make significant gains across the board. In this sense, scaling laws de-risk investments in large models. We say more on this in Section REF  and provide more technical details on how developers may use scaling laws in Appendix REF .
To further illustrate how smooth general scaling correlates with task performance, and how a scale-based analysis can be used to forecast the potential economic value of a given model, we outline a small original experiment in Appendix REF  that analyzes the relationship between scale and GPT-3 like language models [1]} to be used as recommendation systems with zero-shot learning. We chose this example because recommendation systems have tangible economic relevance, known societal impact, are well studied in machine learning with domain specific algorithms [2]}, but are not typically studied with large scale generative models (yet). Surprisingly, we find that that generative models can increasingly operate as simple recommendation systems as they scale with minimal effort and extremely limited access to explicit training data. We leave a detailed analysis and discussion in Appendix REF .
<FIGURE>