At the time of writing, the largest language models that are free and publicly available are BigScience T0 (11B) [1]}, and Eleuther AI’s GPT-J (6B) [2]} and GPT-NeoX (20B) [3]}, which are one to two orders of magnitude smaller than those developed by industry. Although academics can easily access (at least some of) the larger models, it is typically only possible to do so through a (potentially expensive) company-controlled API. This is part of a broader and longer-running trend towards high-compute research migrating from academia to industry that can be quantified (See Appendix REF  for details ). Figure REF  (Left) shows that in recent years the compute required for large-scale AI experiments has increased by more than \(300,000\) X relative to a decade ago.Some people have noted that this trend may not be sustainable [4]} Along with this rise in resource intensity, we see a corresponding (and sharp) fall in the proportion of these results that come from academia (Figure REF , Right). This suggests that, although academics may be strongly motivated by scientific curiosity, and well-poised to research safety issues, they may be significantly challenged by the high financial and engineering costs.
