Predictability and Surprise in Large Generative Models
DEEP GANGULI∗, DANNY HERNANDEZ∗, LIANE LOVITT∗, NOVA DASSARMA†, TOM HENIG
Predictability and Surprise in Large Generative Models
Large-scale pre-training has recently emerged as a technique for creating
capable, general purpose, generative models such as GPT-3, Megatron-Turing NLG,
Gopher, and many others. In this paper, we highlight a counterintuitive
property of such models and discuss the policy implications of this property.
Namely, these generative models have an unusual combination of predictable loss
on a broad training distribution (as embodied in their "scaling laws"), and
unpredictable specific capabilities, inputs, and outputs. We believe that the
high-level predictability and appearance of useful capabilities drives rapid
development of such models, while the unpredictable qualities make it difficult
to anticipate the consequences of model deployment. We go through examples of
how this combination can lead to socially harmful behavior with examples from
the literature and real world observations, and we also perform two novel
experiments to illustrate our point about harms from unpredictability.
Furthermore, we analyze how these conflicting properties combine to give model
developers various motivations for deploying these models, and challenges that
can hinder deployment. We conclude with a list of possible interventions the AI
community may take to increase the chance of these models having a beneficial
impact. We intend this paper to be useful to policymakers who want to
understand and regulate AI systems, technologists who care about the potential
policy impact of their work, and academics who want to analyze, critique, and
potentially develop large generative models.
Deep Ganguli Danny Hernandez Liane Lovitt Nova DasSarma Tom Henighan Andy Jones Nicholas Joseph Jackson Kernion Ben Mann Amanda Askell Yuntao Bai Anna Chen Tom Conerly Dawn Drain Nelson Elhage Sheer El Showk Stanislav Fort Zac Hatfield-Dodds Scott Johnston Shauna Kravec Neel Nanda Kamal Ndousse Catherine Olsson Daniela Amodei Dario Amodei Tom Brown Jared Kaplan Sam McCandlish Chris Olah Jack Clark