Kyle Steinfeld &ndash; University of California, Berkeley, CA, USA
Clever little tricks: A socio-technical history of text-to-image generative models
The emergence of text-to-image generative models (e.g., Midjourney, DALL-E 2, Stable Diffusion) in the summer of 2022 impacted architectural visual culture suddenly, severely, and seemingly out of nowhere. To contextualize this phenomenon, this text offers a socio-technical history of text-to-image generative systems. Three moments in time, or “scenes,” are presented here: the first at the advent of AI in the middle of the last century; the second at the “reawakening” of a specific approach to machine learning at the turn of this century; the third that documents a rapid sequence of innovations, dubbed “clever little tricks,” that occurred across just 18 months. This final scene is the crux, and represents the first formal documentation of the recent history of a specific set of informal innovations. These innovations were produced by non-affiliated researchers and communities of creative contributors, and directly led to the technologies that so compellingly captured the architectural imagination in the summer of 2022. Across these scenes, we examine the technologies, application domains, infrastructures, social contexts, and practices that drive technical research and shape creative practice in this space.
The LAION-400 dataset consists of 400 million image-caption pairs extracted from random selections of web pages from a web scrape that captured sites between 2014 and 2021 that was conducted by Common Crawl (a separate non- proﬁt established in 2011 “with the goal of democratizing access to web information by producing and maintaining an open repository of web crawl data”).⁷⁵ [⁷⁵ Gil E. About common crawl. 2011. https://commoncrawl.org/about/ (accessed 04 December 2022).] Although it speciﬁcally was “not meant for any real-world production or application,”⁷⁶ [⁷⁶ Schuhmann C. LAION-400-Million open dataset. December 12, 2022. https://laion.ai/blog/laion-400-open-dataset (accessed 04 December 2022).] this dataset was used by Google to train its text-to-image generative model “Imagen” in 2022.⁷⁷ [⁷⁷ Saharia C, Chan W, Saxena S, et al. Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding. Epub ahead of print May 2022. DOI: 10.48550/arXiv.2205.11487]