Scaling up large generative models requires a significant financial investment. For example, GPT-3 was estimated to cost several million dollars to train.https://lambdalabs.com/blog/demystifying-gpt-3/ Scaling up large generative models also requires specific engineering competencies, e.g., distributed systems engineering, familiarity with cluster management tools like Kubernetes, low-level GPU programming, managing continuous integration testing, etc. The size of these models has led to longer development timelines and more complex workflows than previous systems over the past decade. For example, only \(\sim 10\)  years ago, one of the larger scale AI models at the time, AlexNetThough not a generative model, AlexNet was, at the time, a frontier model in terms of computational consumption, hence why we include it as a comparison. [1]}, was trained by a graduate student for a few thousand of dollars on a single desktop machine with 2 GPUs.
