As a third example, a recently developed class of program synthesis models from Google display dramatic improvements in their ability to create computer programs as they increase in size from 10B to 100B parameters [1]} (Figure REF , Right). For example, the percentage of generated synthetic programs that solve a given programming problem jumps substantially from 6% to 13% when the model size increases by \(\sim 2\) x from 68B to 138B parameters, despite very small increases over the previous two orders of magnitude.
