Initial model scaling involves taking a fixed architecture and individually increasing depth [1]} and width [2]} in separation or together [3]}, [4]}. Further work in compound scaling yielded model families varying in depth, width, and resolution simultaneously and systematically [5]}. Scaling is also more recently used in constructing larger models in conjunction with NAS [6]}, [5]}.
