Large-scale language models have showcased immense capabilities, but they also come with substantial overhead. Balancing the trade-off between performance and overhead may require significant attention in future research endeavors.