Figure 1: The accuracy increases with ensemble size across Llama2-13B, Llama2-70B and GPT-3.5-Turbo in GSM8K. When the ensemble size scales up to \(15\), Llama2-13B achieves comparable accuracy with Llama2-70B. Similarly, When the ensemble size scales up to \(15\) and \(20\), Llama2-70B and GPT-3.5-Turbo achieve comparable accuracy with their more powerful counterparts.