Table 2 and Figure 3 show that our method generally enhances performance across all tasks and LLMs by increasing the ensemble size. Specifically, in arithmetic reasoning tasks, the accuracy gains range from \(12\%\) to \(24\%\) on the GSM8K and from \(6\%\) to \(10\%\) on the MATH. In general reasoning tasks, the accuracy gains range from \(1\%\) to \(4\%\) on the Chess and from \(5\%\) to \(11\%\) on the MMLU. In code generation task, the accuracy gains range from \(4\%\) to \(9\%\) on HumanEval. Surprisingly, our method enables a smaller LLM to outperform a larger counterpart by simply scaling up the ensemble size. For instance, the enhanced Llama2-13B model achieves \(59\%\) accuracy on the GSM8K dataset, outperforming the Llama2-70B model, which scores \(54\%\).