This paper introduces _WizardCoder_, a Code _Evol-Instruct_ fine-tuned Code LLM. The experimental results demonstrate that _WizardCoder_ achieves SOTA performance surpassing all existing open-source Code LLMs on four widely recognized code generation benchmarks: HumanEval, HumanEval+, MBPP, and DS-1000. Furthermore, _WizardCoder_ exhibits superior performance compared to the largest closed LLMs, including Anthropic's Claude and Google's Bard.