HackerPapers

arXiv:2306.08568cs.CL

WizardCoder: Empowering Code Large Language Models with Evol-Instruct

Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, Daxin Jiang

Abstract

Code Large Language Models (Code LLMs), such as StarCoder, have demonstrated exceptional performance in code-related tasks. However, most existing models are solely pre-trained on extensive raw code data without instruction fine-tuning. In this paper, we introduce WizardCoder, which empowers Code LLMs with complex instruction fine-tuning, by adapting the Evol-Instruct method to the domain of code. Through comprehensive experiments on four prominent code generation benchmarks, namely HumanEval, HumanEval+, MBPP, and DS-1000, we unveil the exceptional capabilities of our model. It surpasses all other open-source Code LLMs by a substantial margin. Moreover, our model even outperforms the largest closed LLMs, Anthropic's Claude and Google's Bard, on HumanEval and HumanEval+. Our code, model weights, and data are public at https://github.com/nlpxucan/WizardLM

arXiv comments
Large Language model, Code Generation, Code LLMs.This paper has been accepted to ICLR 2024. Please cite the ICLR version
Journal reference
The Twelfth International Conference on Learning Representations (ICLR 2024)
DOI
Subjects
cs.CLComputation and Language cs.AIArtificial Intelligence

On Hacker News

Linked by 2 identifiable commenters in 2 comments across 2 threads, from Jul 2023 to Dec 2023.

Comments linking this paper, per year.

Discussed in comments2 threads

  1. Ask HN: AI/ML papers to catch up with current state of AI? 1 comment

    • To that, what can these express, precisely, is an interesting question; so for transformer encoders: https://arxiv.org/abs/2301.10743 Another interesting research topic is the trusted generation of tasks for finetuning https://arxiv.org/abs/2306.08568 And I suppose too running…

      meltynessView on HNParent context

  2. LLaMA2 Chat 70B outperformed ChatGPT 1 comment

    • While there's some contamination, it's not like the community isn't aware of it. For example, here's this discussion: https://huggingface.co/sahil2801/replit-code-instruct-glaive... This was the HumanEval contamination one dev measured: ``` replit_glaive: 56.71% replit: 7.32%…

      lhlView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.