| This is my reproduction of the Microsoft team's work, WarriorCoder: Learning from Expert Battles to Augment Code Large Language Models. It is fully based on open-source models to construct training data and adopt supervised fine-tuning (SFT) to train the model. Also, I reproduced the experimental results in the paper. These results are excellent, confirming that the idea of 'learning from expert battles' proposed in the paper has great potential. I have also published the training data constructed during my reproduction of the paper in another repository, and everyone is welcome to use it. Original paper link: https://arxiv.org/pdf/2412.17395 I have also published the training data constructed during my reproduction of the paper in another repository: https://huggingface.co/datasets/HuggingMicah/warrior_reproduce . | |
| | Models | Matplotlib (155) | NumPy (220) | Pandas (291) | PyTorch (68) | SciPy (106) | Sklearn (115) | TensorFlow (45) | Overall (1000) | | |
| | --- | --- | --- | --- | --- | --- | --- | --- | --- | | |
| | INCODER (6.7B) | 28.3 | 4.4 | 3.1 | 4.4 | 2.8 | 2.8 | 3.8 | 7.4 | | |
| | CodeGen-Mono (16B) | 31.7 | 10.9 | 3.4 | 7.0 | 9.0 | 10.8 | 15.2 | 11.7 | | |
| | Code-Cushman-001 | 40.7 | 21.8 | 7.9 | 12.4 | 11.3 | 18.0 | 12.2 | 18.1 | | |
| | StarCoder (15B) | 51.7 | 29.7 | 11.4 | 21.4 | 20.2 | 29.5 | 24.5 | 26.0 | | |
| | WizardCoder-SC (15B) | 55.2 | 33.6 | 16.7 | 26.2 | 24.2 | 24.9 | 26.7 | 29.2 | | |
| | CodeLlama-Python (6.7B) | 55.3 | 34.5 | 16.4 | 19.9 | 22.3 | 17.6 | 28.5 | 28.0 | | |
| | WizardCoder-CL (6.7B) | 53.5 | 34.4 | 15.2 | 25.7 | 21.0 | 24.5 | 28.9 | 28.4 | | |
| | Magicoder-CL (6.7B) | 54.6 | 34.8 | 19.0 | 24.7 | 25.0 | 22.6 | 28.9 | 29.9 | | |
| | MagicoderS-CL (6.7B) | 55.9 | 40.6 | 28.4 | 40.4 | 28.8 | 35.8 | 37.6 | 37.5 | | |
| | WarriorCoder_published_in_paper (6.7B) | 55.5 | 41.8 | 26.1 | 41.2 | 33.0 | 39.1 | 42.2 | 38.1 | | |
| | WarriorCoder_my_reproduce (6.7B) | 56.1 | 45.0 | 32.0 | 38.2 | 36.8 | 44.3 | 48.9 | 41.7 | | |
| | Models | HumanEval | HumanEval+ | MBPP | MBPP+ | | |
| | --- | --- | --- | --- | --- | | |
| | WizardCoder-CL (6.7B) | 48.7 | 40.5 | 56.4 | 47.0 | | |
| | WizardCoder-SC (15B) | 51.4 | 45.3 | 61.6 | 50.7 | | |
| | Magicoder-CL (6.7B) | 60.4 | 55.7 | 64.2 | 52.5 | | |
| | MagicoderS-CL (6.7B) | 70.7 | 66.4 | 68.3 | 56.4 | | |
| | WarriorCoder (6.7B) | 79.9 | 75.4 | 75.8 | 64.5 | |