Update README.md
Browse files
README.md
CHANGED
|
@@ -4,7 +4,7 @@ frameworks:
|
|
| 4 |
tasks:
|
| 5 |
- text-generation
|
| 6 |
base_model:
|
| 7 |
-
- XGenerationLab/XiYanSQL-QwenCoder-
|
| 8 |
base_model_relation: adapter
|
| 9 |
language:
|
| 10 |
- en
|
|
@@ -14,10 +14,11 @@ license: apache-2.0
|
|
| 14 |
|
| 15 |
### Important Links
|
| 16 |
π[Github](https://github.com/XGenerationLab/XiYanSQL-QwenCoder) |
|
|
|
|
| 17 |
π€[ModelScope](https://modelscope.cn/collections/XiYanSQL-Models-4483337b614241) |
|
| 18 |
π[XiYan-SQL](https://github.com/XGenerationLab/XiYan-SQL) |
|
| 19 |
π[ζθ¨GBI](https://bailian.console.aliyun.com/xiyan) |
|
| 20 |
-
π»[
|
| 21 |
|
| 22 |
|
| 23 |
## Introduction
|
|
@@ -33,33 +34,37 @@ In this evaluation, we have also added **a real-world SQL benchmark (the DW test
|
|
| 33 |
|
| 34 |
| **Model** | **Download Latest** |
|
| 35 |
|-----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
|
|
|
| 36 |
|XiYanSQL-QwenCoder-7B | π€[HuggingFace](https://huggingface.co/XGenerationLab/XiYanSQL-QwenCoder-7B-2504) π€[Modelscope](https://www.modelscope.cn/models/XGenerationLab/XiYanSQL-QwenCoder-7B-2504) |
|
|
|
|
| 37 |
|XiYanSQL-QwenCoder-32B | π€[HuggingFace](https://huggingface.co/XGenerationLab/XiYanSQL-QwenCoder-32B-2504) π€[Modelscope](https://www.modelscope.cn/models/XGenerationLab/XiYanSQL-QwenCoder-32B-2504) |
|
| 38 |
|
| 39 |
-
|
| 40 |
-
|XiYanSQL-QwenCoder-14B | π€[HuggingFace](https://huggingface.co/XGenerationLab/XiYanSQL-QwenCoder-14B-2504) π€[Modelscope](https://www.modelscope.cn/models/XGenerationLab/XiYanSQL-QwenCoder-14B-2504) |
|
| 41 |
-
-->
|
| 42 |
|
| 43 |
## Performance
|
| 44 |
The XiYanSQL-QwenCoder models, as multi-dialect SQL base models, demonstrating robust SQL generation capabilities. The following presents the evaluation results at the time of release. We conducted a comprehensive evaluation of the model's performance under two schema formats, M-Schema, and original DDL, using the BIRD and Spider as SQLite benchmarks in the Text-to-SQL domain, as well as DW benchmarks for PostgreSQL and MySQL dialects.
|
| 45 |
|
| 46 |
-
| Model name |
|
| 47 |
-
|
| 48 |
-
| GPT-4o-0806 |
|
| 49 |
-
| GPT-4.1-0414 |
|
| 50 |
-
| Claude3.5-sonnet-1022 |
|
| 51 |
-
| Claude3.7-sonnet |
|
| 52 |
-
| Gemini-1.5-Pro |
|
| 53 |
-
| DeepSeek-V2.5-1210 |
|
| 54 |
-
| DeepSeek-V3 |
|
| 55 |
-
| DeepSeek-R1 |
|
| 56 |
-
| DeepSeek-R1-Distill-Qwen-32B |
|
| 57 |
-
| Deepseek-Coder-33B-Instruct |
|
| 58 |
-
| OmniSQL-32B |
|
| 59 |
-
| XiYanSQL-QwenCoder-
|
| 60 |
-
| XiYanSQL-QwenCoder-
|
| 61 |
-
| XiYanSQL-QwenCoder-
|
| 62 |
-
| XiYanSQL-QwenCoder-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 63 |
|
| 64 |
|
| 65 |
|
|
|
|
| 4 |
tasks:
|
| 5 |
- text-generation
|
| 6 |
base_model:
|
| 7 |
+
- XGenerationLab/XiYanSQL-QwenCoder-7B-2502
|
| 8 |
base_model_relation: adapter
|
| 9 |
language:
|
| 10 |
- en
|
|
|
|
| 14 |
|
| 15 |
### Important Links
|
| 16 |
π[Github](https://github.com/XGenerationLab/XiYanSQL-QwenCoder) |
|
| 17 |
+
π€[HuggingFace](https://huggingface.co/collections/XGenerationLab/xiyansql-models-67c9844307b49f87436808fc) |
|
| 18 |
π€[ModelScope](https://modelscope.cn/collections/XiYanSQL-Models-4483337b614241) |
|
| 19 |
π[XiYan-SQL](https://github.com/XGenerationLab/XiYan-SQL) |
|
| 20 |
π[ζθ¨GBI](https://bailian.console.aliyun.com/xiyan) |
|
| 21 |
+
π»[Modelscope Space](https://www.modelscope.cn/studios/XGenerationLab/XiYanSQL-QwenCoder-32B)
|
| 22 |
|
| 23 |
|
| 24 |
## Introduction
|
|
|
|
| 34 |
|
| 35 |
| **Model** | **Download Latest** |
|
| 36 |
|-----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
| 37 |
+
|XiYanSQL-QwenCoder-3B | π€[HuggingFace](https://huggingface.co/XGenerationLab/XiYanSQL-QwenCoder-3B-2504) π€[Modelscope](https://www.modelscope.cn/models/XGenerationLab/XiYanSQL-QwenCoder-3B-2504) |
|
| 38 |
|XiYanSQL-QwenCoder-7B | π€[HuggingFace](https://huggingface.co/XGenerationLab/XiYanSQL-QwenCoder-7B-2504) π€[Modelscope](https://www.modelscope.cn/models/XGenerationLab/XiYanSQL-QwenCoder-7B-2504) |
|
| 39 |
+
|XiYanSQL-QwenCoder-14B | π€[HuggingFace](https://huggingface.co/XGenerationLab/XiYanSQL-QwenCoder-14B-2504) π€[Modelscope](https://www.modelscope.cn/models/XGenerationLab/XiYanSQL-QwenCoder-14B-2504) |
|
| 40 |
|XiYanSQL-QwenCoder-32B | π€[HuggingFace](https://huggingface.co/XGenerationLab/XiYanSQL-QwenCoder-32B-2504) π€[Modelscope](https://www.modelscope.cn/models/XGenerationLab/XiYanSQL-QwenCoder-32B-2504) |
|
| 41 |
|
| 42 |
+
|
|
|
|
|
|
|
| 43 |
|
| 44 |
## Performance
|
| 45 |
The XiYanSQL-QwenCoder models, as multi-dialect SQL base models, demonstrating robust SQL generation capabilities. The following presents the evaluation results at the time of release. We conducted a comprehensive evaluation of the model's performance under two schema formats, M-Schema, and original DDL, using the BIRD and Spider as SQLite benchmarks in the Text-to-SQL domain, as well as DW benchmarks for PostgreSQL and MySQL dialects.
|
| 46 |
|
| 47 |
+
| Model name | Size | BIRD Dev@M-Schema | BIRD Dev@DDL | Spider Test@M-Schema | Spider Test@DDL | DW PostgreSQL@M-Schema | DW MySQL@M-Schema |
|
| 48 |
+
|------------------------------|:----:|:-----------------:|:------------:|:--------------------:|:---------------:|:----------------------:|:-----------------:|
|
| 49 |
+
| GPT-4o-0806 | UNK | 58.47% | 54.82% | 82.89% | 78.45% | 46.79% | 57.77% |
|
| 50 |
+
| GPT-4.1-0414 | UNK | 59.39% | 54.11% | 84.45% | 79.86% | 54.29% | 63.18% |
|
| 51 |
+
| Claude3.5-sonnet-1022 | UNK | 53.32% | 50.46% | 76.27% | 73.04% | 55.22% | 52.84% |
|
| 52 |
+
| Claude3.7-sonnet | UNK | 54.82% | 49.22% | 78.04% | 74.66% | 53.23% | 54.61% |
|
| 53 |
+
| Gemini-1.5-Pro | UNK | 61.34% | 57.89% | 85.11% | 84.00% | 52.78% | 62.78% |
|
| 54 |
+
| DeepSeek-V2.5-1210 | 236B | 55.74% | 55.61% | 82.08% | 80.57% | 45.74% | 52.18% |
|
| 55 |
+
| DeepSeek-V3 | 685B | 59.58% | 56.71% | 81.52% | 79.91% | 52.56% | 55.95% |
|
| 56 |
+
| DeepSeek-R1 | 685B | 58.15% | 55.61% | 80.72% | 78.85% | 60.56% | 62.00% |
|
| 57 |
+
| DeepSeek-R1-Distill-Qwen-32B | 32B | 50.65% | 48.31% | 78.65% | 77.33% | 37.22% | 44.72% |
|
| 58 |
+
| Deepseek-Coder-33B-Instruct | 33B | 47.52% | 44.72% | 72.39% | 62.0% | 31.48% | 36.17% |
|
| 59 |
+
| OmniSQL-32B | 32B | 60.37% | 55.87% | 85.16% | 83.19% | 38.19% | 42.34% |
|
| 60 |
+
| XiYanSQL-QwenCoder-3B-2502 | 3B | 53.52% | 52.54% | 83.34% | 79.10% | 34.75% | 35.62% |
|
| 61 |
+
| XiYanSQL-QwenCoder-3B-2504 | 3B | 55.08% | 52.09% | 84.10% | 80.57% | 36.65% | 37.63% |
|
| 62 |
+
| XiYanSQL-QwenCoder-7B-2502 | 7B | 59.65% | 56.32% | 84.15% | 80.01% | 39.38% | 42.10% |
|
| 63 |
+
| XiYanSQL-QwenCoder-7B-2504 | 7B | 62.13% | 57.43% | 85.97% | 82.48% | 42.08% | 44.67% |
|
| 64 |
+
| XiYanSQL-QwenCoder-14B-2502 | 14B | 63.23% | 60.10% | 85.31% | 82.84% | 38.51% | 41.62% |
|
| 65 |
+
| XiYanSQL-QwenCoder-14B-2504 | 14B | 65.32% | 60.17% | 86.82% | 83.75% | 40.52% | 44.60% |
|
| 66 |
+
| XiYanSQL-QwenCoder-32B-2412 | 32B | 67.07% | 63.04% | 88.39% | 85.46% | 45.07% | 52.84% |
|
| 67 |
+
| XiYanSQL-QwenCoder-32B-2504 | 32B | 67.14% | 62.26% | 89.20% | 86.17% | 53.52% | 57.74% |
|
| 68 |
|
| 69 |
|
| 70 |
|