|
|
--- |
|
|
pipeline_tag: image-text-to-text |
|
|
library_name: transformers |
|
|
license: mit |
|
|
--- |
|
|
|
|
|
# Skywork-R1V2 |
|
|
|
|
|
<div align="center"> |
|
|
<img src="skywork-logo.png" alt="Skywork Logo" width="500" height="400"> |
|
|
</div> |
|
|
|
|
|
## 📖 [R1V2 Report](https://arxiv.org/abs/2504.16656) | 💻 [GitHub](https://github.com/SkyworkAI/Skywork-R1V) | 🌐 [ModelScope](https://modelscope.cn/models/Skywork/Skywork-R1V2-38B) |
|
|
<p align="center"> |
|
|
<a href="https://github.com/SkyworkAI/Skywork-R1V/stargazers"> |
|
|
<img src="https://img.shields.io/github/stars/SkyworkAI/Skywork-R1V" alt="GitHub Stars" /> |
|
|
</a> |
|
|
<a href="https://github.com/SkyworkAI/Skywork-R1V/fork"> |
|
|
<img src="https://img.shields.io/github/forks/SkyworkAI/Skywork-R1V" alt="GitHub Forks" /> |
|
|
</a> |
|
|
</p> |
|
|
|
|
|
|
|
|
|
|
|
## 1. Model Introduction |
|
|
|
|
|
Skywork-R1V2-38B is a **state-of-the-art open-source multimodal reasoning model**, achieving top-tier performance across multiple benchmarks: |
|
|
- On **MMMU**, it scores **73.6%**, the **highest among all open-source models** to date. |
|
|
- On **OlympiadBench**, it achieves **62.6%**, leading **by a large margin** over other open models. |
|
|
- R1V2 also performs strongly on **MathVision**, **MMMU-Pro**, and **MathVista**, **rivaling proprietary commercial models**. |
|
|
- Overall, R1V2 stands out as a **high-performing, open-source VLM** combining powerful **visual reasoning** and **text understanding**. |
|
|
|
|
|
### 🔧 Model Details |
|
|
<table> |
|
|
<thead> |
|
|
<tr> |
|
|
<th><strong>Model Name</strong></th> |
|
|
<th><strong>Vision Encoder</strong></th> |
|
|
<th><strong>Language Model</strong></th> |
|
|
<th><strong>Hugging Face Link</strong></th> |
|
|
</tr> |
|
|
</thead> |
|
|
<tbody> |
|
|
<tr> |
|
|
<td>Skywork-R1V2-38B</td> |
|
|
<td><a href="https://huggingface.co/OpenGVLab/InternViT-6B-448px-V2_5">InternViT-6B-448px-V2_5</a></td> |
|
|
<td><a href="https://huggingface.co/Qwen/QwQ-32B">Qwen/QwQ-32B</a></td> |
|
|
<td><a href="https://huggingface.co/Skywork/Skywork-R1V2-38B">🤗 Link</a></td> |
|
|
</tr> |
|
|
</tbody> |
|
|
</table> |
|
|
|
|
|
|
|
|
--- |
|
|
|
|
|
## 2. Evaluation |
|
|
|
|
|
|
|
|
<style> |
|
|
section { |
|
|
margin-bottom: 4em; |
|
|
} |
|
|
figure { |
|
|
margin: 2em 0; |
|
|
text-align: center; |
|
|
} |
|
|
figcaption { |
|
|
font-weight: bold; |
|
|
margin-top: 0.5em; |
|
|
} |
|
|
table { |
|
|
margin: 3em auto; |
|
|
width: 100%; |
|
|
border-collapse: collapse; |
|
|
} |
|
|
table th, table td { |
|
|
padding: 0.6em; |
|
|
border: 1px solid #ddd; |
|
|
text-align: center; |
|
|
} |
|
|
</style> |
|
|
|
|
|
<section> |
|
|
<figure> |
|
|
<img src="open_source.png" alt="Open Source" width="100%" /> |
|
|
<figcaption>Comparison with Larger-Scale Open-Source Models</figcaption> |
|
|
</figure> |
|
|
</section> |
|
|
|
|
|
<section> |
|
|
<figure> |
|
|
<img src="properitary.png" alt="Proprietary" width="100%" /> |
|
|
<figcaption>Comparison with Proprietary Models</figcaption> |
|
|
</figure> |
|
|
</section> |
|
|
|
|
|
<section> |
|
|
<figure> |
|
|
<table> |
|
|
<thead> |
|
|
<tr> |
|
|
<th>Model</th> |
|
|
<th align="center"><strong>Supports Vision</strong></th> |
|
|
<th align="center" colspan="6"><strong>Text Reasoning (%)</strong></th> |
|
|
<th align="center" colspan="5"><strong>Multimodal Reasoning (%)</strong></th> |
|
|
</tr> |
|
|
<tr> |
|
|
<th></th> |
|
|
<th></th> |
|
|
<th align="center">AIME24</th> |
|
|
<th align="center">LiveCodebench</th> |
|
|
<th align="center">liveBench</th> |
|
|
<th align="center">IFEVAL</th> |
|
|
<th align="center">BFCL</th> |
|
|
<th align="center">GPQA</th> |
|
|
<th align="center">MMMU(val)</th> |
|
|
<th align="center">MathVista(mini)</th> |
|
|
<th align="center">MathVision(mini)</th> |
|
|
<th align="center">OlympiadBench</th> |
|
|
<th align="center">mmmu‑pro</th> |
|
|
</tr> |
|
|
</thead> |
|
|
<tbody> |
|
|
<tr> |
|
|
<td>R1V2‑38B</td> |
|
|
<td align="center">✅</td> |
|
|
<td align="center">78.9</td> |
|
|
<td align="center">63.6</td> |
|
|
<td align="center">73.2</td> |
|
|
<td align="center">82.9</td> |
|
|
<td align="center">66.3</td> |
|
|
<td align="center">61.6</td> |
|
|
<td align="center">73.6</td> |
|
|
<td align="center">74.0</td> |
|
|
<td align="center">49.0</td> |
|
|
<td align="center">62.6</td> |
|
|
<td align="center">52.0</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>R1V1‑38B</td> |
|
|
<td align="center">✅</td> |
|
|
<td align="center">72.0</td> |
|
|
<td align="center">57.2</td> |
|
|
<td align="center">54.6</td> |
|
|
<td align="center">72.5</td> |
|
|
<td align="center">53.5</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">68.0</td> |
|
|
<td align="center">67.0</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">40.4</td> |
|
|
<td align="center">–</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>Deepseek‑R1‑671B</td> |
|
|
<td align="center">❌</td> |
|
|
<td align="center">74.3</td> |
|
|
<td align="center">65.9</td> |
|
|
<td align="center">71.6</td> |
|
|
<td align="center">83.3</td> |
|
|
<td align="center">60.3</td> |
|
|
<td align="center">71.5</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>GPT‑o1</td> |
|
|
<td align="center">❌</td> |
|
|
<td align="center">79.8</td> |
|
|
<td align="center">63.4</td> |
|
|
<td align="center">72.2</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>GPT‑o4‑mini</td> |
|
|
<td align="center">✅</td> |
|
|
<td align="center">93.4</td> |
|
|
<td align="center">74.6</td> |
|
|
<td align="center">78.1</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">49.9</td> |
|
|
<td align="center">81.6</td> |
|
|
<td align="center">84.3</td> |
|
|
<td align="center">58.0</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>Claude 3.5 Sonnet</td> |
|
|
<td align="center">✅</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">65.0</td> |
|
|
<td align="center">66.4</td> |
|
|
<td align="center">65.3</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>Kimi k1.5 long-cot</td> |
|
|
<td align="center">✅</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">70.0</td> |
|
|
<td align="center">74.9</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>Qwen2.5‑VL‑72B‑Instruct</td> |
|
|
<td align="center">✅</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">70.2</td> |
|
|
<td align="center">74.8</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>InternVL2.5‑78B</td> |
|
|
<td align="center">✅</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">70.1</td> |
|
|
<td align="center">72.3</td> |
|
|
<td align="center">–</td> |
|
|
<td align="center">33.2</td> |
|
|
<td align="center">–</td> |
|
|
</tr> |
|
|
</tbody> |
|
|
</table> |
|
|
<figcaption>Evaluation Results of State-of-the-Art LLMs and VLMs</figcaption> |
|
|
</figure> |
|
|
</section> |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
--- |
|
|
|
|
|
|
|
|
## 3. Usage |
|
|
|
|
|
### 1. Clone the Repository |
|
|
|
|
|
```shell |
|
|
git clone https://github.com/SkyworkAI/Skywork-R1V.git |
|
|
cd skywork-r1v/inference |
|
|
``` |
|
|
### 2. Set Up the Environment |
|
|
|
|
|
```shell |
|
|
# For Transformers |
|
|
conda create -n r1-v python=3.10 && conda activate r1-v |
|
|
bash setup.sh |
|
|
# For vLLM |
|
|
conda create -n r1v-vllm python=3.10 && conda activate r1v-vllm |
|
|
pip install -U vllm |
|
|
``` |
|
|
|
|
|
### 3. Run the Inference Script |
|
|
transformers inference |
|
|
|
|
|
```shell |
|
|
CUDA_VISIBLE_DEVICES="0,1" python inference_with_transformers.py \ |
|
|
--model_path path \ |
|
|
--image_paths image1_path \ |
|
|
--question "your question" |
|
|
``` |
|
|
|
|
|
vllm inference |
|
|
```shell |
|
|
python inference_with_vllm.py \ |
|
|
--model_path path \ |
|
|
--image_paths image1_path image2_path \ |
|
|
--question "your question" \ |
|
|
--tensor_parallel_size 4 |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## 4. Citation |
|
|
If you use Skywork-R1V in your research, please cite: |
|
|
|
|
|
``` |
|
|
@misc{chris2025skyworkr1v2multimodalhybrid, |
|
|
title={Skywork R1V2: Multimodal Hybrid Reinforcement Learning for Reasoning}, |
|
|
author={Chris and Yichen Wei and Yi Peng and Xiaokun Wang and Weijie Qiu and Wei Shen and Tianyidan Xie and Jiangbo Pei and Jianhao Zhang and Yunzhuo Hao and Xuchen Song and Yang Liu and Yahui Zhou}, |
|
|
year={2025}, |
|
|
eprint={2504.16656}, |
|
|
archivePrefix={arXiv}, |
|
|
primaryClass={cs.CV}, |
|
|
url={https://arxiv.org/abs/2504.16656}, |
|
|
} |
|
|
``` |
|
|
|
|
|
``` |
|
|
@misc{peng2025skyworkr1vpioneeringmultimodal, |
|
|
title={Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought}, |
|
|
author={Yi Peng and Chris and Xiaokun Wang and Yichen Wei and Jiangbo Pei and Weijie Qiu and Ai Jian and Yunzhuo Hao and Jiachun Pan and Tianyidan Xie and Li Ge and Rongxian Zhuang and Xuchen Song and Yang Liu and Yahui Zhou}, |
|
|
year={2025}, |
|
|
eprint={2504.05599}, |
|
|
archivePrefix={arXiv}, |
|
|
primaryClass={cs.CV}, |
|
|
url={https://arxiv.org/abs/2504.05599}, |
|
|
} |
|
|
``` |
|
|
|
|
|
*This project is released under an open-source license.* |
|
|
|
