scofield7419 commited on
Commit
302ccff
Β·
verified Β·
1 Parent(s): 6718cde

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +48 -4
README.md CHANGED
@@ -28,19 +28,63 @@ pinned: false
28
  <h1 align="center" style="color:#F27E7E"><em>
29
  Does higher performance across tasks indicate a stronger capability of MLLM, and closer to AGI?
30
  <br>
31
- NO! <b style="color:red">Synergy</b> does.
32
  </em></h1>
33
 
34
 
35
- This project introduces:
36
 
37
- 1. **General-Level**, a 5-scale level evaluation system with a new norm for assessing the multimodal generalists (multimodal LLMs/agents). The core is the use of Synergy as the evaluative criterion, categorizing capabilities based on whether MLLMs preserve synergy across comprehension and generation, as well as across multimodal interactions.
38
 
39
- 2. **General-Bench**, a companion massive multimodal benchmark dataset, encompasses a broader spectrum of skills, modalities, formats, and capabilities, including over 700 tasks and 325K instances.
40
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
41
 
42
 
43
 
44
 
45
 
46
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
  <h1 align="center" style="color:#F27E7E"><em>
29
  Does higher performance across tasks indicate a stronger capability of MLLM, and closer to AGI?
30
  <br>
31
+ NO! But <b style="color:red">synergy</b> does.
32
  </em></h1>
33
 
34
 
35
+ Most current MLLMs predominantly build on the language intelligence of LLMs to simulate the indirect intelligence of multimodality, which is merely extending language intelligence to aid multimodal understanding. While LLMs (e.g., ChatGPT) have already demonstrated such synergy in NLP, reflecting language intelligence, unfortunately, the vast majority of MLLMs do not really achieve it across modalities and tasks.
36
 
37
+ We argue that the key to advancing towards AGI lies in the synergy effectβ€”a capability that enables knowledge learned in one modality or task to generalize and enhance mastery in other modalities or tasks, fostering mutual improvement across different modalities and tasks through interconnected learning.
38
 
 
39
 
40
+ <div align="center">
41
+ <img src='https://cdn-uploads.huggingface.co/production/uploads/647773a1168cb428e00e9a8f/-Asn68kJGjgqbGqZMrk4E.png' width=950px>
42
+ </div>
43
+
44
+ ---
45
+ πŸ†πŸ†πŸ† Overall Leaderboad
46
+
47
+ <div align="center">
48
+ <img src='https://cdn-uploads.huggingface.co/production/uploads/647773a1168cb428e00e9a8f/32goE-PYuwOwRvYg4GcfK.png' width=900px>
49
+ </div>
50
+
51
+
52
+ ---
53
+ ---
54
+
55
+ This project introduces **General-Level** and **General-Bench**.
56
+
57
+ ---
58
+ πŸš€πŸš€πŸš€ **General-Level**: a 5-scale level evaluation system with a new norm for assessing the multimodal generalists (multimodal LLMs/agents). The core is the use of Synergy as the evaluative criterion, categorizing capabilities based on whether MLLMs preserve synergy across comprehension and generation, as well as across multimodal interactions.
59
+
60
+
61
+ <div align="center">
62
+ <img src='https://cdn-uploads.huggingface.co/production/uploads/647773a1168cb428e00e9a8f/lnvh5Qri9O23uk3BYiedX.jpeg'>
63
+ </div>
64
 
65
 
66
 
67
 
68
 
69
 
70
+ <div align="center">
71
+ <img src='https://cdn-uploads.huggingface.co/production/uploads/647773a1168cb428e00e9a8f/BPqs-3UODQWvjFzvZYkI4.png' width=1000px>
72
+ </div>
73
+
74
+
75
+
76
+ ---
77
+ 🌐 **General-Bench**, a companion massive multimodal benchmark dataset, encompasses a broader spectrum of skills, modalities, formats, and capabilities, including over 700 tasks and 325K instances.
78
+
79
+ <div align="center">
80
+ <img src='https://cdn-uploads.huggingface.co/production/uploads/647773a1168cb428e00e9a8f/d4TIWw3rlWuxpBCEpHYJB.jpeg'>
81
+ </div>
82
+
83
+
84
+
85
+
86
+
87
+ <div align="center">
88
+ <img src='https://cdn-uploads.huggingface.co/production/uploads/647773a1168cb428e00e9a8f/qkD43ne58w31Z7jpkTKjr.jpeg'>
89
+ </div>
90
+