Update README.md
Browse files
README.md
CHANGED
@@ -14,7 +14,7 @@ widget:
|
|
14 |
---
|
15 |
# Model Details
|
16 |
|
17 |
-
|
18 |
|
19 |
## How to use
|
20 |
|
@@ -45,15 +45,18 @@ inference_pipeline("شهر برلین در کشور <mask> واقع شده اس
|
|
45 |
|
46 |
You can use this model to fine-tune it over your dataset and prepare it for your task.
|
47 |
|
|
|
|
|
|
|
48 |
## Evaluation
|
49 |
|
50 |
-
|
51 |
Here are some key performance results:
|
52 |
|
53 |
| Model name | DeepSentiPers (f1/acc) | MultiCoNER-v2 (f1/acc) | PQuAD (best_exact/best_f1/HasAns_exact/HasAns_f1) | FarsTail (f1/acc) | ParsiNLU-Multiple-choice (f1/acc) | ParsiNLU-Reading-comprehension (exact/f1) | ParsiNLU-QQP (f1/acc) |
|
54 |
|------------------|------------------------|------------------------|-----------------------------------------------------|--------------------|-----------------------------------|-------------------------------------------|-----------------------|
|
55 |
-
|
|
56 |
-
|
|
57 |
| Shiraz | 81.17/81.08 | 59.1/92.83 | 65.96/81.25/59.63/81.31 | 77.76/77.75 | <u>34.73/34.53</u> | 17.6/39.61 | 79.68/79.51 |
|
58 |
| ParsBERT | 80.22/80.23 | 64.91/93.23 | 71.41/84.21/66.29/84.57 | 80.89/80.94 | **35.34/35.25** | 20/39.58 | 80.15/80.07 |
|
59 |
| XLM-V-base | <u>83.43/83.36</u> | 58.83/92.23 | <u>73.26</u>/<u>85.69</u>/<u>68.21</u>/<u>85.56</u> | 81.1/81.2 | **35.28/35.25** | 8/26.66 | 80.1/79.96 |
|
|
|
14 |
---
|
15 |
# Model Details
|
16 |
|
17 |
+
TookaBERT models are a family of encoder models trained on Persian in two sizes base and large. These Models pre-trained on over 300GB of Persian data including a variety of topics such as News, Blogs, Forums, Books, etc. They pre-trained with the MLM (WWM) objective using two context lengths.
|
18 |
|
19 |
## How to use
|
20 |
|
|
|
45 |
|
46 |
You can use this model to fine-tune it over your dataset and prepare it for your task.
|
47 |
|
48 |
+
- DeepSentiPers (Sentiment Analysis) <a href="https://colab.research.google.com/drive/1Vn5QTYutdCo6iXVTmsPW9K4t8xVk14ji#scrollTo=1B1YrypZxajF"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Colab Code" width="87" height="15"/></a>
|
49 |
+
- ParsiNLU - Multiple-choice (Multiple-choice) <a href="https://colab.research.google.com/drive/1boXMnRIwqAYGU7oxJtRjgib7Fu-O--x5#scrollTo=7jVb9E4SDPNb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Colab Code" width="87" height="15"/></a>
|
50 |
+
|
51 |
## Evaluation
|
52 |
|
53 |
+
TookaBERT models are evaluated on a wide range of NLP downstream tasks, such as Sentiment Analysis (SA), Text Classification, Multiple-choice, Question Answering, and Named Entity Recognition (NER).
|
54 |
Here are some key performance results:
|
55 |
|
56 |
| Model name | DeepSentiPers (f1/acc) | MultiCoNER-v2 (f1/acc) | PQuAD (best_exact/best_f1/HasAns_exact/HasAns_f1) | FarsTail (f1/acc) | ParsiNLU-Multiple-choice (f1/acc) | ParsiNLU-Reading-comprehension (exact/f1) | ParsiNLU-QQP (f1/acc) |
|
57 |
|------------------|------------------------|------------------------|-----------------------------------------------------|--------------------|-----------------------------------|-------------------------------------------|-----------------------|
|
58 |
+
| TookaBERT-large | **85.66/85.78** | **69.69/94.07** | **75.56/88.06/70.24/87.83** | **89.71/89.72** | **36.13/35.97** | **33.6/60.5** | **82.72/82.63** |
|
59 |
+
| TookaBERT-base | <u>83.93/83.93</u> | <u>66.23/93.3</u> | <u>73.18</u>/<u>85.71</u>/<u>68.29</u>/<u>85.94</u> | <u>83.26/83.41</u> | 33.6/<u>33.81</u> | 20.8/42.52 | <u>81.33/81.29</u> |
|
60 |
| Shiraz | 81.17/81.08 | 59.1/92.83 | 65.96/81.25/59.63/81.31 | 77.76/77.75 | <u>34.73/34.53</u> | 17.6/39.61 | 79.68/79.51 |
|
61 |
| ParsBERT | 80.22/80.23 | 64.91/93.23 | 71.41/84.21/66.29/84.57 | 80.89/80.94 | **35.34/35.25** | 20/39.58 | 80.15/80.07 |
|
62 |
| XLM-V-base | <u>83.43/83.36</u> | 58.83/92.23 | <u>73.26</u>/<u>85.69</u>/<u>68.21</u>/<u>85.56</u> | 81.1/81.2 | **35.28/35.25** | 8/26.66 | 80.1/79.96 |
|