AliMGH commited on
Commit
8503845
·
verified ·
1 Parent(s): 172b49b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -4
README.md CHANGED
@@ -14,7 +14,7 @@ widget:
14
  ---
15
  # Model Details
16
 
17
- TukaBERT models are a family of encoder models trained on Persian in two sizes base and large. These Models pre-trained on over 300GB of Persian data including a variety of topics such as News, Blogs, Forums, Books, etc. They pre-trained with the MLM (WWM) objective using two context lengths.
18
 
19
  ## How to use
20
 
@@ -45,15 +45,18 @@ inference_pipeline("شهر برلین در کشور <mask> واقع شده اس
45
 
46
  You can use this model to fine-tune it over your dataset and prepare it for your task.
47
 
 
 
 
48
  ## Evaluation
49
 
50
- TukaBERT models are evaluated on a wide range of NLP downstream tasks, such as Sentiment Analysis (SA), Text Classification, Multiple-choice, Question Answering, and Named Entity Recognition (NER).
51
  Here are some key performance results:
52
 
53
  | Model name | DeepSentiPers (f1/acc) | MultiCoNER-v2 (f1/acc) | PQuAD (best_exact/best_f1/HasAns_exact/HasAns_f1) | FarsTail (f1/acc) | ParsiNLU-Multiple-choice (f1/acc) | ParsiNLU-Reading-comprehension (exact/f1) | ParsiNLU-QQP (f1/acc) |
54
  |------------------|------------------------|------------------------|-----------------------------------------------------|--------------------|-----------------------------------|-------------------------------------------|-----------------------|
55
- | TukaBERT-large | **85.66/85.78** | **69.69/94.07** | **75.56/88.06/70.24/87.83** | **89.71/89.72** | **36.13/35.97** | **33.6/60.5** | **82.72/82.63** |
56
- | TukaBERT-base | <u>83.93/83.93</u> | <u>66.23/93.3</u> | <u>73.18</u>/<u>85.71</u>/<u>68.29</u>/<u>85.94</u> | <u>83.26/83.41</u> | 33.6/<u>33.81</u> | 20.8/42.52 | <u>81.33/81.29</u> |
57
  | Shiraz | 81.17/81.08 | 59.1/92.83 | 65.96/81.25/59.63/81.31 | 77.76/77.75 | <u>34.73/34.53</u> | 17.6/39.61 | 79.68/79.51 |
58
  | ParsBERT | 80.22/80.23 | 64.91/93.23 | 71.41/84.21/66.29/84.57 | 80.89/80.94 | **35.34/35.25** | 20/39.58 | 80.15/80.07 |
59
  | XLM-V-base | <u>83.43/83.36</u> | 58.83/92.23 | <u>73.26</u>/<u>85.69</u>/<u>68.21</u>/<u>85.56</u> | 81.1/81.2 | **35.28/35.25** | 8/26.66 | 80.1/79.96 |
 
14
  ---
15
  # Model Details
16
 
17
+ TookaBERT models are a family of encoder models trained on Persian in two sizes base and large. These Models pre-trained on over 300GB of Persian data including a variety of topics such as News, Blogs, Forums, Books, etc. They pre-trained with the MLM (WWM) objective using two context lengths.
18
 
19
  ## How to use
20
 
 
45
 
46
  You can use this model to fine-tune it over your dataset and prepare it for your task.
47
 
48
+ - DeepSentiPers (Sentiment Analysis) <a href="https://colab.research.google.com/drive/1Vn5QTYutdCo6iXVTmsPW9K4t8xVk14ji#scrollTo=1B1YrypZxajF"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Colab Code" width="87" height="15"/></a>
49
+ - ParsiNLU - Multiple-choice (Multiple-choice) <a href="https://colab.research.google.com/drive/1boXMnRIwqAYGU7oxJtRjgib7Fu-O--x5#scrollTo=7jVb9E4SDPNb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Colab Code" width="87" height="15"/></a>
50
+
51
  ## Evaluation
52
 
53
+ TookaBERT models are evaluated on a wide range of NLP downstream tasks, such as Sentiment Analysis (SA), Text Classification, Multiple-choice, Question Answering, and Named Entity Recognition (NER).
54
  Here are some key performance results:
55
 
56
  | Model name | DeepSentiPers (f1/acc) | MultiCoNER-v2 (f1/acc) | PQuAD (best_exact/best_f1/HasAns_exact/HasAns_f1) | FarsTail (f1/acc) | ParsiNLU-Multiple-choice (f1/acc) | ParsiNLU-Reading-comprehension (exact/f1) | ParsiNLU-QQP (f1/acc) |
57
  |------------------|------------------------|------------------------|-----------------------------------------------------|--------------------|-----------------------------------|-------------------------------------------|-----------------------|
58
+ | TookaBERT-large | **85.66/85.78** | **69.69/94.07** | **75.56/88.06/70.24/87.83** | **89.71/89.72** | **36.13/35.97** | **33.6/60.5** | **82.72/82.63** |
59
+ | TookaBERT-base | <u>83.93/83.93</u> | <u>66.23/93.3</u> | <u>73.18</u>/<u>85.71</u>/<u>68.29</u>/<u>85.94</u> | <u>83.26/83.41</u> | 33.6/<u>33.81</u> | 20.8/42.52 | <u>81.33/81.29</u> |
60
  | Shiraz | 81.17/81.08 | 59.1/92.83 | 65.96/81.25/59.63/81.31 | 77.76/77.75 | <u>34.73/34.53</u> | 17.6/39.61 | 79.68/79.51 |
61
  | ParsBERT | 80.22/80.23 | 64.91/93.23 | 71.41/84.21/66.29/84.57 | 80.89/80.94 | **35.34/35.25** | 20/39.58 | 80.15/80.07 |
62
  | XLM-V-base | <u>83.43/83.36</u> | 58.83/92.23 | <u>73.26</u>/<u>85.69</u>/<u>68.21</u>/<u>85.56</u> | 81.1/81.2 | **35.28/35.25** | 8/26.66 | 80.1/79.96 |