readme: update leaderboard and introduce new LeTemps benchmarks
Browse files
README.md
CHANGED
@@ -26,21 +26,21 @@ More details can be found in [our GitHub repository](https://github.com/stefan-i
|
|
26 |
We test our pretrained language models on various datasets from HIPE-2020, HIPE-2022 and Europeana. The following table
|
27 |
shows an overview of used datasets.
|
28 |
|
29 |
-
| Language |
|
30 |
-
|
31 |
-
| English | [AjMC](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-ajmc.md)
|
32 |
-
| German | [AjMC](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-ajmc.md)
|
33 |
-
| French | [AjMC](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-ajmc.md)
|
34 |
-
| Finnish | [NewsEye](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-newseye.md)
|
35 |
-
| Swedish | [NewsEye](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-newseye.md)
|
36 |
-
| Dutch | [ICDAR-Europeana](https://github.com/stefan-it/historic-domain-adaptation-icdar)
|
37 |
|
38 |
Results:
|
39 |
|
40 |
-
| Model | English AjMC | German AjMC | French AjMC | Finnish NewsEye | Swedish NewsEye | Dutch ICDAR | French ICDAR | Avg. |
|
41 |
-
|
42 |
-
| hmBERT (32k) [Schweter et al.](https://ceur-ws.org/Vol-3180/paper-87.pdf) | 85.36 ± 0.94 | 89.08 ± 0.09 | 85.10 ± 0.60 | 77.28 ± 0.37 | 82.85 ± 0.83 | 82.11 ± 0.61 | 77.21 ± 0.16 |
|
43 |
-
| hmTEAMS (Ours) | 86.41 ± 0.36 | 88.64 ± 0.42 | 85.41 ± 0.67 | 79.27 ± 1.88 | 82.78 ± 0.60 | 88.21 ± 0.39 | 78.03 ± 0.39 | **
|
44 |
|
45 |
# Acknowledgements
|
46 |
|
|
|
26 |
We test our pretrained language models on various datasets from HIPE-2020, HIPE-2022 and Europeana. The following table
|
27 |
shows an overview of used datasets.
|
28 |
|
29 |
+
| Language | Datasets
|
30 |
+
|----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
31 |
+
| English | [AjMC](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-ajmc.md) |
|
32 |
+
| German | [AjMC](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-ajmc.md) |
|
33 |
+
| French | [AjMC](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-ajmc.md) - [ICDAR-Europeana](https://github.com/stefan-it/historic-domain-adaptation-icdar) - [LeTemps](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-letemps.md) |
|
34 |
+
| Finnish | [NewsEye](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-newseye.md) |
|
35 |
+
| Swedish | [NewsEye](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-newseye.md) |
|
36 |
+
| Dutch | [ICDAR-Europeana](https://github.com/stefan-it/historic-domain-adaptation-icdar) |
|
37 |
|
38 |
Results:
|
39 |
|
40 |
+
| Model | English AjMC | German AjMC | French AjMC | Finnish NewsEye | Swedish NewsEye | Dutch ICDAR | French ICDAR | French LeTemps | Avg. |
|
41 |
+
|----------------------------------------------------------------------------------------|--------------|--------------|--------------|-----------------|-----------------|--------------|--------------|----------------|-----------|
|
42 |
+
| hmBERT (32k) [Schweter et al.](https://ceur-ws.org/Vol-3180/paper-87.pdf) | 85.36 ± 0.94 | 89.08 ± 0.09 | 85.10 ± 0.60 | 77.28 ± 0.37 | 82.85 ± 0.83 | 82.11 ± 0.61 | 77.21 ± 0.16 | 65.73 ± 0.56 | 80.59 |
|
43 |
+
| hmTEAMS (Ours) | 86.41 ± 0.36 | 88.64 ± 0.42 | 85.41 ± 0.67 | 79.27 ± 1.88 | 82.78 ± 0.60 | 88.21 ± 0.39 | 78.03 ± 0.39 | 66.71 ± 0.46 | **81.93** |
|
44 |
|
45 |
# Acknowledgements
|
46 |
|