stefan-it commited on
Commit
d4d7916
·
1 Parent(s): e51fdb1

readme: update leaderboard and introduce new LeTemps benchmarks

Browse files
Files changed (1) hide show
  1. README.md +12 -12
README.md CHANGED
@@ -26,21 +26,21 @@ More details can be found in [our GitHub repository](https://github.com/stefan-i
26
  We test our pretrained language models on various datasets from HIPE-2020, HIPE-2022 and Europeana. The following table
27
  shows an overview of used datasets.
28
 
29
- | Language | Dataset | Additional Dataset |
30
- |----------|--------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------|
31
- | English | [AjMC](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-ajmc.md) | - |
32
- | German | [AjMC](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-ajmc.md) | - |
33
- | French | [AjMC](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-ajmc.md) | [ICDAR-Europeana](https://github.com/stefan-it/historic-domain-adaptation-icdar) |
34
- | Finnish | [NewsEye](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-newseye.md) | - |
35
- | Swedish | [NewsEye](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-newseye.md) | - |
36
- | Dutch | [ICDAR-Europeana](https://github.com/stefan-it/historic-domain-adaptation-icdar) | - |
37
 
38
  Results:
39
 
40
- | Model | English AjMC | German AjMC | French AjMC | Finnish NewsEye | Swedish NewsEye | Dutch ICDAR | French ICDAR | Avg. |
41
- |----------------------------------------------------------------------------------------|--------------|--------------|--------------|-----------------|-----------------|--------------|--------------|-----------|
42
- | hmBERT (32k) [Schweter et al.](https://ceur-ws.org/Vol-3180/paper-87.pdf) | 85.36 ± 0.94 | 89.08 ± 0.09 | 85.10 ± 0.60 | 77.28 ± 0.37 | 82.85 ± 0.83 | 82.11 ± 0.61 | 77.21 ± 0.16 | 82.71 |
43
- | hmTEAMS (Ours) | 86.41 ± 0.36 | 88.64 ± 0.42 | 85.41 ± 0.67 | 79.27 ± 1.88 | 82.78 ± 0.60 | 88.21 ± 0.39 | 78.03 ± 0.39 | **84.11** |
44
 
45
  # Acknowledgements
46
 
 
26
  We test our pretrained language models on various datasets from HIPE-2020, HIPE-2022 and Europeana. The following table
27
  shows an overview of used datasets.
28
 
29
+ | Language | Datasets
30
+ |----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
31
+ | English | [AjMC](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-ajmc.md) |
32
+ | German | [AjMC](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-ajmc.md) |
33
+ | French | [AjMC](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-ajmc.md) - [ICDAR-Europeana](https://github.com/stefan-it/historic-domain-adaptation-icdar) - [LeTemps](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-letemps.md) |
34
+ | Finnish | [NewsEye](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-newseye.md) |
35
+ | Swedish | [NewsEye](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-newseye.md) |
36
+ | Dutch | [ICDAR-Europeana](https://github.com/stefan-it/historic-domain-adaptation-icdar) |
37
 
38
  Results:
39
 
40
+ | Model | English AjMC | German AjMC | French AjMC | Finnish NewsEye | Swedish NewsEye | Dutch ICDAR | French ICDAR | French LeTemps | Avg. |
41
+ |----------------------------------------------------------------------------------------|--------------|--------------|--------------|-----------------|-----------------|--------------|--------------|----------------|-----------|
42
+ | hmBERT (32k) [Schweter et al.](https://ceur-ws.org/Vol-3180/paper-87.pdf) | 85.36 ± 0.94 | 89.08 ± 0.09 | 85.10 ± 0.60 | 77.28 ± 0.37 | 82.85 ± 0.83 | 82.11 ± 0.61 | 77.21 ± 0.16 | 65.73 ± 0.56 | 80.59 |
43
+ | hmTEAMS (Ours) | 86.41 ± 0.36 | 88.64 ± 0.42 | 85.41 ± 0.67 | 79.27 ± 1.88 | 82.78 ± 0.60 | 88.21 ± 0.39 | 78.03 ± 0.39 | 66.71 ± 0.46 | **81.93** |
44
 
45
  # Acknowledgements
46