Update README.md
Browse files
README.md
CHANGED
@@ -16,6 +16,15 @@ should probably proofread and complete it, then remove this comment. -->
|
|
16 |
|
17 |
# bart-large-summary-map-reduce-1024
|
18 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
19 |
This model is a fine-tuned version of [facebook/bart-large](https://huggingface.co/facebook/bart-large) on the pszemraj/summary-map-reduce dataset.
|
20 |
It achieves the following results on the evaluation set:
|
21 |
- Loss: 0.7894
|
@@ -75,23 +84,3 @@ The following hyperparameters were used during training:
|
|
75 |
- lr_scheduler_type: cosine
|
76 |
- lr_scheduler_warmup_ratio: 0.05
|
77 |
- num_epochs: 3.0
|
78 |
-
|
79 |
-
### Training results
|
80 |
-
|
81 |
-
| Training Loss | Epoch | Step | Validation Loss | Input Tokens Seen |
|
82 |
-
|:-------------:|:------:|:----:|:---------------:|:-----------------:|
|
83 |
-
| 1.0645 | 0.3834 | 100 | 0.9265 | 1844404 |
|
84 |
-
| 1.0769 | 0.7668 | 200 | 0.8621 | 3640408 |
|
85 |
-
| 0.849 | 1.1503 | 300 | 0.8502 | 5504644 |
|
86 |
-
| 0.8612 | 1.5337 | 400 | 0.8289 | 7316212 |
|
87 |
-
| 0.7934 | 1.9171 | 500 | 0.8072 | 9167936 |
|
88 |
-
| 0.6701 | 2.3005 | 600 | 0.8051 | 10969348 |
|
89 |
-
| 0.6579 | 2.6839 | 700 | 0.7903 | 12814620 |
|
90 |
-
|
91 |
-
|
92 |
-
### Framework versions
|
93 |
-
|
94 |
-
- Transformers 4.46.0.dev0
|
95 |
-
- Pytorch 2.5.1+cu124
|
96 |
-
- Datasets 3.1.0
|
97 |
-
- Tokenizers 0.20.2
|
|
|
16 |
|
17 |
# bart-large-summary-map-reduce-1024
|
18 |
|
19 |
+
A text2text model to "map-reduce" summaries of a chunked long document into one.
|
20 |
+
|
21 |
+
An explanation of this model's role:
|
22 |
+
|
23 |
+

|
24 |
+
|
25 |
+
<small> modified flowchart from Google's blog [here](https://cloud.google.com/blog/products/ai-machine-learning/long-document-summarization-with-workflows-and-gemini-models) </small>
|
26 |
+
|
27 |
+
## Details
|
28 |
This model is a fine-tuned version of [facebook/bart-large](https://huggingface.co/facebook/bart-large) on the pszemraj/summary-map-reduce dataset.
|
29 |
It achieves the following results on the evaluation set:
|
30 |
- Loss: 0.7894
|
|
|
84 |
- lr_scheduler_type: cosine
|
85 |
- lr_scheduler_warmup_ratio: 0.05
|
86 |
- num_epochs: 3.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|