tomaarsen HF Staff commited on
Commit
b39c033
·
verified ·
1 Parent(s): 9de7761

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +21 -0
README.md CHANGED
@@ -104,6 +104,10 @@ model-index:
104
 
105
  This is a [Cross Encoder](https://www.sbert.net/docs/cross_encoder/usage/usage.html) model finetuned from [answerdotai/ModernBERT-large](https://huggingface.co/answerdotai/ModernBERT-large) using the [sentence-transformers](https://www.SBERT.net) library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.
106
 
 
 
 
 
107
  ## Model Details
108
 
109
  ### Model Description
@@ -211,6 +215,23 @@ You can finetune this model on your own dataset.
211
 
212
  #### Cross Encoder Reranking
213
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
214
  * Datasets: `NanoMSMARCO_R100`, `NanoNFCorpus_R100` and `NanoNQ_R100`
215
  * Evaluated with [<code>CrossEncoderRerankingEvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderRerankingEvaluator) with these parameters:
216
  ```json
 
104
 
105
  This is a [Cross Encoder](https://www.sbert.net/docs/cross_encoder/usage/usage.html) model finetuned from [answerdotai/ModernBERT-large](https://huggingface.co/answerdotai/ModernBERT-large) using the [sentence-transformers](https://www.SBERT.net) library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.
106
 
107
+ See [training_gooaq_bce.py](https://github.com/UKPLab/sentence-transformers/blob/feat/cross_encoder_trainer/examples/cross_encoder/training/rerankers/training_gooaq_bce.py) for the training script - only the base model was updated from [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) to [answerdotai/ModernBERT-large](https://huggingface.co/answerdotai/ModernBERT-large). This script is also described in the [Cross Encoder > Training Overview](https://sbert.net/docs/cross_encoder/training_overview.html) documentation and the [Training and Finetuning Reranker Models with Sentence Transformers v4](https://huggingface.co/blog/train-reranker) blogpost.
108
+
109
+ ![Model size vs NDCG for Rerankers on GooAQ](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/train-reranker/reranker_gooaq_model_size_ndcg.png)
110
+
111
  ## Model Details
112
 
113
  ### Model Description
 
215
 
216
  #### Cross Encoder Reranking
217
 
218
+ * Dataset: `gooaq-dev`
219
+ * Evaluated with [<code>CrossEncoderRerankingEvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderRerankingEvaluator) with these parameters:
220
+ ```json
221
+ {
222
+ "at_k": 10,
223
+ "always_rerank_positives": true
224
+ }
225
+ ```
226
+
227
+ | Metric | Value |
228
+ |:------------|:---------------------|
229
+ | map | 0.8176 (+0.2865) |
230
+ | mrr@10 | 0.8166 (+0.2926) |
231
+ | **ndcg@10** | **0.8581 (+0.2669)** |
232
+
233
+ #### Cross Encoder Reranking
234
+
235
  * Datasets: `NanoMSMARCO_R100`, `NanoNFCorpus_R100` and `NanoNQ_R100`
236
  * Evaluated with [<code>CrossEncoderRerankingEvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderRerankingEvaluator) with these parameters:
237
  ```json