Transformers
Safetensors
bert
Inference Endpoints
philipphager commited on
Commit
51fb168
1 Parent(s): 925ce6c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -1
README.md CHANGED
@@ -10,4 +10,19 @@ metrics:
10
  - dcg@10
11
  - ndcg@10
12
  - mrr@10
13
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
  - dcg@10
11
  - ndcg@10
12
  - mrr@10
13
+ ---
14
+
15
+ # Naive Listwise MonoBERT trained on Baidu-ULTR
16
+ A flax-based MonoBERT cross encoder trained on the Baidu-ULTR dataset with a **listwise softmax cross-entropy loss on clicks**. The loss is called "naive" as we use user clicks as a signal of relevance without any additional position bias correction. For more info, read our paper here.
17
+
18
+ ## Usage
19
+ ```
20
+
21
+ ```
22
+
23
+ ## Test Results on Baidu-ULTR Expert Annotations
24
+
25
+ | Model | log-likelihood | DCG@1 | DCG@3 | DCG@5 | DCG@10 | nDCG@10 | MRR@10 |
26
+ |---------------------|----------------|--------|--------|--------|--------|---------|--------|
27
+ | Naive Pointwise | 0.2272 | 1.6836 | 3.5616 | 4.8822 | 7.4244 | 0.3640 | 0.6096 |
28
+ | **Naive Listwise** | - | 1.9738 | 4.1609 | 5.6861 | 8.5432 | 0.4091 | 0.6436 |