philipphager
commited on
Commit
•
51fb168
1
Parent(s):
925ce6c
Update README.md
Browse files
README.md
CHANGED
@@ -10,4 +10,19 @@ metrics:
|
|
10 |
- dcg@10
|
11 |
- ndcg@10
|
12 |
- mrr@10
|
13 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
10 |
- dcg@10
|
11 |
- ndcg@10
|
12 |
- mrr@10
|
13 |
+
---
|
14 |
+
|
15 |
+
# Naive Listwise MonoBERT trained on Baidu-ULTR
|
16 |
+
A flax-based MonoBERT cross encoder trained on the Baidu-ULTR dataset with a **listwise softmax cross-entropy loss on clicks**. The loss is called "naive" as we use user clicks as a signal of relevance without any additional position bias correction. For more info, read our paper here.
|
17 |
+
|
18 |
+
## Usage
|
19 |
+
```
|
20 |
+
|
21 |
+
```
|
22 |
+
|
23 |
+
## Test Results on Baidu-ULTR Expert Annotations
|
24 |
+
|
25 |
+
| Model | log-likelihood | DCG@1 | DCG@3 | DCG@5 | DCG@10 | nDCG@10 | MRR@10 |
|
26 |
+
|---------------------|----------------|--------|--------|--------|--------|---------|--------|
|
27 |
+
| Naive Pointwise | 0.2272 | 1.6836 | 3.5616 | 4.8822 | 7.4244 | 0.3640 | 0.6096 |
|
28 |
+
| **Naive Listwise** | - | 1.9738 | 4.1609 | 5.6861 | 8.5432 | 0.4091 | 0.6436 |
|