Transformers
Safetensors
bert
Inference Endpoints
philipphager commited on
Commit
79b73b4
1 Parent(s): 59d07ef

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -6
README.md CHANGED
@@ -13,9 +13,18 @@ metrics:
13
  ---
14
 
15
  # Naive Listwise MonoBERT trained on Baidu-ULTR
16
- A flax-based MonoBERT cross encoder trained on the Baidu-ULTR dataset with a **listwise softmax cross-entropy loss on clicks**. The loss is called "naive" as we use user clicks as a signal of relevance without any additional position bias correction. For more info, read our paper here.
 
 
 
 
 
 
 
17
 
18
  ## Usage
 
 
19
  ```Python
20
  import jax.numpy as jnp
21
 
@@ -58,9 +67,5 @@ outputs = model(batch)
58
  print(outputs)
59
  ```
60
 
61
- ## Test Results on Baidu-ULTR Expert Annotations
62
 
63
- | Model | log-likelihood | DCG@1 | DCG@3 | DCG@5 | DCG@10 | nDCG@10 | MRR@10 |
64
- |---------------------|----------------|--------|--------|--------|--------|---------|--------|
65
- | Naive Pointwise | 0.2272 | 1.6836 | 3.5616 | 4.8822 | 7.4244 | 0.3640 | 0.6096 |
66
- | **Naive Listwise** | - | 1.9738 | 4.1609 | 5.6861 | 8.5432 | 0.4091 | 0.6436 |
 
13
  ---
14
 
15
  # Naive Listwise MonoBERT trained on Baidu-ULTR
16
+ A flax-based MonoBERT cross encoder trained on the [Baidu-ULTR](https://arxiv.org/abs/2207.03051) dataset with a **listwise softmax cross-entropy loss on clicks**. The loss is called "naive" as we use user clicks as a signal of relevance without any additional position bias correction. For more info, [read our paper here](https://arxiv.org/abs/2404.02543).
17
+
18
+ ## Test Results on Baidu-ULTR Expert Annotations
19
+
20
+ | Model | log-likelihood | DCG@1 | DCG@3 | DCG@5 | DCG@10 | nDCG@10 | MRR@10 |
21
+ |---------------------|----------------|--------|--------|--------|--------|---------|--------|
22
+ | Naive Pointwise | 0.2272 | 1.6836 | 3.5616 | 4.8822 | 7.4244 | 0.3640 | 0.6096 |
23
+ | **Naive Listwise** | - | 1.9738 | 4.1609 | 5.6861 | 8.5432 | 0.4091 | 0.6436 |
24
 
25
  ## Usage
26
+ Here is an example with a mock input batch for how to download and call the model:
27
+
28
  ```Python
29
  import jax.numpy as jnp
30
 
 
67
  print(outputs)
68
  ```
69
 
70
+ For more details on how to use the model with real data from Baidu-ULTR, take a look at the [evaluation script of our model repository](https://github.com/philipphager/baidu-bert-model/blob/main/eval.py).
71