philipphager
commited on
Commit
•
044ea52
1
Parent(s):
9679a8a
Update README.md
Browse files
README.md
CHANGED
@@ -21,7 +21,8 @@ A flax-based MonoBERT cross encoder trained on the [Baidu-ULTR](https://arxiv.or
|
|
21 |
|-----------------|----------------|--------|--------|--------|--------|---------|--------|
|
22 |
| Pointwise Naive | 0.2272 | 1.6836 | 3.5616 | 4.8822 | 7.4244 | 0.3640 | 0.6096 |
|
23 |
| Pointwise IPS | 0.2436 | 0.8842 | 2.0510 | 2.9535 | 4.8816 | 0.2363 | 0.4472 |
|
24 |
-
| **Naive
|
|
|
25 |
|
26 |
## Usage
|
27 |
Here is an example of downloading the model and calling it for inference on a mock batch of input data. For more details on how to use the model on the Baidu-ULTR dataset, take a look at our [training](https://github.com/philipphager/baidu-bert-model/blob/main/main.py) and [evaluation scripts](https://github.com/philipphager/baidu-bert-model/blob/main/eval.py) in our code repository.
|
@@ -47,7 +48,7 @@ batch = {
|
|
47 |
[2, 21448, 21874, 21436, 1, 16794, 4522, 2082],
|
48 |
[2, 21448, 21874, 21436, 1, 20206, 10082, 9773],
|
49 |
[2, 21448, 21874, 21436, 1, 2618, 8520, 2860],
|
50 |
-
|
51 |
# Specify if a token id belongs to the query (0) or document (1)
|
52 |
"token_types": jnp.array([
|
53 |
[0, 0, 0, 0, 1, 1, 1, 1],
|
|
|
21 |
|-----------------|----------------|--------|--------|--------|--------|---------|--------|
|
22 |
| Pointwise Naive | 0.2272 | 1.6836 | 3.5616 | 4.8822 | 7.4244 | 0.3640 | 0.6096 |
|
23 |
| Pointwise IPS | 0.2436 | 0.8842 | 2.0510 | 2.9535 | 4.8816 | 0.2363 | 0.4472 |
|
24 |
+
| **Listwise Naive** | **0.7535** | **1.9738** | **4.1609** | **5.6861** | **8.5432** | **0.4091** | **0.6436** |
|
25 |
+
| Listwise IPS | 1.2193 | 1.7466 | 3.6378 | 4.9797 | 7.5790 | 0.3665 | 0.6112 |
|
26 |
|
27 |
## Usage
|
28 |
Here is an example of downloading the model and calling it for inference on a mock batch of input data. For more details on how to use the model on the Baidu-ULTR dataset, take a look at our [training](https://github.com/philipphager/baidu-bert-model/blob/main/main.py) and [evaluation scripts](https://github.com/philipphager/baidu-bert-model/blob/main/eval.py) in our code repository.
|
|
|
48 |
[2, 21448, 21874, 21436, 1, 16794, 4522, 2082],
|
49 |
[2, 21448, 21874, 21436, 1, 20206, 10082, 9773],
|
50 |
[2, 21448, 21874, 21436, 1, 2618, 8520, 2860],
|
51 |
+
]),
|
52 |
# Specify if a token id belongs to the query (0) or document (1)
|
53 |
"token_types": jnp.array([
|
54 |
[0, 0, 0, 0, 1, 1, 1, 1],
|