Commit
•
c927baa
1
Parent(s):
3e8cb87
Update README.md
Browse files
README.md
CHANGED
@@ -57,15 +57,17 @@ print(output)
|
|
57 |
### Details on data and training
|
58 |
The code for preparing the data and training & evaluating the model is fully open-source here: https://github.com/MoritzLaurer/zeroshot-classifier/tree/main
|
59 |
|
|
|
|
|
60 |
## Metrics
|
61 |
|
62 |
-
Balanced accuracy
|
63 |
`deberta-v3-base-zeroshot-v1.1-all-33` was trained on all datasets, with only maximum 500 texts per class to avoid overfitting.
|
64 |
-
The metrics on these datasets are therefore not strictly zeroshot, as the model has seen some data for each task.
|
65 |
`deberta-v3-base-zeroshot-v1.1-heldout` indicates zeroshot performance on the respective dataset.
|
66 |
To calculate these zeroshot metrics, the pipeline was run 28 times, each time with one dataset held out from training to simulate a zeroshot setup.
|
67 |
|
68 |
-
![figure_base_v1.1](https://
|
69 |
|
70 |
| | deberta-v3-base-mnli-fever-anli-ling-wanli-binary | deberta-v3-base-zeroshot-v1.1-heldout | deberta-v3-base-zeroshot-v1.1-all-33 |
|
71 |
|:---------------------------|---------------------------:|----------------------------------------:|---------------------------------------:|
|
|
|
57 |
### Details on data and training
|
58 |
The code for preparing the data and training & evaluating the model is fully open-source here: https://github.com/MoritzLaurer/zeroshot-classifier/tree/main
|
59 |
|
60 |
+
Hyperparameters and other details are available in this Weights & Biases repo: https://wandb.ai/moritzlaurer/deberta-v3-base-zeroshot-v1-1-all-33/table?workspace=user-
|
61 |
+
|
62 |
## Metrics
|
63 |
|
64 |
+
Balanced accuracy is reported for all datasets.
|
65 |
`deberta-v3-base-zeroshot-v1.1-all-33` was trained on all datasets, with only maximum 500 texts per class to avoid overfitting.
|
66 |
+
The metrics on these datasets are therefore not strictly zeroshot, as the model has seen some data for each task during training.
|
67 |
`deberta-v3-base-zeroshot-v1.1-heldout` indicates zeroshot performance on the respective dataset.
|
68 |
To calculate these zeroshot metrics, the pipeline was run 28 times, each time with one dataset held out from training to simulate a zeroshot setup.
|
69 |
|
70 |
+
![figure_base_v1.1](https://raw.githubusercontent.com/MoritzLaurer/zeroshot-classifier/main/results/fig_base_v1.1.png)
|
71 |
|
72 |
| | deberta-v3-base-mnli-fever-anli-ling-wanli-binary | deberta-v3-base-zeroshot-v1.1-heldout | deberta-v3-base-zeroshot-v1.1-all-33 |
|
73 |
|:---------------------------|---------------------------:|----------------------------------------:|---------------------------------------:|
|