kelingwang
commited on
Commit
•
000eba8
1
Parent(s):
8ae5705
Update README.md
Browse files
README.md
CHANGED
@@ -19,21 +19,18 @@ model-index:
|
|
19 |
name: rating_dr2
|
20 |
type: dataset
|
21 |
metrics:
|
22 |
-
- name: log-based ordinal loss with distance power 3.0
|
23 |
-
type: loss
|
24 |
-
value: 0.004970851354300976
|
25 |
- name: off by 1 accuracy
|
26 |
type: accuracy
|
27 |
-
value:
|
28 |
- name: mean squared error for ordinal data
|
29 |
type: mse
|
30 |
-
value: 0.
|
31 |
- name: weighted F1 score
|
32 |
type: f1
|
33 |
-
value:
|
34 |
- name: Kendall's tau coefficient
|
35 |
type: Kendall's tau
|
36 |
-
value:
|
37 |
source:
|
38 |
name: Keling Wang
|
39 |
url: https://github.com/Keling-Wang
|
@@ -59,12 +56,12 @@ This model predicts strength of causation (SoC) labels based on the text inputs
|
|
59 |
# Performance and hyperparameters
|
60 |
## Test metrics
|
61 |
This model achieves the following results on the test dataset. The test dataset is a 25% held-out stratified split of the entire dataset with `SEED=114514`.
|
62 |
-
* Loss:
|
63 |
-
* Off-by-1 accuracy:
|
64 |
-
* Off-by-2 accuracy:
|
65 |
-
* MSE for ordinal data: 0.
|
66 |
-
* Weighted F1:
|
67 |
-
* Kendall's Tau:
|
68 |
|
69 |
## Hyperparameter tuning metrics
|
70 |
This model achieves the following averaged results during 4-fold cross-validation with best hyperparameters in hyperparameter tuning process:
|
|
|
19 |
name: rating_dr2
|
20 |
type: dataset
|
21 |
metrics:
|
|
|
|
|
|
|
22 |
- name: off by 1 accuracy
|
23 |
type: accuracy
|
24 |
+
value: 74.78991596638656
|
25 |
- name: mean squared error for ordinal data
|
26 |
type: mse
|
27 |
+
value: 0.773109243697479
|
28 |
- name: weighted F1 score
|
29 |
type: f1
|
30 |
+
value: 0.76386248572931
|
31 |
- name: Kendall's tau coefficient
|
32 |
type: Kendall's tau
|
33 |
+
value: 0.8081294201575603
|
34 |
source:
|
35 |
name: Keling Wang
|
36 |
url: https://github.com/Keling-Wang
|
|
|
56 |
# Performance and hyperparameters
|
57 |
## Test metrics
|
58 |
This model achieves the following results on the test dataset. The test dataset is a 25% held-out stratified split of the entire dataset with `SEED=114514`.
|
59 |
+
* Loss: 18.2347
|
60 |
+
* Off-by-1 accuracy: 74.7899
|
61 |
+
* Off-by-2 accuracy: 91.5966
|
62 |
+
* MSE for ordinal data: 0.7731
|
63 |
+
* Weighted F1: 0.7639
|
64 |
+
* Kendall's Tau: 0.8081
|
65 |
|
66 |
## Hyperparameter tuning metrics
|
67 |
This model achieves the following averaged results during 4-fold cross-validation with best hyperparameters in hyperparameter tuning process:
|