Evaluation results for devpranjal/deberta-v3-base-devrev-data model as a base model for other tasks
#1
by
eladven
- opened
README.md
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# devpranjal/deberta-v3-base-devrev-data model
|
| 2 |
+
This model is based on microsoft/deberta-v3-base pretrained model.
|
| 3 |
+
|
| 4 |
+
|
| 5 |
+
## Model Recycling
|
| 6 |
+
|
| 7 |
+
[Evaluation on 36 datasets](https://ibm.github.io/model-recycling/model_gain_chart?avg=0.54&mnli_lp=nan&20_newsgroup=-0.92&ag_news=-0.51&amazon_reviews_multi=-0.14&anli=-0.59&boolq=2.27&cb=5.36&cola=-0.28&copa=11.60&dbpedia=-0.40&esnli=-0.74&financial_phrasebank=3.22&imdb=-0.48&isear=-0.61&mnli=-0.30&mrpc=0.75&multirc=1.38&poem_sentiment=-3.08&qnli=0.30&qqp=0.24&rotten_tomatoes=-0.18&rte=1.77&sst2=-0.34&sst_5bins=0.66&stsb=1.16&trec_coarse=-0.56&trec_fine=0.38&tweet_ev_emoji=0.18&tweet_ev_emotion=-0.70&tweet_ev_hate=1.46&tweet_ev_irony=-2.40&tweet_ev_offensive=0.87&tweet_ev_sentiment=-0.87&wic=-0.37&wnli=1.62&wsc=-0.63&yahoo_answers=0.47&model_name=devpranjal%2Fdeberta-v3-base-devrev-data&base_name=microsoft%2Fdeberta-v3-base) using devpranjal/deberta-v3-base-devrev-data as a base model yields average score of 79.58 in comparison to 79.04 by microsoft/deberta-v3-base.
|
| 8 |
+
|
| 9 |
+
The model is ranked 3rd among all tested models for the microsoft/deberta-v3-base architecture as of 07/02/2023
|
| 10 |
+
Results:
|
| 11 |
+
|
| 12 |
+
| 20_newsgroup | ag_news | amazon_reviews_multi | anli | boolq | cb | cola | copa | dbpedia | esnli | financial_phrasebank | imdb | isear | mnli | mrpc | multirc | poem_sentiment | qnli | qqp | rotten_tomatoes | rte | sst2 | sst_5bins | stsb | trec_coarse | trec_fine | tweet_ev_emoji | tweet_ev_emotion | tweet_ev_hate | tweet_ev_irony | tweet_ev_offensive | tweet_ev_sentiment | wic | wnli | wsc | yahoo_answers |
|
| 13 |
+
|---------------:|----------:|-----------------------:|--------:|--------:|--------:|--------:|-------:|----------:|--------:|-----------------------:|-------:|--------:|--------:|-------:|----------:|-----------------:|--------:|--------:|------------------:|--------:|--------:|------------:|--------:|--------------:|------------:|-----------------:|-------------------:|----------------:|-----------------:|---------------------:|---------------------:|--------:|-------:|--------:|----------------:|
|
| 14 |
+
| 85.4886 | 89.9333 | 66.72 | 58.1875 | 85.2599 | 80.3571 | 86.2895 | 70 | 79.0333 | 91.1849 | 87.7 | 94.012 | 71.2516 | 89.4833 | 89.951 | 63.6345 | 83.6538 | 93.8129 | 92.0257 | 90.2439 | 84.1155 | 94.7248 | 57.6471 | 91.4423 | 97.2 | 91.4 | 46.374 | 83.2512 | 57.6768 | 77.4235 | 85.9302 | 70.9297 | 70.8464 | 71.831 | 63.4615 | 72.5 |
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
For more information, see: [Model Recycling](https://ibm.github.io/model-recycling/)
|