Initial Commit

README.md

@@ -0,0 +1,78 @@
+---
+license: mit
+base_model: microsoft/mdeberta-v3-base
+tags:
+- generated_from_trainer
+metrics:
+- accuracy
+- f1
+model-index:
+- name: scenario-NON-KD-PR-COPY-CDF-EN-D2_data-en-cardiff_eng_only44
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# scenario-NON-KD-PR-COPY-CDF-EN-D2_data-en-cardiff_eng_only44
+
+This model is a fine-tuned version of [microsoft/mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 5.3429
+- Accuracy: 0.4330
+- F1: 0.4179
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-05
+- train_batch_size: 32
+- eval_batch_size: 32
+- seed: 44
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 30
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1     |
+|:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|
+| No log        | 1.72  | 100  | 1.1160          | 0.4074   | 0.3433 |
+| No log        | 3.45  | 200  | 1.2658          | 0.4149   | 0.3707 |
+| No log        | 5.17  | 300  | 1.7230          | 0.4599   | 0.4582 |
+| No log        | 6.9   | 400  | 1.7742          | 0.4383   | 0.4266 |
+| 0.6464        | 8.62  | 500  | 2.4181          | 0.4334   | 0.4215 |
+| 0.6464        | 10.34 | 600  | 2.9688          | 0.4444   | 0.4397 |
+| 0.6464        | 12.07 | 700  | 3.2008          | 0.4374   | 0.4342 |
+| 0.6464        | 13.79 | 800  | 4.2300          | 0.4233   | 0.4012 |
+| 0.6464        | 15.52 | 900  | 4.2439          | 0.4414   | 0.4382 |
+| 0.0996        | 17.24 | 1000 | 4.6596          | 0.4506   | 0.4385 |
+| 0.0996        | 18.97 | 1100 | 4.7742          | 0.4343   | 0.4252 |
+| 0.0996        | 20.69 | 1200 | 4.9060          | 0.4440   | 0.4336 |
+| 0.0996        | 22.41 | 1300 | 5.0117          | 0.4365   | 0.4288 |
+| 0.0996        | 24.14 | 1400 | 5.0594          | 0.4444   | 0.4348 |
+| 0.0222        | 25.86 | 1500 | 5.1602          | 0.4440   | 0.4351 |
+| 0.0222        | 27.59 | 1600 | 5.2402          | 0.4405   | 0.4325 |
+| 0.0222        | 29.31 | 1700 | 5.3429          | 0.4330   | 0.4179 |
+
+
+### Framework versions
+
+- Transformers 4.33.3
+- Pytorch 2.1.1+cu121
+- Datasets 2.14.5
+- Tokenizers 0.13.3

config.json

@@ -0,0 +1,45 @@
+{
+  "_name_or_path": "microsoft/mdeberta-v3-base",
+  "architectures": [
+    "DebertaForSequenceClassificationKD"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "LABEL_0",
+    "1": "LABEL_1",
+    "2": "LABEL_2"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "LABEL_0": 0,
+    "LABEL_1": 1,
+    "LABEL_2": 2
+  },
+  "layer_norm_eps": 1e-07,
+  "max_position_embeddings": 512,
+  "max_relative_positions": -1,
+  "model_type": "deberta-v2",
+  "norm_rel_ebd": "layer_norm",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 6,
+  "pad_token_id": 0,
+  "pooler_dropout": 0,
+  "pooler_hidden_act": "gelu",
+  "pooler_hidden_size": 768,
+  "pos_att_type": [
+    "p2c",
+    "c2p"
+  ],
+  "position_biased_input": false,
+  "position_buckets": 256,
+  "relative_attention": true,
+  "share_att_key": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.33.3",
+  "type_vocab_size": 0,
+  "vocab_size": 251000
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f14543a2780ba14a8832b9247686cd172a71125615144d298bd90be787e48e6
+size 946740394

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b01979b4dd2e54f66f1736fe4facfa31a436cbed55e0d4714215ed588f696e2
+size 4600