Model save

README.md

@@ -0,0 +1,86 @@
+---
+library_name: transformers
+license: other
+base_model: trl-lib/qwen1.5-0.5b-sft
+tags:
+- trl
+- simpo
+- generated_from_trainer
+model-index:
+- name: qwen_cUNL_entropy
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# qwen_cUNL_entropy
+
+This model is a fine-tuned version of [trl-lib/qwen1.5-0.5b-sft](https://huggingface.co/trl-lib/qwen1.5-0.5b-sft) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.5443
+- Rewards/chosen: -10.4574
+- Rewards/rejected: -11.6657
+- Rewards/accuracies: 0.7211
+- Rewards/margins: 1.2083
+- Logps/rejected: -11.6657
+- Logps/chosen: -10.4574
+- Logits/rejected: 1.7006
+- Logits/chosen: 1.6599
+- Semantic Entropy: 0.0009
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 1e-06
+- train_batch_size: 2
+- eval_batch_size: 4
+- seed: 42
+- distributed_type: multi-GPU
+- gradient_accumulation_steps: 16
+- total_train_batch_size: 32
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_ratio: 0.1
+- num_epochs: 3.0
+
+### Training results
+
+| Training Loss | Epoch  | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen | Semantic Entropy |
+|:-------------:|:------:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|:----------------:|
+| 1.0568        | 0.2141 | 400  | 1.0641          | -2.1120        | -2.3958          | 0.5497             | 0.2838          | -2.3958        | -2.1120      | 0.4624          | 0.3653        | 0.5089           |
+| 0.602         | 0.4282 | 800  | 0.5817          | -7.8264        | -8.3563          | 0.6855             | 0.5299          | -8.3563        | -7.8264      | 1.3572          | 1.3145        | 0.0047           |
+| 0.6143        | 0.6422 | 1200 | 0.5509          | -8.8961        | -9.6072          | 0.6996             | 0.7110          | -9.6072        | -8.8961      | 1.6397          | 1.5818        | 0.0021           |
+| 0.4586        | 0.8563 | 1600 | 0.5386          | -9.1154        | -9.8342          | 0.7166             | 0.7188          | -9.8342        | -9.1154      | 1.6747          | 1.6254        | 0.0017           |
+| 0.5717        | 1.0704 | 2000 | 0.5367          | -9.0664        | -9.8093          | 0.7151             | 0.7429          | -9.8093        | -9.0664      | 1.6674          | 1.6192        | 0.0019           |
+| 0.4984        | 1.2845 | 2400 | 0.5360          | -9.2711        | -10.2641         | 0.7307             | 0.9930          | -10.2641       | -9.2711      | 1.8156          | 1.7480        | 0.0018           |
+| 0.4938        | 1.4986 | 2800 | 0.5318          | -8.7814        | -9.7107          | 0.7255             | 0.9293          | -9.7107        | -8.7814      | 1.5667          | 1.5074        | 0.0028           |
+| 0.5379        | 1.7127 | 3200 | 0.5329          | -9.7086        | -10.6885         | 0.7151             | 0.9799          | -10.6885       | -9.7086      | 1.6242          | 1.5778        | 0.0014           |
+| 0.5434        | 1.9267 | 3600 | 0.5306          | -9.6408        | -10.6193         | 0.7211             | 0.9784          | -10.6193       | -9.6408      | 1.5137          | 1.4731        | 0.0014           |
+| 0.3725        | 2.1408 | 4000 | 0.5395          | -10.2459       | -11.3888         | 0.7240             | 1.1430          | -11.3888       | -10.2459     | 1.6740          | 1.6276        | 0.0011           |
+| 0.4469        | 2.3549 | 4400 | 0.5429          | -10.1108       | -11.3001         | 0.7218             | 1.1893          | -11.3001       | -10.1108     | 1.6134          | 1.5684        | 0.0012           |
+| 0.3893        | 2.5690 | 4800 | 0.5416          | -10.2988       | -11.4914         | 0.7240             | 1.1926          | -11.4914       | -10.2988     | 1.6694          | 1.6281        | 0.0010           |
+| 0.4342        | 2.7831 | 5200 | 0.5442          | -10.3998       | -11.6056         | 0.7226             | 1.2058          | -11.6056       | -10.3998     | 1.7148          | 1.6724        | 0.0010           |
+| 0.3904        | 2.9972 | 5600 | 0.5443          | -10.4574       | -11.6657         | 0.7211             | 1.2083          | -11.6657       | -10.4574     | 1.7006          | 1.6599        | 0.0009           |
+
+
+### Framework versions
+
+- Transformers 4.44.2
+- Pytorch 2.2.2+cu121
+- Datasets 2.18.0
+- Tokenizers 0.19.1

all_results.json

@@ -0,0 +1,9 @@
+{
+    "epoch": 2.999297541394882,
+    "total_flos": 0.0,
+    "train_loss": 0.541753981451236,
+    "train_runtime": 28781.2396,
+    "train_samples": 59790,
+    "train_samples_per_second": 6.232,
+    "train_steps_per_second": 0.195
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "bos_token_id": 151643,
+  "eos_token_id": 151643,
+  "max_new_tokens": 2048,
+  "transformers_version": "4.44.2"
+}

train_results.json

@@ -0,0 +1,9 @@
+{
+    "epoch": 2.999297541394882,
+    "total_flos": 0.0,
+    "train_loss": 0.541753981451236,
+    "train_runtime": 28781.2396,
+    "train_samples": 59790,
+    "train_samples_per_second": 6.232,
+    "train_steps_per_second": 0.195
+}