metadata
license: llama2
base_model: meta-llama/Llama-2-7b-hf
tags:
- trl
- sft
- generated_from_trainer
model-index:
- name: llama-3-8b-local-definitivo
results: []
llama-3-8b-local-definitivo
This model is a fine-tuned version of meta-llama/Llama-2-7b-hf on the None dataset. It achieves the following results on the evaluation set:
- Loss: 1.4658
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 2
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 4
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 10
- num_epochs: 1
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 1.893 | 0.03 | 25 | 1.7774 |
| 1.714 | 0.05 | 50 | 1.7412 |
| 1.7498 | 0.08 | 75 | 1.7188 |
| 1.6899 | 0.1 | 100 | 1.7036 |
| 1.7176 | 0.13 | 125 | 1.6906 |
| 1.7198 | 0.15 | 150 | 1.6782 |
| 1.7086 | 0.18 | 175 | 1.6694 |
| 1.7236 | 0.21 | 200 | 1.6559 |
| 1.7616 | 0.23 | 225 | 1.6457 |
| 1.6756 | 0.26 | 250 | 1.6341 |
| 1.7199 | 0.28 | 275 | 1.6236 |
| 1.7121 | 0.31 | 300 | 1.6175 |
| 1.7542 | 0.34 | 325 | 1.6071 |
| 1.6884 | 0.36 | 350 | 1.5967 |
| 1.6089 | 0.39 | 375 | 1.5892 |
| 1.6556 | 0.41 | 400 | 1.5821 |
| 1.5784 | 0.44 | 425 | 1.5745 |
| 1.6317 | 0.46 | 450 | 1.5636 |
| 1.6375 | 0.49 | 475 | 1.5571 |
| 1.6772 | 0.52 | 500 | 1.5498 |
| 1.5775 | 0.54 | 525 | 1.5408 |
| 1.6307 | 0.57 | 550 | 1.5335 |
| 1.6938 | 0.59 | 575 | 1.5304 |
| 1.5764 | 0.62 | 600 | 1.5211 |
| 1.5834 | 0.64 | 625 | 1.5164 |
| 1.5483 | 0.67 | 650 | 1.5090 |
| 1.5883 | 0.7 | 675 | 1.5013 |
| 1.6268 | 0.72 | 700 | 1.4993 |
| 1.5515 | 0.75 | 725 | 1.4919 |
| 1.665 | 0.77 | 750 | 1.4882 |
| 1.6557 | 0.8 | 775 | 1.4848 |
| 1.635 | 0.83 | 800 | 1.4829 |
| 1.6689 | 0.85 | 825 | 1.4782 |
| 1.6995 | 0.88 | 850 | 1.4742 |
| 1.6312 | 0.9 | 875 | 1.4714 |
| 1.6465 | 0.93 | 900 | 1.4689 |
| 1.51 | 0.95 | 925 | 1.4667 |
| 1.5953 | 0.98 | 950 | 1.4658 |
Framework versions
- Transformers 4.34.0
- Pytorch 2.1.2+cu121
- Datasets 2.12.0
- Tokenizers 0.14.1