Hugging Face's logo

kanishka
/
opt-babylm2-20-epochs_seed-42_3e-4

Text Generation
Transformers
TensorBoard
Safetensors
opt
Generated from Trainer
Eval Results
text-generation-inference
Inference Endpoints
Model card Files Metrics Community
kanishka commited on
Commit
6c2f4a4
•
1 Parent(s): 23714d1

Model save

Browse files
Files changed (2) hide show
  1. README.md +28 -39
  2. generation_config.json +1 -1
README.md CHANGED
@@ -1,23 +1,12 @@
1
  ---
 
2
  tags:
3
  - generated_from_trainer
4
- datasets:
5
- - kanishka/babylm2-sentence-tokenized
6
  metrics:
7
  - accuracy
8
  model-index:
9
  - name: opt-babylm2-20-epochs_seed-42_3e-4
10
- results:
11
- - task:
12
- name: Causal Language Modeling
13
- type: text-generation
14
- dataset:
15
- name: kanishka/babylm2-sentence-tokenized
16
- type: kanishka/babylm2-sentence-tokenized
17
- metrics:
18
- - name: Accuracy
19
- type: accuracy
20
- value: 0.5192785875103685
21
  ---
22
 
23
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -25,9 +14,9 @@ should probably proofread and complete it, then remove this comment. -->
25
 
26
  # opt-babylm2-20-epochs_seed-42_3e-4
27
 
28
- This model was trained from scratch on the kanishka/babylm2-sentence-tokenized dataset.
29
  It achieves the following results on the evaluation set:
30
- - Loss: 2.4961
31
  - Accuracy: 0.5193
32
 
33
  ## Model description
@@ -61,31 +50,31 @@ The following hyperparameters were used during training:
61
 
62
  | Training Loss | Epoch | Step | Validation Loss | Accuracy |
63
  |:-------------:|:-----:|:------:|:---------------:|:--------:|
64
- | 2.8033 | 1.0 | 21397 | 2.9065 | 0.4672 |
65
- | 2.57 | 2.0 | 42794 | 2.6933 | 0.4900 |
66
- | 2.4599 | 3.0 | 64191 | 2.6011 | 0.5007 |
67
- | 2.3967 | 4.0 | 85588 | 2.5534 | 0.5063 |
68
- | 2.3376 | 5.0 | 106985 | 2.5249 | 0.5100 |
69
- | 2.3036 | 6.0 | 128382 | 2.5098 | 0.5122 |
70
- | 2.2667 | 7.0 | 149779 | 2.4972 | 0.5142 |
71
- | 2.2441 | 8.0 | 171176 | 2.4890 | 0.5155 |
72
- | 2.2176 | 9.0 | 192573 | 2.4842 | 0.5163 |
73
- | 2.1899 | 10.0 | 213970 | 2.4832 | 0.5170 |
74
- | 2.1699 | 11.0 | 235367 | 2.4810 | 0.5177 |
75
- | 2.1554 | 12.0 | 256764 | 2.4812 | 0.5182 |
76
- | 2.131 | 13.0 | 278161 | 2.4790 | 0.5187 |
77
- | 2.1231 | 14.0 | 299558 | 2.4810 | 0.5188 |
78
- | 2.0991 | 15.0 | 320955 | 2.4826 | 0.5191 |
79
- | 2.0851 | 16.0 | 342352 | 2.4822 | 0.5192 |
80
- | 2.0694 | 17.0 | 363749 | 2.4871 | 0.5192 |
81
- | 2.0582 | 18.0 | 385146 | 2.4890 | 0.5194 |
82
- | 2.0422 | 19.0 | 406543 | 2.4923 | 0.5194 |
83
- | 2.0188 | 20.0 | 427940 | 2.4961 | 0.5193 |
84
 
85
 
86
  ### Framework versions
87
 
88
- - Transformers 4.42.4
89
- - Pytorch 2.2.0+cu121
90
- - Datasets 2.16.1
91
- - Tokenizers 0.19.1
 
1
  ---
2
+ library_name: transformers
3
  tags:
4
  - generated_from_trainer
 
 
5
  metrics:
6
  - accuracy
7
  model-index:
8
  - name: opt-babylm2-20-epochs_seed-42_3e-4
9
+ results: []
 
 
 
 
 
 
 
 
 
 
10
  ---
11
 
12
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
14
 
15
  # opt-babylm2-20-epochs_seed-42_3e-4
16
 
17
+ This model was trained from scratch on an unknown dataset.
18
  It achieves the following results on the evaluation set:
19
+ - Loss: 2.4950
20
  - Accuracy: 0.5193
21
 
22
  ## Model description
 
50
 
51
  | Training Loss | Epoch | Step | Validation Loss | Accuracy |
52
  |:-------------:|:-----:|:------:|:---------------:|:--------:|
53
+ | 2.8017 | 1.0 | 21397 | 2.9043 | 0.4674 |
54
+ | 2.5697 | 2.0 | 42794 | 2.6914 | 0.4903 |
55
+ | 2.4593 | 3.0 | 64191 | 2.5998 | 0.5009 |
56
+ | 2.3962 | 4.0 | 85588 | 2.5532 | 0.5062 |
57
+ | 2.3371 | 5.0 | 106985 | 2.5247 | 0.5100 |
58
+ | 2.3029 | 6.0 | 128382 | 2.5101 | 0.5121 |
59
+ | 2.2663 | 7.0 | 149779 | 2.4970 | 0.5143 |
60
+ | 2.2435 | 8.0 | 171176 | 2.4892 | 0.5155 |
61
+ | 2.2171 | 9.0 | 192573 | 2.4831 | 0.5163 |
62
+ | 2.1902 | 10.0 | 213970 | 2.4811 | 0.5171 |
63
+ | 2.1695 | 11.0 | 235367 | 2.4788 | 0.5177 |
64
+ | 2.1548 | 12.0 | 256764 | 2.4811 | 0.5182 |
65
+ | 2.1307 | 13.0 | 278161 | 2.4788 | 0.5186 |
66
+ | 2.1228 | 14.0 | 299558 | 2.4802 | 0.5188 |
67
+ | 2.0984 | 15.0 | 320955 | 2.4807 | 0.5190 |
68
+ | 2.0845 | 16.0 | 342352 | 2.4828 | 0.5192 |
69
+ | 2.0687 | 17.0 | 363749 | 2.4844 | 0.5193 |
70
+ | 2.0578 | 18.0 | 385146 | 2.4892 | 0.5193 |
71
+ | 2.0413 | 19.0 | 406543 | 2.4918 | 0.5193 |
72
+ | 2.0185 | 20.0 | 427940 | 2.4950 | 0.5193 |
73
 
74
 
75
  ### Framework versions
76
 
77
+ - Transformers 4.45.1
78
+ - Pytorch 2.4.1+cu121
79
+ - Datasets 3.0.1
80
+ - Tokenizers 0.20.0
generation_config.json CHANGED
@@ -3,5 +3,5 @@
3
  "bos_token_id": 1,
4
  "eos_token_id": 2,
5
  "pad_token_id": 0,
6
- "transformers_version": "4.42.4"
7
  }
 
3
  "bos_token_id": 1,
4
  "eos_token_id": 2,
5
  "pad_token_id": 0,
6
+ "transformers_version": "4.45.1"
7
  }