witcher
commited on
Commit
•
6ddf34d
1
Parent(s):
ae0fc7b
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,14 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
## MechDistilGPT2
|
| 2 |
This model is fine-tuned on 200k text scraped from Mechanical/Automotive pdf books.
|
| 3 |
|
|
@@ -6,7 +17,6 @@ Base model is DistilGPT2(https://huggingface.co/gpt2) (the smallest version of G
|
|
| 6 |
## Fine-Tuning
|
| 7 |
* Default Training Args
|
| 8 |
* Epochs = 3
|
| 9 |
-
* Perplexity = 48
|
| 10 |
* Training set = 200k sentences
|
| 11 |
* Validation set = 40k sentences
|
| 12 |
|
|
@@ -14,4 +24,4 @@ Base model is DistilGPT2(https://huggingface.co/gpt2) (the smallest version of G
|
|
| 14 |
* Transformers 4.7.0.dev0
|
| 15 |
* Pytorch 1.8.1+cu111
|
| 16 |
* Datasets 1.6.2
|
| 17 |
-
* Tokenizers 0.10.2
|
|
|
|
| 1 |
+
|
| 2 |
+
---
|
| 3 |
+
tags:
|
| 4 |
+
- Causal Language modeling
|
| 5 |
+
model_index:
|
| 6 |
+
- name: MechDistilGPT2
|
| 7 |
+
results:
|
| 8 |
+
- task:
|
| 9 |
+
name: Causal Language modeling
|
| 10 |
+
type: Causal Language modeling
|
| 11 |
+
---
|
| 12 |
## MechDistilGPT2
|
| 13 |
This model is fine-tuned on 200k text scraped from Mechanical/Automotive pdf books.
|
| 14 |
|
|
|
|
| 17 |
## Fine-Tuning
|
| 18 |
* Default Training Args
|
| 19 |
* Epochs = 3
|
|
|
|
| 20 |
* Training set = 200k sentences
|
| 21 |
* Validation set = 40k sentences
|
| 22 |
|
|
|
|
| 24 |
* Transformers 4.7.0.dev0
|
| 25 |
* Pytorch 1.8.1+cu111
|
| 26 |
* Datasets 1.6.2
|
| 27 |
+
* Tokenizers 0.10.2
|