| ## MechDistilGPT2 | |
| This model is fine-tuned on 200k text scraped from Mechanical/Automotive pdf books. | |
| Base model is DistilGPT2(https://huggingface.co/gpt2) (the smallest version of GPT2) | |
| ## Fine-Tuning | |
| Default Training Args | |
| Epochs = 3 | |
| Perplexity = 48 | |
| Training set = 200k sentences | |
| Validation set = 40k sentences | |
| ## Framework versions | |
| Transformers 4.7.0.dev0 | |
| Pytorch 1.8.1+cu111 | |
| Datasets 1.6.2 | |
| Tokenizers 0.10.2 |