## MechDistilGPT2
This model is fine-tuned on 200k text scraped from Mechanical/Automotive pdf books.

Base model is DistilGPT2(https://huggingface.co/gpt2) (the smallest version of GPT2)

## Fine-Tuning
Default Training Args
Epochs = 3
Perplexity = 48
Training set = 200k sentences
Validation set = 40k sentences

## Framework versions
Transformers 4.7.0.dev0
Pytorch 1.8.1+cu111
Datasets 1.6.2
Tokenizers 0.10.2