Update README.md

README.md

@@ -1,3 +1,32 @@
 ---
+language: 
+  - "en"
+thumbnail: "https://styles.redditmedia.com/t5_2to41/styles/communityIcon_qedoavxzocr61.png?width=256&s=9c7c19b81474c3788279b8d6d6823e791d0524fc"
+datasets:
+  - "reddit_tifu (subset: short)"
 license: mit
 ---
+
+# mgfrantz/distilgpt2-finetuned-reddit-tifu
+
+This model was trained to as practice for fine-tuning a causal language model.
+There was no intended use case for this model besides having some fun seeing how different things might be screwed up.
+
+## Data
+
+This model was trained on "short" subset of [`reddit_tifu`](https://huggingface.co/datasets/reddit_tifu) dataset.
+The data was split into 90% train and 10% validation using `dataset.train_test_split`, with a seed of 0.
+
+To prepare the data for training, the `"tldr"` and `"documents"` fields were joined by `"\n\n"`.
+When multiple items were in the `"tldr"` or `"documents"` fields, only the first item was selected for joining.
+These joined documents were tokenized using the `"distilgpt2"` tokenizer.
+
+Finally, tokenized texts were concatenated end-to-end and split into blocks of 128 tokens.
+
+**TODO:** Add a different separation token between documents that can be used to stop generation.
+
+## Training
+
+This model was trained in Colab by fine-tuning [`distilgpt2`](https://huggingface.co/distilgpt2) for 174390 steps (3 epochs).
+Default training arguments were used, except for `learning_rate=2e-5` and `weight_decay=0.01`.
+At the conclusion of training, a training loss of 3.52 and a validation loss of 3.44 were observed.