Pretraining step 2-- 100k openwebtext (val loss 4.834130)

Step Training Loss Validation Loss
1000 5.358700 5.325854
2000 5.171800 5.178083
3000 5.111800 5.047909
4000 4.987200 4.929996
5000 4.991600 4.834130

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ea896f682b475151113632d3461a627a7b3641f41f7f028c870a4e8ad094bd5a
+oid sha256:50279c12019a6f0dbb3680e52e6b4e2a281c92c869e48da093721eb027a0c94a
 size 1665286432