
|Hyperparameter       |Value    |
|---------------------|---------|
|Steps                |     150k|
|Max length           |      256|
|LR                   |     1e-4|
|LR schedule          | constant|
|Optimizer            |    AdamW|
|beta_1, beta_2       |0.9, 0.95|
|Final eval loss      |    2.245|
|Final eval perplexity|     9.44|

{"language": ["en"], "datasets": ["hlillemark/c4_t5_corrupted_seqlen256"], "metrics": ["perplexity"]}
pytorch tensorboard en dataset:hlillemark/c4_t5_corrupted_seqlen256 region:us
TAGS
#pytorch #tensorboard #en #dataset-hlillemark/c4_t5_corrupted_seqlen256 #region-us 

TAGS
#pytorch #tensorboard #en #dataset-hlillemark/c4_t5_corrupted_seqlen256 #region-us 
