Edit model card

This is karpathy's model from the llm.c project converted to HF format to investigate bfloat16 performance.

The training run was 150B tokens, 1.5 epochs over the 100B FineWeb sample dataset.

There's active work underway at https://github.com/karpathy/llm.c so I'd suggest following the developments there!

Downloads last month
635
Safetensors
Model size
774M params
Tensor type
BF16
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Dataset used to train rhysjones/gpt2-774M-fineweb-150B