metadata

license: apache-2.0
datasets:
  - gair-prox/FineWeb-pro
language:
  - en
tags:
  - llama
pipeline_tag: text-generation

FW-ProX-1.7B

FW-ProX-1.7B is a small language model. It was and trained on the FineWeb-pro for 50B tokens.

Evaluations

ProX models are evaluated over 10 language model benchmarks in zero-shot setting.

	ArC-c	ARC-e	CSQA	HellaS	MMLU	OBQA	PiQA	SIQA	WinoG	SciQ	AVG
raw	28.5	52.6	33.9	53.2	29.8	32.6	72.9	40.2	53.0	77.1	47.4
ours	34.4	63.9	32.6	53.0	33.1	34.4	73.1	39.3	52.7	81.5	49.8

@misc{TBD
}