FW-ProX-1.7B / README.md
koalazf99's picture
Update README.md
22d0ce7 verified
|
raw
history blame
1.1 kB
metadata
license: apache-2.0
datasets:
  - gair-prox/FineWeb-pro
language:
  - en
tags:
  - llama
pipeline_tag: text-generation

FW-ProX-1.7B

ArXiv | Models | Data | Code

FW-ProX-1.7B is a small language model. It was and trained on the FineWeb-pro for 50B tokens.

Evaluations

ProX models are evaluated over 10 language model benchmarks in zero-shot setting.

ArC-c ARC-e CSQA HellaS MMLU OBQA PiQA SIQA WinoG SciQ AVG
raw 28.5 52.6 33.9 53.2 29.8 32.6 72.9 40.2 53.0 77.1 47.4
ours 34.4 63.9 32.6 53.0 33.1 34.4 73.1 39.3 52.7 81.5 49.8

Citation

@misc{TBD
}