We choose three families of LLMs to evaluate SmoothQuant: OPT [1]}, BLOOM [2]}, and GLM-130B [3]}.
We use seven zero-shot evaluation tasks: LAMBADA [4]}, HellaSwag [5]}, PIQA [6]}, WinoGrande [7]}, OpenBookQA [8]}, RTE [9]}, COPA [10]}, and one language modeling dataset WikiText [11]} to evaluate the OPT and BLOOM models. We use MMLU [12]}, MNLI [13]}, QNLI [9]} and LAMBADA to evaluate the GLM-130B model because some of the aforementioned benchmarks appear in the training set of GLM-130B.
We use lm-eval-harnesshttps://github.com/EleutherAI/lm-evaluation-harness to evaluate OPT and BLOOM models, and GLM-130B's official repo\(\endcsname \)https://github.com/THUDM/GLM-130B
 for its own evaluation.
Note that we focus on the relative performance change before and after quantization but not the absolute value.
