Post-training quantization (PTQ)  techniques applied to weights, activations, and the KV cache greatly reduce memory usage, latency, and power consumption of Large Language Models (LLMs) , but may lead to large quantization errors when outliers are present. Recent findings suggest that rotating activation or weight matrices helps remove outliers and benefits quantization. In this work, we identify a collection of applicable rotation parameterizations that lead to identical outputs in full-precision Transformer architectures, and find that some random rotations lead to much better quantization than others, with an up to13 pointsdifference in downstream zero-shot reasoning performance. As a result, we proposeSpinQuantthatoptimizes(orlearns)  the rotation matrices withCayleyoptimization on a small validation set. With 4-bit quantization of weight, activation, and KV-cache,SpinQuantnarrows the accuracy gap on zero-shot reasoning tasks with full precision to merely 2.9 points on the LLaMA-2 7B model, surpassing LLM-QAT by 19.1 points and SmoothQuant by 25.0 points.SpinQuantalso outperforms concurrent work QuaRot, which applies random rotations to remove outliers. In particular, for LLaMA-2 7B/LLaMA-3 8B models that are hard to quantize,SpinQuantreduces the gap to full precision by 30.2%/34.1% relative to QuaRot.