In this paper, we focus on high-fidelity attacks. Most prior high-fidelity attacks exploit specific properties of deep neural networks with ReLU activations. Milli et al. (2019) first showed that if an attacker can compute _gradients_ of a target two-layer ReLU model, then they can steal a nearly bit-for-bit equivalent model. Jagielski et al. (2020) observed that if the attacker only has query access to model outputs, they can approximate gradients with finite differences. Subsequent work extended these attacks to efficiently extract deeper ReLU models (Carlini et al., 2020; Rolnick and Kording, 2020; Shamir et al., 2023). Unfortunately, none of these approaches scale to production language models, because they (1) accept tokens as inputs (and so performing finite differences is intractable); (2) use activations other than ReLUs; (3) contain architectural components such as attention, layer normalization, residual connections, etc. that current attacks cannot handle; (4) are orders-of-magnitude larger than prior extracted models; and (5) expose only limited-precision outputs.