For a simple test, we quantize Llama-7B at both 8-bits and 4-bits, and compare our baseline attack (Section 4.1) to the default 16-bit implementation. We quantize using bitsandbytes(Dettmers et al., 2022), which HuggingFace supports for out-of-the-box quantization of model weights and lower-precision inference (Figure 6). We observe no meaningful differences at different levels of quantization; querying each model results in recovering the same same embedding matrix dimension \(h\) in the same number of queries. Given that 8-bit and 4-bit quantization are generally observed to not have a large impact on performance, this is perhaps an unsurprising result; any noise from quantization does not seem to have a meaningful impact on the logits (in the context of our attack).