Our attack is effective and efficient, and is applicable to production models whose APIs expose full logprobs, or a "logit bias". This included Google's PaLM-2 and OpenAI's GPT-4 (Anil et al., 2023; OpenAI et al., 2023); after responsible disclosure, both APIs have implemented defenses to prevent our attack or make it more expensive. We extract the embedding layer of several OpenAI models with a mean squared error of \(10^{-4}\) (up to unavoidable symmetries). We apply a limited form of our attack to gpt-3.5 at a cost of under $200 USD and, instead of recovering the full embedding layer, recover just the size of the embedding dimension.