<story><title>Text embeddings reveal almost as much as text</title><url>https://arxiv.org/abs/2310.06816</url></story><parent_chain></parent_chain><comment><author>gojomo</author><text>One classic old estimate of the amount of information per English word, from Claude Shannon, is 11.82 bits per word. So very roughly, a 32-token text – the length they highlight to demonstrate their technique – should be representable in  about 379 bits.&lt;p&gt;One of the embeddings they demonstrate the use of their technique against is the `text-embedding-ada-002` OpenAI offering, which gives back a 1,536-dimension representation, where every dimension is a floating-point number.&lt;p&gt;So as a matter of theory, just in the &lt;i&gt;sign-bits&lt;/i&gt; of those vectors, there&amp;#x27;s enough raw state to potentially encode 32-word texts, and more.&lt;p&gt;If those float-dimensions are 4-byte floats, as are common, a single `text-embedding-ada-002` text vector takes (1536 * 4 bytes =) 6,144 bytes of storage (49,152 bits), While the dense&amp;#x2F;continuous nature of these values, and all the desirable constraints&amp;#x2F;uses packed into them, means you won&amp;#x27;t be getting that much precise&amp;#x2F;lossless text-representation from the values, it&amp;#x27;s plenty spacious for capturing a pretty-good compression of some pretty-long texts.&lt;p&gt;The interesting thing here is how often that short texts can be perfectly or nearly-perfectly recovered, via the authors&amp;#x27; iterative method – even without that being an intended designed-in capability of the text embedding.&lt;p&gt;You could also see the result as an example of the old saw, &amp;quot;compression is intelligence&amp;quot; (or vice-versa) – an insight which has motivated, among other things, the Hutter Prize.&lt;p&gt;Some interesting questions for future work on the same codebase could be:&lt;p&gt;• at what text length does accuracy peak, and at what length does it precipitously decline?&lt;p&gt;• are even the failures still reasonable summaries of the original text?&lt;p&gt;• if for some applications such recovery is undesirable – in many it&amp;#x27;s no problem – are there ways to change the models, with different&amp;#x2F;extra training&amp;#x2F;ablation&amp;#x2F;whatever, that retains other aspects of the embeddings&amp;#x27; usefulness but impairs verbatim text recovery?</text></comment>
<story><title>Text embeddings reveal almost as much as text</title><url>https://arxiv.org/abs/2310.06816</url></story><parent_chain></parent_chain><comment><author>esafak</author><text>Why wouldn&amp;#x27;t they? If you think of embeddings as a learned hash, and your hash space is wide enough, the embedding would simply be another, lossless representation of the input. The challenge, of course, is that inverting hashes is difficult. Except in machine learning, the hashes are typically intended not to be so; to preserve semantic and syntactic relationships, as word2vec famously demonstrated. And there are even text embeddings that use sub-word information like character n-grams, which can trivially represent rare words, like the kind embodied by personal information.&lt;p&gt;edit: Given the author agrees, I suppose the research question is how well and cheaply you can do it, across different embedding algorithms. For practitioners, the lesson is to treat embeddings like personally identifiable information, as the authors state in the conclusion.</text></comment>