<story><title>LeftoverLocals: Listening to LLM responses through leaked GPU local memory</title><url>https://blog.trailofbits.com/2024/01/16/leftoverlocals-listening-to-llm-responses-through-leaked-gpu-local-memory/</url></story><parent_chain></parent_chain><comment><author>ComputerGuru</author><text>At this point, I assume this is the default and don’t expect data recovery to not be provide on the same physical machine (even across virtualization barriers).&lt;p&gt;If your data is that sensitive, run it on dedicated hardware. Papering over this with mitigation over mitigation is a fool’s errand: both a genuine waste of compute resources and guaranteed to be a game of cat and mouse.</text></comment>
<story><title>LeftoverLocals: Listening to LLM responses through leaked GPU local memory</title><url>https://blog.trailofbits.com/2024/01/16/leftoverlocals-listening-to-llm-responses-through-leaked-gpu-local-memory/</url></story><parent_chain></parent_chain><comment><author>Kab1r</author><text>I&amp;#x27;ve been told that historically performance has been prioritized over security in the GPU space. Mitigating things like this does incur a performance penalty.</text></comment>