<story><title>Docker not ready for primetime</title><url>http://blog.goodstuff.im/docker_not_prime_time</url></story><parent_chain><item><author>tptacek</author><text>It&amp;#x27;s not worth the bother. Apart from keeping patches up today --- which is a good idea --- it&amp;#x27;s probably not really buying you anything.&lt;p&gt;It&amp;#x27;s not crazy to periodically rotate keys, but attackers don&amp;#x27;t acquire keys by, you know, stumbling over them on the street or picking them up when you&amp;#x27;ve accidentally left them on the bar. They get them because you have a vulnerability --- usually in your own code or configuration. Rebuilding will regenerate those kinds of vulnerabilities. Attackers will reinfect in seconds.</text></item><item><author>jacques_chester</author><text>&amp;gt; &lt;i&gt;Rebuilding continuously for security is not something I would recommend.&lt;/i&gt;&lt;p&gt;So that I understand, could you elaborate?&lt;p&gt;Particularly, do you mean &amp;quot;not recommend&amp;quot; as in &amp;quot;recommend &lt;i&gt;against&lt;/i&gt;&amp;quot; or &amp;quot;not worth the bother&amp;quot;?</text></item><item><author>tptacek</author><text>Having everything patched as soon as patches are available (or within, say, 6 hours of availability, for &amp;quot;routine&amp;quot; patches, with better responsiveness for critical patches) is a win.&lt;p&gt;The rest: not so much.&lt;p&gt;Rebuilding continuously for security is not something I would recommend.</text></item><item><author>jacques_chester</author><text>The security exec at Pivotal, where I work, has been talking about &amp;quot;repaving&amp;quot; servers as a &lt;i&gt;security&lt;/i&gt; tactic (along with rotating keys and repairing vulnerabilities).[0]&lt;p&gt;The theory runs that attackers need time to accrue and compound their incomplete positions into a successful compromise.&lt;p&gt;But if you keep patching continuously, attackers have fewer vulnerabilities to work with. If you keep rotating keys frequently, the keys they do capture become useless in short order. And if you rebuild the servers frequently, any system they&amp;#x27;ve taken control of simply vanishes and they have to start from scratch.&lt;p&gt;I&amp;#x27;m not &lt;i&gt;completely&lt;/i&gt; sold on the difference between repair and repave, myself. And I expect that sophisticated attackers will begin to rely more on identifying local holes and quickly encoding those in automated tools so that they can re-establish their positions after a repaving happens.&lt;p&gt;But it raises the cost for casual attackers, which is still worthy.&lt;p&gt;[0] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;medium.com&amp;#x2F;built-to-adapt&amp;#x2F;the-three-r-s-of-enterprise-security-rotate-repave-and-repair-f64f6d6ba29d#.k3dj6wlp1&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;medium.com&amp;#x2F;built-to-adapt&amp;#x2F;the-three-r-s-of-enterpris...&lt;/a&gt;</text></item><item><author>bphogan</author><text>woah, wait.....&lt;p&gt;It &amp;quot;geneerally works if&amp;quot; you &amp;quot; rebuild docker hosts on a daily or more frequent basis.&amp;quot;&lt;p&gt;Perhaps I&amp;#x27;m misunderstanding, but needing to rebuild my prod env several times a day seems pretty &amp;quot;not ready for prime time&amp;quot; to me.&lt;p&gt;That&amp;#x27;s like when we&amp;#x27;d say that Rails ran great in production in 2005, as long as you had a cron task to bounce fastCGI processes every hour or so.&lt;p&gt;So, can you elaborate on why rebuilding the containers is good advice?</text></item><item><author>siliconc0w</author><text>We use it in production.&lt;p&gt;It generally works if:&lt;p&gt;* you don&amp;#x27;t use it to store data&lt;p&gt;* don&amp;#x27;t use &amp;#x27;ambassador&amp;#x27;, &amp;#x27;buddy&amp;#x27;, or &amp;#x27;data&amp;#x27; container patterns.&lt;p&gt;* use tooling available to quickly and easily nuke and rebuild docker hosts on a daily or more frequent basis.&lt;p&gt;* use tooling available to &amp;#x27;orchestrate&amp;#x27; what gets run where - if you&amp;#x27;re manually running containers you&amp;#x27;re doing it wrong.&lt;p&gt;* wrap docker pulls with &amp;#x27;flock&amp;#x27; so they don&amp;#x27;t deadlock&lt;p&gt;* don&amp;#x27;t use swarm - use mesos, kube, or fleet(simpler, smaller clusters)</text></item></parent_chain><comment><author>lawnchair_larry</author><text>A lot of companies do lose their keys that way. www roots, gists, hardcoded in products, github history, etc.&lt;p&gt;The win to rotating them is not so much because you&amp;#x27;ll be regularly evicting attackers you didn&amp;#x27;t know had your keys, but because when you do have a fire, you won&amp;#x27;t be finding out for the first time that you can&amp;#x27;t actually rotate them.&lt;p&gt;It also forces you to design things much more reliably which helps continuity in non-security scenarios.&lt;p&gt;After redeploying and realizing that Todd has to ssh in and hand edit that one hostname and fix a symlink that was supposed to be temporary so the new version of A can talk to B, that&amp;#x27;s going to get rolled in pretty quickly. Large operations not doing this tend to quickly end up in the &amp;quot;nobody is allowed to touch this pile of technical debt because we don&amp;#x27;t know how to re-create it anymore&amp;quot; problem.</text></comment>
<story><title>Docker not ready for primetime</title><url>http://blog.goodstuff.im/docker_not_prime_time</url></story><parent_chain><item><author>tptacek</author><text>It&amp;#x27;s not worth the bother. Apart from keeping patches up today --- which is a good idea --- it&amp;#x27;s probably not really buying you anything.&lt;p&gt;It&amp;#x27;s not crazy to periodically rotate keys, but attackers don&amp;#x27;t acquire keys by, you know, stumbling over them on the street or picking them up when you&amp;#x27;ve accidentally left them on the bar. They get them because you have a vulnerability --- usually in your own code or configuration. Rebuilding will regenerate those kinds of vulnerabilities. Attackers will reinfect in seconds.</text></item><item><author>jacques_chester</author><text>&amp;gt; &lt;i&gt;Rebuilding continuously for security is not something I would recommend.&lt;/i&gt;&lt;p&gt;So that I understand, could you elaborate?&lt;p&gt;Particularly, do you mean &amp;quot;not recommend&amp;quot; as in &amp;quot;recommend &lt;i&gt;against&lt;/i&gt;&amp;quot; or &amp;quot;not worth the bother&amp;quot;?</text></item><item><author>tptacek</author><text>Having everything patched as soon as patches are available (or within, say, 6 hours of availability, for &amp;quot;routine&amp;quot; patches, with better responsiveness for critical patches) is a win.&lt;p&gt;The rest: not so much.&lt;p&gt;Rebuilding continuously for security is not something I would recommend.</text></item><item><author>jacques_chester</author><text>The security exec at Pivotal, where I work, has been talking about &amp;quot;repaving&amp;quot; servers as a &lt;i&gt;security&lt;/i&gt; tactic (along with rotating keys and repairing vulnerabilities).[0]&lt;p&gt;The theory runs that attackers need time to accrue and compound their incomplete positions into a successful compromise.&lt;p&gt;But if you keep patching continuously, attackers have fewer vulnerabilities to work with. If you keep rotating keys frequently, the keys they do capture become useless in short order. And if you rebuild the servers frequently, any system they&amp;#x27;ve taken control of simply vanishes and they have to start from scratch.&lt;p&gt;I&amp;#x27;m not &lt;i&gt;completely&lt;/i&gt; sold on the difference between repair and repave, myself. And I expect that sophisticated attackers will begin to rely more on identifying local holes and quickly encoding those in automated tools so that they can re-establish their positions after a repaving happens.&lt;p&gt;But it raises the cost for casual attackers, which is still worthy.&lt;p&gt;[0] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;medium.com&amp;#x2F;built-to-adapt&amp;#x2F;the-three-r-s-of-enterprise-security-rotate-repave-and-repair-f64f6d6ba29d#.k3dj6wlp1&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;medium.com&amp;#x2F;built-to-adapt&amp;#x2F;the-three-r-s-of-enterpris...&lt;/a&gt;</text></item><item><author>bphogan</author><text>woah, wait.....&lt;p&gt;It &amp;quot;geneerally works if&amp;quot; you &amp;quot; rebuild docker hosts on a daily or more frequent basis.&amp;quot;&lt;p&gt;Perhaps I&amp;#x27;m misunderstanding, but needing to rebuild my prod env several times a day seems pretty &amp;quot;not ready for prime time&amp;quot; to me.&lt;p&gt;That&amp;#x27;s like when we&amp;#x27;d say that Rails ran great in production in 2005, as long as you had a cron task to bounce fastCGI processes every hour or so.&lt;p&gt;So, can you elaborate on why rebuilding the containers is good advice?</text></item><item><author>siliconc0w</author><text>We use it in production.&lt;p&gt;It generally works if:&lt;p&gt;* you don&amp;#x27;t use it to store data&lt;p&gt;* don&amp;#x27;t use &amp;#x27;ambassador&amp;#x27;, &amp;#x27;buddy&amp;#x27;, or &amp;#x27;data&amp;#x27; container patterns.&lt;p&gt;* use tooling available to quickly and easily nuke and rebuild docker hosts on a daily or more frequent basis.&lt;p&gt;* use tooling available to &amp;#x27;orchestrate&amp;#x27; what gets run where - if you&amp;#x27;re manually running containers you&amp;#x27;re doing it wrong.&lt;p&gt;* wrap docker pulls with &amp;#x27;flock&amp;#x27; so they don&amp;#x27;t deadlock&lt;p&gt;* don&amp;#x27;t use swarm - use mesos, kube, or fleet(simpler, smaller clusters)</text></item></parent_chain><comment><author>tkiley</author><text>It seems like it&amp;#x27;s good to be able to rebuild everything at a moment&amp;#x27;s notice after patching against a major exploit, though. You should have a fast way to rebuild secrets and servers after the next heartbleed-scale vulnerability.</text></comment>