<story><title>Massive Dyn DNS outage</title><text>Sites down:&lt;p&gt;- DYN&lt;p&gt;- Twitter&lt;p&gt;- Etsy&lt;p&gt;- Github&lt;p&gt;- soundcloud&lt;p&gt;- spotify&lt;p&gt;- heroku&lt;p&gt;- pagerduty&lt;p&gt;- shopify&lt;p&gt;- intercom (app, not landing page)&lt;p&gt;Note that if these sites seem to be up to you, it&amp;#x27;s likely that your machine has cached the DNS response for these sites.&lt;p&gt;Some of these sites seem to work when using a UK VPN</text></story><parent_chain><item><author>6d6b73</author><text>All this talk about redundancy, real-time apps, scalable architecture and and a &amp;quot;simple&amp;quot; DDOS against DNS architecture brings half of the internet down. 
Honestly did nobody think about having a spare dns at some other company? or even backup dns server exactly for a situation like that?</text></item></parent_chain><comment><author>mancerayder</author><text>If ONLY they had 10X developers ...&lt;p&gt;EDIT:  joking aside, the issue with multiple DNS providers is primarily (in my experience at the company I&amp;#x27;m at having investigated this in the past) intelligent DNS entries.  Example, &amp;#x27;return these A records, in this order, based on the number of requests, roughly balanced&amp;#x27;.&lt;p&gt;There&amp;#x27;s no universal standard, just common aspects.  DNS Provider A has one set of features, names for returning A records in some way, based on some weighted averaging, provider B has a different mechanism.  So as an infrastructure person you have to:&lt;p&gt;1.  Investigate provider A and B features for intelligent DNS (it&amp;#x27;s not even universally called intelligent DNS!), and mentally parse the commonalities and differences&lt;p&gt;2.  Create a custom mechanism to keep them in sync internally.  So you hope that A and B have an API for maintaining the records.&lt;p&gt;3.  Ensure that when someone in your org wants to make an update, A and B update at the same time in the same way&lt;p&gt;4.  APIs don&amp;#x27;t change.</text></comment>
<story><title>Massive Dyn DNS outage</title><text>Sites down:&lt;p&gt;- DYN&lt;p&gt;- Twitter&lt;p&gt;- Etsy&lt;p&gt;- Github&lt;p&gt;- soundcloud&lt;p&gt;- spotify&lt;p&gt;- heroku&lt;p&gt;- pagerduty&lt;p&gt;- shopify&lt;p&gt;- intercom (app, not landing page)&lt;p&gt;Note that if these sites seem to be up to you, it&amp;#x27;s likely that your machine has cached the DNS response for these sites.&lt;p&gt;Some of these sites seem to work when using a UK VPN</text></story><parent_chain><item><author>6d6b73</author><text>All this talk about redundancy, real-time apps, scalable architecture and and a &amp;quot;simple&amp;quot; DDOS against DNS architecture brings half of the internet down. 
Honestly did nobody think about having a spare dns at some other company? or even backup dns server exactly for a situation like that?</text></item></parent_chain><comment><author>ljosa</author><text>The TTL for the glue records of a .com domain is 48 hours, so even if you have Route53 set up and ready to go, it takes a long time to switch the zone away from Dyn.&lt;p&gt;We switched from Dyn to Rout53 a few weeks ago. It took about 12 hours before half of the traffic had shifted over.</text></comment>