blog-template.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0, minimum-scale=1.0, user-scalable=0">
    <meta name="format-detection" content="telephone=no">

    <title>IotSploit DEV</title>

    <link rel="stylesheet" href="./static/dist/style-libs-min.css">
    <link rel="stylesheet" href="./static/dist/styles.css">
</head>
<body>
<header>
    <section class="menu-section">
        <a class="logo" href="/">
            <svg xmlns="http://www.w3.org/2000/svg" width="410" height="43" viewBox="0 0 410 42.94">
                <defs>
                    <style>.cls-1 {
                        fill: #1a1a1a;
                    }

                    .cls-2 {
                        fill: red;
                    }</style>
                </defs>
                <title>logo</title>
                <path class="cls-1" d="M280.31,13.39h4.19v23h-4.19Z"/>
                <path class="cls-1"
                      d="M298,19.5c5.07,0,9.12,3.61,9.12,8.66s-4.06,8.66-9.08,8.66S289,33.26,289,28.16,293,19.5,298,19.5Zm0,13.83A4.93,4.93,0,0,0,303,28.42q0-.12,0-.25a4.93,4.93,0,1,0-9.84,0,4.9,4.9,0,0,0,4.64,5.14H298Z"/>
                <path class="cls-1" d="M313.83,17h-7.56V13.41h19.28V17H318V36.44h-4.15Z"/>
                <path class="cls-1"
                      d="M326.37,31.67a7.23,7.23,0,0,0,4.71,1.82c1.46,0,2.3-.62,2.3-1.65,0-2.59-8.21-1.94-8.21-7.37,0-3.25,2.85-5,6.33-5a8.49,8.49,0,0,1,5.58,1.88l-1.52,2.88a7,7,0,0,0-4.19-1.43c-1.43,0-2.3.55-2.3,1.59,0,2.62,8.21,1.88,8.21,7.3,0,2.92-2.46,5.1-6.26,5.1a9.46,9.46,0,0,1-6.49-2.43Z"/>
                <path class="cls-1"
                      d="M341.25,19.88H345v1.24A11.29,11.29,0,0,1,345,22.3H345a6.07,6.07,0,0,1,5.25-2.79c4.48,0,7.33,3.54,7.33,8.66s-3.22,8.66-7.56,8.66a6,6,0,0,1-4.78-2.33h0a9.6,9.6,0,0,1,.1,1.43v7h-4.11Zm8.11,13.47c2.27,0,4.16-1.85,4.16-5.13s-1.73-5.2-4.13-5.2c-2.17,0-4.19,1.56-4.19,5.23,0,2.57,1.42,5.1,4.12,5.1Z"/>
                <path class="cls-1"
                      d="M361.59,13.39h4.09V30.66c0,1.81.62,2.27,1.69,2.27a4.4,4.4,0,0,0,.62,0v3.61a10.64,10.64,0,0,1-1.3.1c-2.27,0-5.1-.58-5.1-5.32Z"/>
                <path class="cls-1"
                      d="M379.17,19.5c5.07,0,9.12,3.61,9.12,8.66s-4.06,8.66-9.08,8.66-9.08-3.57-9.08-8.66S374.17,19.5,379.17,19.5Zm0,13.83a4.93,4.93,0,0,0,4.94-4.91q0-.12,0-.25a4.93,4.93,0,1,0-9.84,0,4.9,4.9,0,0,0,4.64,5.14h.26Z"/>
                <path class="cls-1" d="M392.18,19.88h4.16V36.44h-4.13Z"/>
                <path class="cls-1"
                      d="M401.87,23.39h-2.14V20.11H402V15.37h4v4.74h3.77v3.28H406v6.46A2.94,2.94,0,0,0,409.25,33a3.31,3.31,0,0,0,.75-.06v3.61a10.58,10.58,0,0,1-1.26.1c-2.3,0-6.88-.68-6.88-6.23Z"/>
                <circle class="cls-2" cx="394.23" cy="15.09" r="1.93"/>
                <path class="cls-2"
                      d="M282.4,6.74a1.93,1.93,0,0,0-1.87,1.45H276.3l-2.41-2.42H0v1H273.47l2.42,2.42h4.63a1.93,1.93,0,1,0,1.87-2.42Z"/>
                <path class="cls-2"
                      d="M274,11.56a1.92,1.92,0,0,0-1.16.4L271,10.11H0v1H270.57l1.64,1.64A1.93,1.93,0,1,0,274,11.59Z"/>
                <path class="cls-2" d="M278,3.87a1.93,1.93,0,1,0-1.83-2.42H0v1H276.15A1.93,1.93,0,0,0,278,3.87Z"/>
            </svg>
        </a>
        <div class="menu">
            <ul>
                <li><a href="/dahua-template.html">Dahua Vulnerability</a></li>
                <li><a href="/report-template.html">Report</a></li>
                <li><a href="#">How to get started</a></li>
                <li><a href="#">Testimonials</a></li>
                <li><a href="/blog-template.html">Blog</a></li>
            </ul>
        </div>
        <button class="mobile-menu" onclick="$('.menu').slideToggle('slow')">
            <img src="./static/img/icons/icon-menu.svg" alt="">
        </button>
    </section>
</header>
<main class="template">
    <section class="blog">
        <h1 class="title">How Stack Overflow plans to survive the next DNS attack</h1>
        <div class="date">20.10.2010 19:54</div>
        <h3 class="author">Mark Henderson</h3>
        <p>Let’s talk about DNS. After all, what could go wrong? It’s just <a href="#">cache invalidation and naming
            things.</a></p>
        <h2>tl;dr</h2>
        <p>This blog post is about how Stack Overflow and the rest of the Stack Exchange network approaches DNS:</p>
        <ul>
            <li>By bench-marking different DNS providers and how we chose between them</li>
            <li>By implementing multiple DNS providers</li>
            <li>By deliberately breaking DNS to measure its impact</li>
            <li>By validating our assumptions and testing implementations of the DNS standard</li>
        </ul>
        <p>The good stuff in this post is in the middle, so feel free to scroll down to “The Dyn Attack” if you want to
            get straight into the meat and potatoes of this blog post.</p>
        <h3>The Domain Name System</h3>
        <p>DNS had its moment in the spotlight in October 2016, with a major Distributed Denial of Service (DDos)
            attack <a href="#">launched against Dyn</a>, which affected the ability for Internet users to connect to
            some of their
            favourite websites, such as <b>Twitter, CNN, imgur, Spotify, and literally </b> thousands of other sites.
        </p>
        <p>But for most systems administrators or website operators, DNS is mostly kept in a little black box,
            outsourced
            to a 3rd party, and mostly forgotten about. And, for the most part, this is the way it should be. But as you
            start to grow to <i>1.3+ billion pageviews a month with a website</i> where performance is a feature, every
            little
            bit
            matters.</p>

        <p>In this post, I’m going to explain some of the decisions we’ve made around DNS in the past, and where we’re
            going with it in the future. I will eschew deep technical details and gloss over low-level DNS
            implementation in favour of the broad strokes.</p>
        <h2>In the beginning</h2>
        <p>So first, a bit of history: In the beginning, we ran our own DNS on-premises using artisanally crafted zone
            files with BIND. It was fast enough when we were doing only a few hundred million hits a month, but
            eventually
            hand-crafted zonefiles were too much hassle to maintain reliably. When we moved to Cloudflare as our CDN,
            their
            service is intimately coupled with DNS, so we demoted our BIND boxes out of production and handed off DNS to
            Cloudflare.
        </p>
        <h3> The search for a new provider</h3>
        <p> Fast forward to early 2016 and we moved our CDN to Fastly. Fastly doesn’t provide DNS service, so we were
            back on our own in that regards and our search for a new DNS provider began. We made a list of every DNS
            provider
            we could think of, and ended up with a shortlist of 10:
        </p>
        <ol>
            <li>Dyn</li>
            <li>NS1</li>
            <li>Amazon Route 53</li>
            <li>Google Cloud DNS</li>
            <li>Azure DNS (beta)</li>
            <li>DNSimple</li>
            <li>Godaddy</li>
            <li>EdgeCast (Verizon)</li>
            <li>Hurricane Electric</li>
            <li>DNS Made Easy</li>
        </ol>
        <p>
            From this list of 10 providers, we did our initial investigations into their service offerings, and started
            eliminating services that were either not suited to our needs, outrageously expensive, had insufficient
            SLAs, or didn’t offer services that we required (such as a fully featured API). Then we started performance
            testing. We did this by embedding a hidden iFrame on 5% of the visitors to <a href="#">stackoverflow.com</a>,
            which forced a request to a different DNS provider. We did this for each provider until we had some pretty
            solid
            performance numbers.
        </p>
        <a href="/static/img/dns-boxplot-us-single-600.png">
            <img class="center" src="./static/img/dns-boxplot-us-single-600.png" alt="">
        </a>
        <p>
            If you don’t know how to interpret a boxplot, <a href="#">here’s a brief primer for you</a>. For the data
            nerds, these were generated with R’s standard boxplot functions, which means the upper and lower whiskers
            are min(max(x), Q_3 + 1.5 * IQR) and max(min(x), Q_1 – 1.5 * IQR), where IQR = Q_3 – Q_1
        </p>
        <a href="/static/img/dns-boxplot-nz-single-600.png">
            <img class="right" src="./static/img/dns-boxplot-nz-single-600.png" alt="">
        </a>
        <p>
            You can see that Hurricane Electric had a quarter of requests return in < 16ms and a median of 32ms, with
            the three “cloud” providers (Azure, Google Cloud DNS and Route 53) being slightly slower (being around 24ms
            first quarter and 45ms median), and DNS Made Easy coming in 2nd place (20ms first quarter, 39ms median).
        </p>
        <p> You might wonder why the scale on that chart goes all the way to 700ms when the whiskers go nowhere near
            that high. This is because we have a worldwide audience, so just looking at data from the United States is
            not sufficient. If we look at data from New Zealand, we see a very different story:
        </p>
        <p>
            Here you can see that Route 53, DNS Made Easy and Azure all have healthy 1st quarters, but Hurricane
            Electric and Google have very poor 1st quarters. Try to remember this, as this becomes important later on.
        </p>
        <p>
            We also have <a href="#">Stack Overflow in Portuguese</a>, so let’s check the performance from Brazil:
        </p>
        <p> Here we can see Hurricane Electric, Route 53 and Azure being favoured, with Google and DNS Made Easy being
            slower.
        </p>
        <p> So how do you reach a decision about which DNS provider to choose, when your main goal is performance? It’s
            difficult, because regardless of which provider you end up with, you are going to be choosing a provider
            that is sub-optimal for part of your audience.
        </p>
        <a href="/static/img/dns-boxplot-br-single-600.png">
            <img class="left" src="./static/img/dns-boxplot-br-single-600.png" alt="">
        </a>
        <p> You know what would be awesome? If we could have two DNS providers, each one servicing the areas that they
            do best! Thankfully this is something that is possible to implement with DNS. However, time was short, so we
            had to put our dual-provider design on the back-burner and just go with a single provider for the time
            being.
        </p>
        <p> Our initial rollout of DNS was using Amazon Route 53 as our provider: they had acceptable performance
            figures over a large number of regions and had very effective pricing (on that note Route 53, Azure DNS, and
            Google Cloud DNS are all priced identically for basic DNS services).
        </p>
        <h2>The DYN attack</h2>
        <p>Roll forwards to October 2016. Route 53 had proven to be a stable, fast, and cost-effective DNS provider. We
            still had dual DNS providers on our backlog of projects, but like a lot of good ideas it got put on the
            back-burner until we had more time. Then the Internet ground to a halt. The DNS provider Dyn had come under
            attack, knocking a large number of authoritative DNS servers off the Internet, and causing widespread issues
            with connecting to major websites. All of a sudden DNS had our attention again. Stack Overflow and Stack
            Exchange were not affected by the Dyn outage, but this was pure luck.

        </p>
        <h3>
            What is the performance impact for our users in having multiple DNS providers, when both providers are
            working properly?
        </h3>
        <p>This one should be fairly easy to test. We’ve already done it once, so let’s just do it again. We fired up
            our tests, as we did in early 2016, but this time we specified two DNS providers:
        </p>
        <ul>
            <li>Route 53 & Google Cloud</li>
            <li>Route 53 & Azure DNS</li>
            <li>Route 53 & Our internal DNS</li>
        </ul>
        <p>We did this simply by listing Name Servers from both providers in our domain registration (and obviously we
            set up the same records in the zones for both providers).</p>
        <h3>Azure</h3>
        <p>Azure has a fatal flaw in their DNS offering, as of the writing of this blog post. They <a href="#">do not
            permit the modification of the NS records in the apex of your zone:</a>
        </p>
        <blockquote>
            You cannot add to, remove, or modify the records in the automatically created NS record set at the zone apex
            (name = “@”). The only change that’s permitted is to modify the record set TTL.
        </blockquote>
        <p>These NS records are what your DNS provider says are authoritative DNS servers for a given domain. It’s very
            important that they are accurate and correct, because they will be cached by clients and DNS resolvers and
            are more authoritative than the records provided by your registrar.</p>
        <p>Without going too much into the actual specifics of how DNS caching and NS records work (it would take me
            another 2,500 words to describe this in detail), what would happen is this: Whichever DNS provider you
            contact first would be the only DNS provider you could contact for that domain until your DNS cache expires.
            If Azure is contacted first, then only Azure’s nameservers will be cached and used. This defeats the purpose
            of having multiple DNS providers, as in the event that the provider you’ve landed on goes offline, which is
            roughly 50:50, you will have no other DNS provider to fall back to.</p>


        <code>
<pre>
function checkParent (src, dest) {<Br>
    while (src != null) {<Br>
        if (src.tagName == dest) return src<Br>
        src = src.parentElement<Br>
    }<Br>
    return null<Br>
}
</pre>
        </code>
        <p>
            If we looked at the CPU from something like Nagios it is going to look quite low (20%). This problem is
            unlikely to show up when using a profiler unless you just happen to load the page during these slow downs.
            High resolution monitoring allows you to discover issues like this that get buried with low sample rates.
            Also having a platform to view multiple data sources allows for the discovery of correlated metrics.
        </p>

        <div id="disqus_thread"></div>
        <script>

						/**
						 *  RECOMMENDED CONFIGURATION VARIABLES: EDIT AND UNCOMMENT THE SECTION BELOW TO INSERT DYNAMIC VALUES FROM YOUR PLATFORM OR CMS.
						 *  LEARN WHY DEFINING THESE VARIABLES IS IMPORTANT: https://disqus.com/admin/universalcode/#configuration-variables*/
            /*
             var disqus_config = function () {
             this.page.url = PAGE_URL;  // Replace PAGE_URL with your page's canonical URL variable
             this.page.identifier = PAGE_IDENTIFIER; // Replace PAGE_IDENTIFIER with your page's unique identifier variable
             };
             */
						(function() { // DON'T EDIT BELOW THIS LINE
								var d = document, s = d.createElement('script');
								s.src = 'https://iotsploit.disqus.com/embed.js';
								s.setAttribute('data-timestamp', +new Date());
								(d.head || d.body).appendChild(s);
						})();
        </script>
        <noscript>Please enable JavaScript to view the <a href="https://disqus.com/?ref_noscript">comments powered by Disqus.</a></noscript>

        <script id="dsq-count-scr" src="//iotsploit.disqus.com/count.js" async></script>

    </section>
</main>
<footer>
    <div class="footer-content">
        <div class="footer-menu">
            <ul>
                <li><a href="#">Why</a></li>
                <li><a href="#">Services</a></li>
                <li><a href="#">How to get started</a></li>
                <li><a href="#">Testimonials</a></li>
                <li><a href="/blog-template.html">Blog</a></li>
            </ul>
        </div>
        <div class="contacts">
            <a class="mail" href="#">hello@iotsploit.ai</a>
            <div class="address">
                350 Fifth Avenue, 34th floor
                New York, NY 10118-3299 USA
            </div>
            <a class="phone" href="#">+996(550)500-000</a>
        </div>
        <div class="footer-social">
            <a class="social--icon fb" href="#">
                <img src="./static/img/icons/icon-facebook.svg" alt="">
            </a>
            <a class="social--icon tw" href="#">
                <img src="./static/img/icons/icon-twitter.svg" alt="">
            </a>
            <a class="social--icon git" href="#">
                <img src="./static/img/icons/icon-github.svg" alt="">
            </a>
            <a class="social--icon medium" href="#">
                <img src="./static/img/icons/icon-medium.svg" alt="">
            </a>
        </div>
    </div>
    <div class="footer-copyright">
        &copy; 2017 Iotsploit
    </div>
</footer>
<script>
		(function (i, s, o, g, r, a, m) {
				i['GoogleAnalyticsObject'] = r;
				i[r] = i[r] || function () {
								(i[r].q = i[r].q || []).push(arguments)
						}, i[r].l = 1 * new Date();
				a = s.createElement(o),
						m = s.getElementsByTagName(o)[0];
				a.async = 1;
				a.src = g;
				m.parentNode.insertBefore(a, m)
		})(window, document, 'script', 'https://www.google-analytics.com/analytics.js', 'ga');

		ga('create', 'UA-83208754-4', 'auto');
		ga('send', 'pageview');

</script>
<script src="./static/dist/libraries-min.js"></script>
<script src="./static/dist/scripts-min.js"></script>
</body>
</html>