What We Know About Friday's Massive East Coast Internet Outage

Friday morning is prime time for some casual news reading, tweeting, and general Internet browsing, but you may have had some trouble accessing your usual sites and services this morning and throughout the day, from Spotify and Reddit to the New York Times and even good ol' WIRED.com. For that, you can thank a distributed denial of service attack (DDoS) that took down a big chunk of the Internet for most of the Eastern seaboard.

This morning's attack started around 7 am ET and was aimed at Dyn, an Internet infrastructure company headquartered in New Hampshire. That first bout was resolved after about two hours; a second attack began just before noon. Dyn reported a third wave of attacks a little after 4 pm ET. In all cases, traffic to Dyn's Internet directory servers throughout the US—primarily on the East Coast but later on the opposite end of the country as well—was stopped by a flood of malicious requests from tens of millions of IP addresses disrupting the system. Late in the day, Dyn described the events as a "very sophisticated and complex attack." Still ongoing, the situation is a definite reminder of the fragility of the web, and the power of the forces that aim to disrupt it.

Ripping Up the Telephone Book

Dyn offers Domain Name System (DNS) services, essentially acting as an address book for the Internet. DNS is a system that resolves the web addresses we see every day, like https://www.WIRED.com, into the IP addresses needed to find and connect with the right servers so browsers can deliver requested content, like the story you're reading right now. A DDoS attack overwhelms a DNS server with lookup requests, rendering it incapable of completing any. That's what makes attacking DNS so effective; rather than targeting individual sites, an attacker can take out the entire Internet for any end user whose DNS requests route through a given server.

"DNS registrars typically provide authoritative DNS services for thousands or tens of thousands of domain names, and so if there is a service-impacting event the collateral damage footprint can be very large," says Roland Dobbins, a principal engineer at Arbor Networks, a security firm that specializes in DDoS attacks.
DDoS is a particularly effective type of attack on DNS services because in addition to overwhelming servers with malicious traffic, those same servers also have to deal with automatic re-requests, and even just well-meaning users hitting refresh over and over to summon up an uncooperative page.

As Dyn absorbs more and more attacks, the scale of the situation becomes more clear. Specifically, that it's really, really big. "There's nothing really new about [this type of DDoS attack]. We've seen them for at least the last three years, they tend to be difficult to stop," says Matthew Prince, the CEO of the Internet infrastructure company Cloudflare. "But Dyn would see them on a regular basis, we see them on a regular basis. The fact that this is causing Dyn so many problems is pretty good evidence that this is an extremely large attack." Prince adds that Cloudflare, too, has seen an "uptick in errors" on its own network. It's not under attack; it's just experiencing fallout from the Dyn disruption.

Indeed, access to dozens of sites and services has been disrupted by the attack. Users in some regions like Asia seemed to experience fewer problems than those in the US. Though the topology of the Internet does not directly correspond to physical geography, it does approximate it to a degree, says Dobbins. Since Dyn says the impact was on its East Coast servers, this probably created the localized effect.

"This attack highlights how critical DNS is to maintaining a stable and secure internet presence, and that the DDOS mitigation processes businesses have in place are just as relevant to their DNS service as it is to the web servers and data centers," Richard Meeus, a vice president of technology at the enterprise security firm NSFOCUS, writes in an email.

What the Botnet

The overall picture is still somewhat hazy, but more information has become available as the day has progressed. Initial reports indicate that the attack was part of a genre of DDoS that infects Internet of Things devices (think webcams, DVRs, routers, etc.) all over the world with malware. Once infected, those Internet-connected devices become part of a botnet army, driving malicious traffic toward a given target. The source code for one of these types of botnets, called Mirai, was recently released to the public, leading to speculation that more Mirai-based DDoS attacks might crop up. Dyn said on Friday evening that the security firms Flashpoint and cloud services provider Akamai detected Mirai bots driving much, but not necessarily all, of the traffic in the attacks. Similarly, Dale Drew, the chief security officer of Internet backbone company Level 3, says that his company sees evidence of their involvement.

There's also a potential motive to use a Mirai hack against Dyn, or at least a certain irony in it. The company's principal data analyst, Chris Baker, wrote about these types of IoT-based attacks just yesterday in a blog post titled "What Is the Impact On Managed DNS Operators?". It appears he has his answer. And that all DNS services, and their customers, should be on notice.

This post has been updated to include new information about Mirai botnets, and to include additional comment from Dyn and Cloudflare CEO Matthew Prince.