Open Resolver Problems

All,

Open resolvers pose a security threat. I wanted to let everyone know about a search tool that can help you find the ones within your organization. Treat it like a big "BETA" stamp is across it, but please try it out and see if you can close down any hosts within your network.

This threat is larger than the SMURF amplification attacks in the past and can result in some quite large attacks. I've seen this spilling out into other mailing lists (e.g.: juniper-nap and others).

Please send feedback about links that should be included or documentation and spelling errors to me.

openresolverproject.org

Some basic stats:

27 million resolvers existed as of this dataset collection

only 2.1 million of them were "closed".

We have a lot to do to close the hosts, please do what you can to help.

Thanks,

- Jared

I would like to be able to request an IP list of open resolvers in my ASN, perhaps sent to the contact details in RIPE whois database to make sure I'm not falsely representing that ASN.

There doesn't seem to be a way to get this done using that webpage?

What are those who provide open resolvers, such as google, doing to
combat the problem?

It would be nice to be able to provide open resolvers as a service and
combat the various threats associated with them.

Cheers,
Harry

I think if we get to that small number from tens of millions then we are in much better shape.

Closing them and setting up rate limiting on your authorities will go a long way.

Jared Mauch

There are a number of open resolvers that are that way by design (i.e.
Google), but most of them are there by misconfiguration, having a small
number (say < 100) of well-known open resolvers in the world is not a
problem, having > 1 million probably is

Mike

What's the current BCP on how to deal with mobile devices that hard-code
your resolvers in their equivalent of /etc/resolv.conf (often because the
owner of the device trusts their emnployers/whatever resolver more than they
trust the DNS server that the hotel DHCP pointed them at)?

(And yes, I *know* that "point at your employers DNS" works against a
threat model of "local provider is an idiot" and fails against "local
provider is willing to spoof replies from other DNS servers")

Why would that matter? This is publicly available information.

A per-asn query would be very useful.

Nick

Some of us have both publicly-facing authoritative DNS, and inward
facing recursive servers that may be open resolvers but can't be
found via NS entries (so the IP addresses of those aren't exactly
publicly available info).

A list of 27 million open resolvers would be a pretty convenient input for
miscreants who want to abuse them, I believe? I assume Jared & co doesn't
want their collected work to be abused like that.

I quickly scripted a RIPE-based ASN lookup tool for myself just so I could
look up my own and customers AS-based IP ranges atleast. Thank you so much
for making the site available, Jared. Very helpful!

Scoping your responses based on query-source should work just fine in this case.

There's documentation on how to do that online here:

http://www.zytrax.com/books/dns/ch9/close.html

I highly recommend doing this with your name server. If you have examples of how to do this you want to share and have me post, as I mentioned, please send me your edits and additions. I want to make this valuable.

- Jared

There are other tools, such as the one linked to at the measurement factory that will email the contacts for the objects (e.g.: RIPE/ARIN otherwise).

Here's the direct link for those that are interested.

As I mentioned, this is alpha/beta quality code, but i feel very confident with the data.

- Jared

http://nmap.org/nsedoc/scripts/dns-recursion.html
http://monkey.org/~provos/dnsscan/

There are 224*2^24 possible unicast hosts, and a whole pile less which are
routed on the DFZ.

I don't think that we can pretend that it's going to help if we hide this
information under a stone and hope that people who are inclined to launch
DNS DDoS attacks are dumb enough not to be able to figure out how to use
these tools.

Highlighting the situation and getting operators to do something will help
fix the problem.

Nick

Well,

    Why would you only go after them?

    Easier target to mitigate the problem?

    That might be just me, but I find those peers allowing their
customers to spoof source IP addresses more at fault.

    PS: Some form of adaptive rate limitation works for it btw =D

DNS servers (recursive and authoritative-only) are the low-hanging fruit du jour. I agree that there are many other effective amplifiers, and that even maximum DNS hygiene will not make the wider problem go away.

A quick note on your final comment, though: whilst adaptive response rate limiting (so-called RRL) is fast developing into an effective mitigation for reflection attacks against authority-only servers, there is far less experience with traffic patterns or the effects of rate-limiting (using RRL or anything else) on recursive servers.

The best advice for operation of recursive servers remains "restrict access to legitimate clients", not "apply rate-limiting".

Joe

Yes, I wish every "broadband speed test" could include spoofed IP. Unfortunately I would imagine most OSes won't allow normal users to spoof sender IP so I guess it's hard to recognise.

We need a hall of shame for that.

    That might be just me, but I find those peers allowing their
customers to spoof source IP addresses more at fault.

that is equally stupid and bad.

    PS: Some form of adaptive rate limitation works for it btw =D

no, it doesn't. In order to ensure that your resolver clients are serviced
properly, you need to keep the DNS query rate high enough that if someone
has a large bcp38-enabled botnet, they can trash the hell out of whoever
they want.

The best solution is to disable open recursion completely, and police your
clients regularly.

Nick

Hi,

    Well...

    That might be just me, but I find those peers allowing their
customers to spoof source IP addresses more at fault.

that is equally stupid and bad.

    In my eyes, those peers are the source of it.

    One can justify Open Relay and the lag into moving into not being an
attack vector... while the case for allowing IP spoofing is a tad
harder to justify.

    PS: Some form of adaptive rate limitation works for it btw =D

no, it doesn't. In order to ensure that your resolver clients are serviced
properly, you need to keep the DNS query rate high enough that if someone
has a large bcp38-enabled botnet, they can trash the hell out of whoever
they want.

    We all need to be more flexible and actually work toward fixing both
end of the issue.

The best solution is to disable open recursion completely, and police your
clients regularly.

Nick

    I just intervene on one of today's DNS Amp... which is going to many
targets mind you... on a client with a NT4.0 Server and another with
FreeBSD 5.1 =D
    ( You can say bye bye to that NT4.0 client revenue :frowning: )

    Now about some of "those" peers start enforcing some form of source
IP rules...

    PS: The Open Relay situation is easy to fix for a subscriber type
corp (like say a Cable provider)... and less for smaller outfit
providing all sort of IT services.

Nothing equal about it. Open resolvers (and other forms of
amplification attacks like the basic smurf) are a problem if and only
if a target's source IP address can be spoofed. Service providers
intentionally or negligently permitting their users to spoof source
addresses outside that ISP's domain are the *root cause* of the
problem.

Even if you close all the open resolvers, most authoritative responses
are larger than the queries. At best you've shrunk the amplification
factor. What will you do next? Insist that everybody host their DNS
somewhere sophisticated rather than running their own server?

Hassling the folks who run open resolvers further victimizes the
innocent. If you want to solve the problem, start by cleaning up your
border so that only locally valid sources can exit. Next, identify
peers who fail to demonstrate adequate control over their sources.
Finally, set filters on those peers so that sources inconsistent with
the received routes are dropped.

They won't like it. They'll find it inconvenient, even disruptive to
their traffic engineering efforts. But at some point, TE has to take a
back seat to closing network abuse issues.

Regards,
Bill Herrin

Could you clarify, here, Jared?

Do "open DNS customer-resolver/recursive servers" *per se* cause a problem?

Or is it merely "customer zone servers which are misconfigured to recurse",
as has always been problematic?

That is: is this just a reminder we never closed the old hole, or
notification of some new and much nastier hole?

Cheers,
-- jra

running open resolvers will continue to be a major problem as a DDoS
platform on the Internet until everyone implements BCP38. When everyone
has implemented ingress filtering, we can have a beer and agree that
running open resolvers is less harmful. Until then, though, they're a menace.

Nick