I am having some very bizzare DNS issues and am wondering if anyone
will be able to shed some light on this. A customer of ours started
recieving thousands of DNS requests for a wide range of domains,
mostly foreign. The requests are coming from a wide range of ips
most of which respond to nslookups "ie are nameservers". I have done
a whois on some of the domains and the 2 name servers having the
problems don't show up, I have also check to root servers and dont see
anything which would direct those domains to the name servers. Their
entire T1 is full from these requests about 1.2 meg. As the customer
is in the business of web hosting they can kill named nor can they
put up a packet filter to fix this. Also because there are literally
hundreds of diffrent domains both preforming the lookups and being
looked up it is not feasable to call the admin of each one to work
this out. Anyone have any ideas?
It is possible that their their server started claiming false authority
for a tld (eg. com) and polluted some caches or another server started
claiming it was authoritative and polluted some caches. That would mean
that these broken servers now think that your customer's server is
authoritative for some tld.
The thing to do to verify that would be to check to see what some of the
servers that are querying your server think are the authoritative servers
for .com, etc. Then, if you find that they do think your customer's
server is authoritative, have them dump their cache to try to track back
where they got that record from, etc.
Oh, and make everyone upgrade their version of BIND. Unfortunately, far
too many people refuse even when they know their whole world can be messed
up by a broken nameserver or two unless they upgrade.
If the above is the problem, then there isn't really any short term fix.
You just have to get the source of the false authority records to stop,
then wait until TTLs expire.
Oh, and make everyone upgrade their version of BIND. Unfortunately, far
too many people refuse even when they know their whole world can be messed
up by a broken nameserver or two unless they upgrade.
To summarize for lazy people (since others would have upgraded
already...): your DNS is vulnerable to manipulation (both on purpose and
by accident; this is _NOT_ some rare thing, but happens more and more),
the machine you run BIND on is vulnerable to root compromises, your job is
vulnerable.
While we are on the topic, are there any known cache pollution problems
with 4.9.7 that are fixed in 8.x?
I had thought that those problems were fixed in 4.9.7 but I keep think I
seeing other people's 4.9.7 machines vulnerable to them.
While we are on the topic, are there any known cache pollution problems
with 4.9.7 that are fixed in 8.x?
Not that I'm aware of.
I had thought that those problems were fixed in 4.9.7 but I keep think I
seeing other people's 4.9.7 machines vulnerable to them.
If you find one, please let us know asap. 8.1 fixes other problems
(performance related) -- cache pollution prevention should be consistent
between 4.9.x and 8.1.x. If there is a problem, we'll need to find it, fix
it, and make a new release.
This morning, Chris Yarnell from NASA pointed out the response
to "dig @a.root-servers.net . soa":
. 2D IN NS PANIC.WEBTEAM.NET.
. 2D IN NS TORGO.WEBTEAM.NET.
PANIC.WEBTEAM.NET. 2D IN A 207.67.50.8
TORGO.WEBTEAM.NET. 2D IN A 207.67.50.7
I don't know how this happened, but it doesn't look pretty. =)
(The "standard" zone files were okay -- .COM, .NET, etc...)
/cah
How did this happen anyway? InterNIC? Postel? Doesn't this error imply that
a percentage of the Internet was unresolvable by the entire planet?
It would really be great to find out what chain of events got us a root
nameserver on our network; not that we wouldn't mind donating the resources
to run one - it would just be nice to know in advance.
Maybe we can get bilateral peering with BBN since we have a root server,
Ryan Brooks
ryan@inc.net
CTO, Internet Connect, Inc.
After looking at the DIG output posted I would have to agree with how this _should_
have manifested itself (mispelled TLDs only, because of the SOA) - But packet dumps
showed the lookups contained lots of valid domains, tons of stuff in .mg, .il,
.uk, .gov and to a latter extent .com and .net (but only largish sites like excite,
yahoo, etc).
Ryan Brooks ryan@inc.net
CTO, Internet Connect, Inc.
> . 2D IN NS PANIC.WEBTEAM.NET.
> . 2D IN NS TORGO.WEBTEAM.NET.
> PANIC.WEBTEAM.NET. 2D IN A 207.67.50.8
> TORGO.WEBTEAM.NET. 2D IN A 207.67.50.7
How did this happen anyway? InterNIC? Postel?
M.I.B.H., no doubt.
Doesn't this error imply that a percentage of the
Internet was unresolvable by the entire planet?
Luckily not. Those servers are running with recursion enabled. So they
sent back a lot of nonauthoritative answers, which were treated as server
failures but forwarded anyway. At least BIND would have done that. Had
the above servers been configured with recursion disabled, then the above
delegation (coming as it did as an authoritative answer from a bootstrap
source -- A.ROOT-SERVERS.NET) would have pretty much rocked the e-commerce
market. Thus do we see that the least secure part of DNS are the procedures
and people, not the protocols or implementation. That's not a slam on the
InterNIC, but it could be correctly taken as a hint that the new IANA has
some serious procedural work to do regarding change control and publication.
I'm not sure what non-BIND servers did, of course. (They aren't common yet.)
Maybe we can get bilateral peering with BBN since we have a root server,
That's what worked for me :-). Except that I'm perfectly willing to say in
public that I get transit connectivity from BBN (and others) and it's great.
