Well well, it looks like a Direct Connect circuit to Google was leaking the
route to this DMZ 153.7.233.0/24 back to Google via BGP.
Return traffic from Google (for only some fraction of DNS queries) was
passing back across this leaked route, and being dropped on this Direct
Connect peering point at Disney.
Gotta love it when a problem is solved, by the OP, within an hour of
resorting to mailing the NANOG community.
Gotta love it when a problem is solved, by the OP, within an hour of
resorting to mailing the NANOG community.
That's the way it is. Posting to a public forum always make you think
about the issue a second time, and that's what it takes.
The weird thing is that I've tried to cheat the system by thinking
without posting, and it doesn't work! Don't know why, but there appears
to be a difference between thinking and thinking
Well well, it looks like a Direct Connect circuit to Google was leaking the
route to this DMZ 153.7.233.0/24 back to Google via BGP.
Return traffic from Google (for only some fraction of DNS queries) was
passing back across this leaked route, and being dropped on this Direct
Connect peering point at Disney.
Gotta love it when a problem is solved, by the OP, within an hour of
resorting to mailing the NANOG community.
None of the NS records/delegations are in agreement. com delegations
don't agree with authoritative in disney.com, and disney.com's
delegations don't agree with studio.disney.com's NSen.
Ignoring the latency impact of your proposal, I wonder what would happen to
the world's authoritative servers if all users hit them directly rather
than going through large recursive resolvers that do caching? I'm guessing
it wouldn't be pretty.
Pragmatically speaking, I strongly suspect the increase in valid queries to authoritative servers even if all “large recursive resolvers” went away would be lost in noise of the overcapacity necessary to deal with even a lower-end DDoS attack.
Perhaps more interestingly, if said recursive resolvers on home routers would implement DNSSEC with RFC 8198 (and the owners of the authoritative zones would sign those zones), an entire class of DDoS attack would be mitigated. Further, if said recursive resolvers also implemented RFC 7706, latency to the root would be reduced and the risk of to the network behind that recursive resolver of a DDoS against the root of the DNS would be removed.
:I know it doesn't help your problem, but friends don't let friends use public DNS resolvers (Google, L3, Open DNS, etc.).
I've been experimenting with using Google's DNS resolvers for Google's
assorted domains. At some point, I keep meaning to add Google's address
space as in-addr.arpa domains, but just haven't gotten there yet.
Why? Just curious, that's all. Thus far, I haven't really noted any
major differences, but wasn't sure what to expect. Maybe something
would be notably faster/slower, maybe different results/ads/whatever,
I dunno. It just seemed reasonable to punt Google DNS to Google DNS
and see how things work. YMMV, void where prohibited.
A 10x increase in baseline queries is still a 10x increase (for whatever
value of "10" the real world would actually throw at us). Although small
by comparison, that still has to be made up in an increase in the overhead
for DDoS.
I'm also led to wonder how much worse it would be if all those CPE were
open recursives instead of open forwarders. I'd like to see CPE
manufacturers' decision making and processes improved BEFORE we start
encouraging them to go around ISPs' DNS servers or the large public
recursive clouds.
A while back, the Québec government, wanting to protect its gambling
monopoly, decided to force ISPs to block a list of gambling sites (list
drawn up by the gambling monopoly to block outside competitors).
Recently, Bell Canada went to government suggesting the government setup
a internet web site block list to prevent canadians from accessing
pirating web sites.
And of course, in the USA, the upcoming decision to drop Title II for
ISPs may result in large ISPs quickly starting to play tricks on DNS
(redirecting traffic to their own properties etc).
While all this is in its infancy and may not happen, this could have
serious impact on the architecture of DNS with large swaths of customers
bypassing their ISP's DNS services.
But it is more likely that everyone would be going to 8.8.8.8 instead of
running their own recursive server. But if the "free" DNS servers also
start to play games or charge money, then CPE equipment may start
including a full bind recursive server and bypass everything.
This is why it is important for network folks to educate politicians to
not play with the internet.
And it is believed that sold end user devices wouldn't just be
required to implement this blacklist themselves? This is reminding me
of the xkcd coming with the encryption and the wrench.
