DNS problems to RoadRunner - tcp vs udp

There is no call for insults on this list - Rather thought this list was
about techincal discussions affecting all of us and keeping DNS alive
for the majority of our customers certainly qualifies.

We/I am more than aware of the DNS mechanisms and WHY there are there
trouble is NO DNS server can handle directed TCP attacks even the root
servers crumbled under directed botnet activity and we have taken the
decision to accept some collateral damage in order to keep services
available. We are a well connected university network with
multi-gigabit ingress and egress with 10G on Abilene so we try to
protect the internet from attacks originating within our borders AND we
really feel the full wrath of botnets as we do not have a relatively
slow WAN link to buffer the effects.

Yes - we are blocking TCP too many problems with drone armies and we
started about a year ago when our DNS servers became unresponsive for no
apparent reason. Investigation showed TCP flows of hundreds of
megabits/sec and connection table overflows from tens of thousands of
bots all trying to simultaneously do zone transfers and failing tried
active denial systems and shunning with limited effectiveness.

We are well aware of the host based mechanisms to control zone
information, Trouble is with TCP if you can open the connection you can
DoS so we don't allow the connection to be opened and this is enforced
at the network level where we can drop at wire speed. Open to better
ideas but if you look at the domain in my email address you will see we
are a target for hostile activity just so someone can 'make their bones'.

Also recall we have a comittment to openess so we would like to make TCP
services available but until we have effective DNS DoS mitigation which
can work with 10Gb links It's not going to happen.

This could be a real problem. Of course, one solution is to take a
shotgun, point it at your foot, and pull the trigger. As you noted, a
long history of operational experience suggests that most resolver
traffic is UDP, which means that you won't have that many problems from
doing this. However, you can kiss that "5 nines of availability" you
claim to be interested in goodbye.

Nathan Ward and Sean Donelan have covered most of the points I would have
made had I written this message earlier. I will also point out, however,
that you don't even necessarily need a load balancer to do the tcp/53
screening, but there are a lot of neat and clever things that you could
do (some depending on with or without).

Given the relatively low level of usage for TCP DNS lookups, plus the fact
that it ought to be trivial to automatically identify hosts that are doing
TCP things that they shouldn't be doing (one zone transfer request might
be a reasonable thing, as many old-timers might do that sort of thing, but
won't try repeatedly if the server refuses), it kind of mystifies me as to
why you seem to imply that this is so difficult.

For example, it only took a few minutes to come up with this, for FreeBSD,
which takes advantage of the radix table support in ipfw.

#! /bin/sh -

grep -i axfr /var/log/messages | grep "zone transfer .* denied" | (
        while read a b c d e f g h i j k l m n; do
                g=`echo "${g}" | sed 's:#.*::'`
                # Maybe do some other processing ... a bad example: log it
                # echo noticed "${g}" "${j}" | logger -p info -t protector
                # Put out the IP's we want to consider filtering
                echo "${g}"
) | sort | uniq -c | (
        while read a b; do
                if [ "${a}" -gt 3 ]; then
                        2> /dev/null ipfw table 1 add "${b}"/32 && (echo banned "${b}" "${a}" | logger -p warning -t protector)

Needs to have a rule like this installed in the system firewall table:

% ipfw add [nnn] deny tcp from table\(1\) to me 53

I think a real solution would be more sophisticated than this, but it's a
starting point. This sort of thing limits "collateral damage" to hosts
that have attempted an unreasonable number of zone transfers, which is
still unpleasant, but is less likely to break legitimate uses.

... JG

In addition to the BCPs already mentioned by Sean and Nathan, a good detection/classification/traceback system plus S/RTBH can be helpful, and there are commercial DDoS mitigation services/scrubbers available from various SPs/vendors which have DNS-specific functionality, as well. Blocking TCP/53 is definitely not an optimal solution, as many have already pointed out.