Inter-provider communications (Re: nobody @home)

By the time law enforcement has to be involved to convince a tier1 to
shut off their ddos sources, it's far past the point of complicity and the
preventable monetary damages have already occurred. You can bet someones
going to get sued.

Sadly, it's probably going to take a high profile lawsuit to get the
tier1s to shape up their act.

-Dan

that people seem to believe that the Tier 1 (what constitutes a tier 1
anyway in todays world?) just needs to throw a switch and turn off a Ddos
attack, but that they are too lazy to flip it.

Reality being a bit different, so lets check into what we have here.

Reality has it that there are:

several tens of thousands of customers, 100k+ interfaces for customers,
all terminated on broken hardware that cannot line rate filter on all
interfaces, 200k ibgp entries, entry point from several thousand peering
interfaces, mostly at OC12 rates or higher, thousands of routers, a
chronic shortage of staff because anyone who is any good at a customer
facing role and dos/abuse are customer facing roles, tends to burn out and
fade away very fast, normally up the engineering hierarchy, leaving the
job to fresh new people, armed with inadequate experience and lacking
tools to do the job.

A DDoS attack by definition is a hard one to trace, no matter what people
(vendors) would have you believe. Putting an acl to do a traceback? What
do we put in the acl, some DDoS attacks involving 500+ machines, each
being carefully rate limited to send a few packets, perhaps with different
information in each? Maybe putting an acl on will crash the router, and
the router cannot be code upgraded because a new and interesting
interaction with the new train tickles some other bugs, causing hard
crashes at random.

The govt. agencies are involved often, but the fundamental problems of
very large networks coupled with inadequate protocols and broken
implementations make traceback of DDoS attacks _very hard_.

This is not to say that some backbones aren't lazy about doing the job, I
suspect that is mostly because the people doing the tracebacks have
realized that it is almost impossible to do adequately with any chance of
success and tend to ignore it. This is not a good thing, but this is what
appears to be happening. On the other hand, people are beating on vendors
to treat this problem seriously and give operators proper debugging
abilities and better hardware. Also please realize that just turning off
someone's circuit because some j. random person called up and claimed it
was sourcing a DDoS attack is often prohibited by policy at various
networks, and an exception must be made by senior mgmt in the chain. If
every noc just started to turn off interfaces because of a phone call, the
results are easy to imagine.

/vijay

Well, let's take a better example, smurf amps.

I have some personal horror stories about running around in circles
getting tier1s to turn off their smurf amps originating from their own
routers or customers. Eg tier1 router was a smurf amp, it was smurfing, it
could be easily verified to smurf, but they would not disable the smurf
amp because it would have a "negative impact" on their customers. The
fact it was being actively used as a smurf amp didnt seem to matter to them.

This was in fact a case of "just flip a switch and turn off the attack".

I'm sure others on this list have their share of horror stories as well.

The hoops the public had to jump through the past couple years to get
tier1s to turn off their smurf amps is mind boggling. And there are
tier1s who are *still* actively running smurf amps in their cores.

I'm actually suprised noone has filed lawsuits over this. Or maybe someone
did and I missed it.

-Dan

Well, in light of all the gloom I would like to say that I had a good
experience with exodus/doubleclick, my network was recently the victim of
a smurf attack, one of the amps was doubleclick.net, I contacted exodus
about it and they (within an hour) put me into contact with
doubleclick.net who had someone call me, I was able to walk the person on
the phone through fixing the problem, and they are no longer a smurf amp.

It's nice to have a few good experiences..

FYI, I am not a customer of Exodus in any way.

      Matthew S. Hallacy
      XtraTyme Technologies
      Systems/Network Administrator