handling ddos attacks

Mailman List Mirror (Read Only)
1

I've been trying to find out what the current BCP is for handling ddos
attacks. Mostly what I find is material about how to be a good
net.citizen (we already are), how to tune a kernel to better withstand
a syn flood, router stuff you can do to protect hosts behind it, how
to track the attack back to the source, how to determine the nature of
the traffic, etc.

But I don't care about most of that. I care that a gazillion
pps are crushing our border routers (7206/npe-g1).

Other than getting bigger routers, is it still the case that the best
we can do is identify the target IP (with netflow, for example) and
have upstreams blackhole it?

Thanks,
-mark

2

I too would be interested if someone could point a good white paper
for cisco DDOS protection mechanisms and best practices in general.

3

or acl it.

  some providers offer blackhole services where you can inject
a route to them via bgp over the same session (with communities) or
over a different session that just takes blackhole routes..

  that can be used by you to cause them to null0/discard the
traffic within their network automatically..

  with junipers being used commonly these days, and their
ability to write long, complex firewall filters, I think you're seeing
more people do fancier things.. I've placed filters for at least
one customer (for the duration of a DoS) that match on specific
packet sizes or packet ranges of a specific type.

  The more you know about the profile of the attack you
have going on, the better others can help you mitigate it..

  - jared

4

I've been trying to find out what the current BCP is for handling ddos
attacks. Mostly what I find is material about how to be a good
net.citizen (we already are), how to tune a kernel to better withstand
a syn flood, router stuff you can do to protect hosts behind it, how
to track the attack back to the source, how to determine the nature of
the traffic, etc.

This depends entirely on your definition of handling. To some people this
means shutting down the victim to save the network as a whole. To others
this means keeping everyone running smoothly, including the victim. The
latter is preferred of course, but it is not for those who aren't willing to
pay for it.

But I don't care about most of that. I care that a gazillion
pps are crushing our border routers (7206/npe-g1).

Other than getting bigger routers, is it still the case that the best
we can do is identify the target IP (with netflow, for example) and
have upstreams blackhole it?

It sounds like you're willing to blackhole the victim. In that case, yes,
netflow is highly useful in finding out just who is getting attacked. Once
you have that information, you can either manually contact your upstreams to
have them null route the destination IP, or better yet, arrange ahead of
time for a way to send properly tagged BGP announcements to them to
blackhole /32s anytime you want.

The alternative is to get bigger links, bigger routers, and protect the
host. For bigger links and bigger routers, keep PPS in mind. Some attacks
are large packets and large bandwidth, with low PPS. Other attacks are low
bandwidth, but high PPS. I get hit pretty regularly with 500k-600k PPS of
SYNs. While this only adds up to a few hundred megabits of traffic, that is
a lot of PPS for many routers, firewalls, servers, or whatever else they
might hit. Junipers, for example, have no problem with high PPS.

Second, you have to figure out how to protect the host(s). We've gone with
Riverhead (recently bought by Cisco) and they work quite well. I've seen
attacks as high as around 650k PPS of spoofed SYNs, and the site running on
a single (relatively weak) server remains up and generally unaffected by the
attack.

5

A paper based on a presentation I did at the PAIX peering forum in
December is here: http://www.stevegibbard.com/ddos-talk.htm

I should probably update it a bit, but that may not happen any time soon.

Slides from another presentation at the same conference are here:
http://www.prostructure.com/content/research/presentations/ddos_intro/

-Steve

6

Is there any quantification on what qualifies as a Large DDOS attack and perhaps a comparison of what type of routers can/can't handle such a load? Typical DDOS's that I've seen are 10-20X the normal incoming packet rate, upto and over 1Mpps. Having to switch that amount of additonal load has a tremendous impact on linecard CPU and any amount of additional features to try and protect your customer will sometimes result in a degradation to *everyone* not just the target. In my experience calling the upstream provider and having it blocked is still the only thing that can be done. When working on the backbone I've spent hours tracking the majority of flows back to one or more peering points and blocking it there where the attack isn't as concentrated and thus safer to filter.

-Brent

7

The dearth of comprehensive BCP asserting the end-all-be-all for
DDoS is likely and largely due to the lack of an end-all-be-all
DDoS.

The range of variants, strains, chewy fillings and flavors of
fuxor out there beg different techniques for alleviation, so
prescribing a single poultice for blanket application does not
seem to be in wide practice outside marketing stratagem and
other blustering. The resources requiring protection and
receiving priority, as well as the trade-off in exacting
reactive measures, also have a say in how things are managed.

In general, however, yeah...identifying the source or target
is a must. Or a source port or destination port or protocol
type or packet size or point of ingress/egress...the list of
signature-worthy candidates is significant and also determines
how a DDoS is triaged.

The only thing that can be said for certain is that *some*
unifying factor must be discovered. :stuck_out_tongue: Furthermore, how you do
that and what you do with that is a fluid thing, and further
refinement or definition of the type of DDoS you are seeking to
relieve may be required before you will be able to root out an
attack management template that is worth its salt.

Blackhole servers, sinkhole routers, IDS, extrusion detection,
heuristic baselining, and definitely bigger routers never hurt
this effort either. :wink:

If you are able to elaborate on what you might be seeking to
accomplish on- or off-list, I will try to proffer any
appropriate resources I have available.

Good luck.

--ra

8

jared@puck.nether.net disait :

>
> I've been trying to find out what the current BCP is for handling ddos
> attacks. Mostly what I find is material about how to be a good
> net.citizen (we already are), how to tune a kernel to better withstand
> a syn flood, router stuff you can do to protect hosts behind it, how
> to track the attack back to the source, how to determine the nature of
> the traffic, etc.
>
> But I don't care about most of that. I care that a gazillion
> pps are crushing our border routers (7206/npe-g1).
>
> Other than getting bigger routers, is it still the case that the best
> we can do is identify the target IP (with netflow, for example) and
> have upstreams blackhole it?

  or acl it.

  some providers offer blackhole services where you can inject
a route to them via bgp over the same session (with communities) or
over a different session that just takes blackhole routes..

  that can be used by you to cause them to null0/discard the
traffic within their network automatically..

At last Ripe meeting, i made a presentation about the way France Telecom
is handling DDOS attack :

  http://www.ripe.net/ripe/meetings/ripe-48/eof.html#nocexp

Slides at

  http://www.ripe.net/ripe/meetings/ripe-48/presentations/ripe48-eof-gillet.pdf

We presented our practice from a NOC perspective (ACL, blackhole, sinkhole,
netflow, sample, ... etc) and our next steps.

We proposed to give this presentation at coming Nanog, but we were not
so succesfull. Next nanog meeting maybe ...

Vincent.

9

There's lots and lots of really-useful-very-often-multi-vendor
stuff here:

ftp://ftp-eng.cisco.com/cons/isp/security/

In particular, under the bootcamp and CPN-summit stuff. Though it may
seem vendor-specific per logos and the like, I know of several (more
than three) vendors that have contributed to this content, most of which is
very practical and generally informative, and should be applicable to most
deployed vendors. There's also some VOD stuff here that expands some
areas of the content:

http://www.getitmm.com/bootcampflash/launch.html

HTH,

-danny

10

mark@noc.mainstreet.net (Mark Kent) writes:

I've been trying to find out what the current BCP is for handling ddos
attacks. Mostly what I find is material about ... But I don't care
about most of that. I care that a gazillion pps are crushing our border
routers (7206/npe-g1).

Other than getting bigger routers, is it still the case that the best
we can do is identify the target IP (with netflow, for example) and
have upstreams blackhole it?

that seems hardly worthwhile. ddos is astonishingly easier to launch than
to defend against. if you stop a flow the attacker *might* get bored and
decide to do something else, but they could also decide to attack you from
a different direction, or wait two days and do it all over again, and every
time they attack and you defend it's 10 minutes of their time and 10 hours
of yours.

far better to involve law enforcement and get some bad guys arrested, if
you possibly can. this changes your costs from 10 hours to 15 hours but it
actually puts some chips on the table and makes the game worthwhile.

11

mark@noc.mainstreet.net (Mark Kent) writes:

> I've been trying to find out what the current BCP is for handling ddos
> attacks. Mostly what I find is material about ... But I don't care
> about most of that. I care that a gazillion pps are crushing our border
> routers (7206/npe-g1).
>
> Other than getting bigger routers, is it still the case that the best
> we can do is identify the target IP (with netflow, for example) and
> have upstreams blackhole it?

that seems hardly worthwhile. ddos is astonishingly easier to launch than
to defend against. if you stop a flow the attacker *might* get bored and
decide to do something else, but they could also decide to attack you from
a different direction, or wait two days and do it all over again, and

every

time they attack and you defend it's 10 minutes of their time and 10 hours
of yours.

far better to involve law enforcement and get some bad guys arrested, if
you possibly can. this changes your costs from 10 hours to 15 hours but

it

actually puts some chips on the table and makes the game worthwhile.
--
Paul Vixie

Hey Paul !

Ok, I 'll buy that right now; we have a DDoS Attack on our core nameservers
from 66.165.10.24. Where do we start, do I call the police in Bellingham or
Washington State Police. We have blocked their ips but, we know they will
come in another way.

Peter

OrgName: Western Washington University
OrgID: WWU
Address: Computer Center
Address: 516 High Street
City: Bellingham
StateProv: WA
PostalCode: 98225
Country: US

NetRange: 66.165.0.0 - 66.165.31.255
CIDR: 66.165.0.0/19
NetName: WWU-RESIDENT-1
NetHandle: NET-66-165-0-0-2
Parent: NET-66-165-0-0-1
NetType: Reassigned
NameServer: VIKING.WWU.EDU
NameServer: HENSON.CC.WWU.EDU
Comment:
RegDate: 2002-08-15
Updated: 2002-08-15

TechHandle: JSW12-ARIN
TechName: Williams, J. Scott
TechPhone: +1-360-650-2868
TechEmail: scott@cc.wwu.edu

12

Call your local branch of the US Secret Service, if you're in the states,
and ask for their electronic crimes division. If you're not in the
states, contact your comprable local authority. They can work with you to
coordinate with other jurisdictions, etc.

You may have some luck directly with the local police at the point of
origin, but it generally helps to have a broader agency involved to
coordinate matters.

13

I'd love to hear from anyone who has actually been successful
prosecuting an attacker for launching a "distributed" DOS attack.
I suspect it occurs very INfrequently (with the recent trend in
extortion aside, as it often results in "paper trails") --
unfortunately.

Any pointers?

-danny

14

I too would be interested if someone could point a good white paper
for cisco DDOS protection mechanisms and best practices in general.

For Cisco specific ideas try:
http://www.ripe.net/ripe/meetings/archive/ripe-41/tutorials/eof-ddos.pdf
specifically slides 86-92 and 105-127.

-Hank