RE: Network Operators and smurf

All,

Given the contributions to this thread have been mostly theoretical in
nature, I'd like to share an experience of mine that in some ways
negates some of the propsed solutions to smurf attacks in the context of
smaller ISPs.

Recently, one of our downstream customers was subject to a smurf attack,
and we placed an access-list on our egress interface to the customers
network. The customer hangs of an SMDS cloud - our link to the cloud is
at 34 Mbps, his T1. We were blocking echo replies destined for his
network. We are connected upstream at 45Mbps. As the attack
intensified, router CPU Utilization jumped to 99%, and the input queue
on our inbound HSSI was at 75/75. We started dropping packets at a rate
of about 7000/sec. The attacks were coming in from all over the world.
The NetFlow cache was growing at an alarming rate, and, after a while
the HSSI just DIED. As the HSSI bounced, our BGP session bounced with
it, causing some mild route flapping (Not vaccilating enough to be
damped, but enough nonetheless). Eventually the attack subsided, and
all went back to normal, but for a period of time, say 10 minutes, we
had a 7507, 64megs, RSP2 WITH the HSSI on a VIP2 on its knees.

We decided that parsing NetFlow logs would give us a better idea of who
was amplifying the attacks, and with a simple shell script, we were able
to build a database of ASNs, with admin contacts from RADB/RIPE/ARIN.
We are planning on sending emails to these customers to ask them to stop
amplifying smurfs (script does that too), because this is unacceptable.

My point, then is this: Filtering echo replies is/can be a futile
attempt at preventing this type of attack. I watched a 7507 die
defending against one attack. More recently, we got hit so hard that
the router was screaming without ANY access-lists blocking ICMP
echo-replies, perhaps because there was no real fast-switching taking
place (each source is different, so the first packet is process
switched. Our NetFlow cache went from 3900 flows to 27000 flows in
about 4 mins.) And, as we were not amplifying the smurfs, source
address verification is a moot point. I am all for allowing these
netblocks time to implement this type of filtering (layer3 to layer2
broadcast translation prevention), but not for very long. It appears
the best way to light a fire under someones rear end is to publicly
shame them into acting. For those who don't act quickly enough, there
can be no quarter.

If I appear hostile, I am...

PS

If anyone has similar experiences, please share them, as there has
already been enough rhetoric filling this thread, and it is clear that
everyone knows the solution lies at edge and beyond, not in the core.

-Christian Martin

==>network. We are connected upstream at 45Mbps. As the attack
==>intensified, router CPU Utilization jumped to 99%, and the input queue
==>on our inbound HSSI was at 75/75. We started dropping packets at a rate
==>of about 7000/sec. The attacks were coming in from all over the world.

Have you read the smurf document found at
http://www.quadrunner.com/~chuegen/smurf.txt?

I'd be interested to know what version of code you were running.

I've seen a provider drop over 120 Mbps of smurf traffic in access-lists
for over an hour and the routers weren't affected one bit.

IOS CA & CC code 11.1(13.5) and later have a fix to the code which handles
access-list drops--called "fast drop"--which fixes some inefficiencies in
packet handling.

***READ*** the document at the URL above. It's amazing how much that URL
has been advertised, through all the advisories, through the NOCs, etc.,
but with the ongoing thread over the last few weeks it almost appears that
a lot of people either haven't heard about it or haven't read it.

Of course, it's been put into mail messages 9 times on NANOG already:
chuegen@quad:3:~>grep "quadrunner.com" mail/nanog | grep "smurf" | wc -l
      9

/cah

Since I have started blacklisting people, the list has grown to more than 40
netwroks.

However, only ONE, and that one was very short (< 5 minutes) smurf has taken
down a customer circuit or our IRC server since the last edits were made to
the amplifier list over a week ago.

I'm going to try to post a web page on this tonight. What we're doing here
WORKS. It inconveniences a few people (those who amplify smurfs), but it
WORKS and it STOPS the smurf attacks from burying your connections.

Our core routers don't even get mildly bothered doing the discards.

What config options/access-lists are you using. If Martin says his
routers choked, and you say yours don't, I'd like to know (as would
anyone, I think) what you are running and how you are running it. I
assume you are using access-lists. Are you using NetFlow? On which
interfaces? Are you blocking inbound echo-reply? To where? Everywhere
or just your core network? The problem I'm faced with, is I'm currently
blocking ICMP to my core networks, NOT to customers or dialup ports. They
will REALLY bitch if I start doing that. I'm going to change it to block
echo-reply to those networks only.

Currently, the measures "I" am taking is:

o outbound access-list permitting ONLY networks which source on our
router.

o inbound access-list denying ICMP (which will be modifiec to block ICMP
echo-reply only) to core networks along with .0 and .255 address blocks.

o no ip directed-broadcast on ALL interfaces

o ip route-cache flow (so I can get some information on these *ssholes)

What else do I need?! What am I doing wrong with the above (if anything)

TiA...

I block *ALL* packets from any netblock which is implicated as a smurf
amplifier, on the first offense.

I will remove those blocks when I can PROVE that they can no longer be used
as a smurf amplifier. To date, NOBODY on the list has come forward and said
"we've audited and fixed, please remove the block".

PSU (which is on the list) said "we're looking into it" but that was more
than two weeks ago! How long does it take to telnet into your routers and
check the ethernet interfaces for the correct configuration? A day or so?
Perhaps, even if you have a HUGE netwokr.

It does not require two weeks.

Blocking just ICMP is pointless - block EVERYTHING. The only way we ever
get this fixed is to make it HURT to be on the blacklist.

>Since I have started blacklisting people, the list has grown to more
>than 40

Currently, the measures "I" am taking is:

. . .

What else do I need?! What am I doing wrong with the above (if anything)

One obvious thing missing from your strategy is that you aren't
blacklisting the SMURF amplifiers by publicly announcing who they are on
this list and by not blocking the entire netblock in which they reside.
That is what Karl is doing.