Hello all,
Can some of you with larger networks let me know about the volume of the DoS attacks you have experienced lately? Our experience has been that the volume (not just occurrence) is going up significantly and I'm curious on the size of attacks that people are experiencing. For reference, while a year or two ago we used to get 50-100 meg attacks, now we're getting 500+ megs.
Thanks
are you seeting these attacks be related to the lack of
anti spoofing filters? where do they tend to be originating these
days?
i suspect that 1) smurf amps that are still not fixed, 2)
high speed connectivity at homes (cable, .. some dsl still,) are allowing
people to send spoofed packets at higher rates.
that combined and the number of windows based servers that
have been exploited (nimda, etc..) and those can be used also to send
spoofed packets at higher rates.
- jared
We have anti-spoofing filters applied, however apparently a large number of
ISPs obviously still see them as unnecessary. The attacks are a combination of
spoofed and real IP's.
The trend there seems to be that if the attack is high PPS but low
bandwidth, the majority of those are spoofed. Now a recent trend has been lower
PPS (increased size) and high bandwidth. The ones that we have been able to
track successfully are coming from real sources, and have indeed been due to
things such as nimda.
There have been several instances of people that were caught doing this
against us with approximately 1000 - 1500 servers under control via nimda, but
being able to notify the owners of all those servers is next to impossible.
I don't have a large network, but I had three yesterday morning
between 7 and 10am MST and apparently one last night between 11:30pm and 2am
MST that rippled through until 5am. That is way high. We typically see one
every six months or so (modulo worms). These appeared to be customer hosts as
unwitting dDoS participants... smaller than usual effects probably because we
had participants/sources rather than targets, but one yesterday was big enough
to take us down. Unix servers. No spoofing or amps involved (we filter).
High pps, average packet size down to 66 bytes. Didn't snag a capture.
These were not nimda or any form thereof as we have cut off
folks who were not fully patched.
...Barb
Our network is pretty much entirely end users. At the moment, we're
seeing a non-spoofed DDoS attack that's been ongoing for several days.
I've been trying to track and block, but the problem is that it seems
to be many different users with infected machines and only one or two
at a time are actually sending packets (SYNs at that, so not so easy
to blanket filter even in the short-term) at any given time. I watched
it for several hours and I would see one user send 10-50k packets then
stop, then the next user, etc. In the whole time I was watching I never
saw the same IP twice. I thought it could be spoofing, but as I ran
pings on whichever source IP I saw I got no response, then when they
stopped attacking I would start to get ping responses. I'm still at it,
but as I approached 100 unique sources I realized there's probably not
a lot of hope of effectively blocking it. I could filter the destination
for that entire pop, but:
a) I can almost guarantee I would be "administratively prohibited" from
doing so, given the popularity of the site in question.
b) It's a major website with gobs of bandwidth, which thus far seems
entirely unaffected by the attack. I am contacting them to verify
this, but every time I've checked the page comes up instantly and
there's no latency to speak of in traceroutes.
c) The amount of time the filter would have to stay in place is
unknown (so same reason as a) basically) because of the amount
of administrative hassle to track down every user and not only
block them but also get them to fix it (which, without having any
real idea what agent they're running, will be difficult in itself).
We are still working at this, but I'm wondering whether any other DSL
or cable providers out there are seeing similar.
-c
I don't run a large network, but I am curious and will help where
possible.
Are you able to say what kind of DoD attacks are taking place?
ICMP Floods? TCP Floods? UDP Floods? A mixture?
If you feel the src addr is spoofed, have you taken a packet capture and
looked for similarities in the packets?
I read a paper about 5 months ago where someone had worked very hard at
analysing the differences in packets generated by various DoS agents.
Maybe you should attempt to trace them back?
If they are Smurf attacks, I may be able to help more, let me know.
On a similar point, does anyone know how to disable Directed IP Broadcasts
on Lucent Portmaster routers?
Since years, IRC (users and/or servers) gets dDoS... We also see a grow of
the dDoS attacks. For example on Undernet some servers get attacked every
day with 100+Mbps for a few minutes, and sometimes for long long hours...
Those attacks are usually comming from users - IRC Operators conflicts,
those users think they may ask anything to an OPER with the power of a dDoS.
We try to provide a free service, and all of us know how it is hard to get a
host with good connectivity for free and on the other side we see those
young 'script kiddies' playing around with hundreds of compromised hosts
like a game and they have no idea how much it costs to all the flooded
networks... Unlikely I have to say that most of these 'script kiddies' are
from Romania. I dont know why it's so many times comming from them....
If you run an well dDoS'ed IRC Server on your network I have a solution for
you... not the best one, but still technically working..
get a /24 (be carefull that there is no bigger network announced which would
include it!!! i mean like if you get 10.10.10/24, 10/8 would include it)
Get a box, and run Zebra BGPD, which will announce that /24 to your network.
Then do a script which monitors the traffic to the irc server, and on a
certain threshold, kill BGPD. wait a certain time, like 15minutes or so, and
restart BGPD. It would be nice to check the traffic every minute and if 2
consecutive checks are positive kill bgpd. That mean that you may be able
to STOP dDoS to irc servers within 2-3 minutes...
just my 0.00001 EUR
Cheers..
Pascal
What about BGP route flap dampening, people use that, don't they?
-Paul
Something that people may want to consider doing is
that assuming you are using hardware/software that can support
rate-limit of specific packet types/rates, you could
generate some rate-limits to limit specific types of traffic
to various ranges.
You can also use these initally to sample the traffic
sufficently that one can determine what your typical rate is.
Example:
Create access-list that matches icmp echo+echo-reply
Assuming a ds3/oc3/oc12 uplink, you can create a rate-limit
on the router that limits the traffic to 1.5M with a burst to 2M.
You can also do "sh int rate" on a cisco router to determine if
these rates are what you are typically forwarding. You obviously need
to adjust these somewhat over time as your traffic and network patterns
change.
You can do the same for tcp-syn http, etc.. by creating multiple
rate-limits. (this is assuming cisco devices running 12.0S w/ the
appropriate linecards that support this feature set just as an example.
your mileage and network topology/linecard mix may not completely support
this. please consult your appropriate vendor as most vendors these days
can support these features. i'm just using cisco example as baseline).
Once you figure this out you can then police your network traffic
or possibly apply the same types of rate-limits on customer facing
interfaces (esp. colo that tends to have high bw avaial but do't use it
all the time).
This is not based on any real-life experiences so collect your own
data, but this may be useful for people to do.
The problem of internet security and keeping your host(s) secure
I think is the most important. Most software vendors are starting to ship
[almost, if not] secure out of the box at this point. The challenge is
upgrading all the existing hosts. There doesn't appear to be a good way
to notify everyone unless it turns into a "cnn" type event where all the
nightly news people are covering it. This also misses a large portion of
the international community. The local media should take it upon themselves
to help notify people to update their machines as well as the software
distributiors and hardware people that sell prepackaged software (windows
for example) that is installed. include the cost of postage and
printing costs for mailing the users a postcard for the next 3 years once
a month with all the things they should check their machines for.
it's a challenge. hopefully everyone involved can step up and
secure their networks to the [known] intrusion methods that allow abuse.
- jared
I think the point is that (despite everyones thoughts
that use it) IRC is not considered a super-important network service
these days. If the irc server is dampened or the attack can't reach it
it just penalizes the compromised host(s) network(s) more than the
person who hosts the irc server.
- jared
One thing we have done is rate limited the amount of traffic to the IRC
server one of our downlinks has. That seems to 'cure' most issues.
Of course, it still doesn't stop someone from hitting other parts of the
infrastructure, which seems to be all the 'rage' with kiddies now.
*sigh*
-Eric
<snip>
I don't know if I can totally agree with that 
I have run an IRC server, and have been the subject of a variety of DoS
and DDoS attacks in the past.
Some of the attacks have had almost an intellegence behind them.
When a server's immediate uplink is a T1(or equiv), there has been just
enough traffic to flood it (eg, T1+1mb).
When the server's uplink has been a 155Mb link, there's again been just
enough traffic to flood that (eg, 155+10Mb).
In each case, turning off the ircd, or blocking ICMP / TCP / UDP /
whatever packets going to that server upstream have stopped the floods in
seconds.
This leads me to believe there are at least 2 types of flooders our there:
Flooders who are careful how much of their resource their use, and
flooders who don't care and would try to cram 1Gb/s down your tiny T1
given the chance.
In either case I believe IRC can be considered an important service, if
only for the reason, that it can keep the attackers attracted. If there
was no IRC I'm sure they'd go after more critical services!
jared@puck.Nether.net disait :
Something that people may want to consider doing is
that assuming you are using hardware/software that can support
rate-limit of specific packet types/rates, you could
generate some rate-limits to limit specific types of traffic
to various ranges.
rate-limite and/or traffic filtering may be available on some
box (GSR) but cannot run concurently with other feature (NetFlow).
That is the biggest problem i see trying to put ACL or rate-limite
on GSR boxes. I think the Cisco is working on it.
Output ACL on some GSR linecard (engine 0/1 i think) make Netflow
inactive on _all_ line card :-((
Thus, we cannot put any ACL nor rate-limit on customer connected on GSR
boxes .... and it is hard to explain to customer that this is because
of vendor limitation !!!
The only tool available for these Customers is blackhole for identified
/32 .... bad granularity !
Vincent.
nguilbaud@chello.com disait :
Vincent,
The Cisco ISE aka Engine 3 cards for GSR allow you to combine those
features, you can even i/egress traffic police or even shape based on
access-list. You still have some constraints but nothing compared to the
E0/1/2.
It looks getting better and better, but large GSR users have
95% LC E0/1/2 .... and i bet 100% customers are connected on E0/1/2 !!
Even single POS oc48 LC are still E2 ....
I am talking about problem we have today.
E3 and above LC will be popular for access in 2003 i guess.
Vincent.
The DDOS attacks we see are generally a mixture. I couldn't care less about the
ICMP (which are frequent) since we already rate-limit that, and protect against Smurf
attacks as well. UDP and SYN floods are harder to try and prevent or rate-limit with
our type of customer base. The destination of our DDOS attacks vary greatly,
normally we don't see the IRC servers as the overwhelming majority.
Spammers, shell hosts, our equipment, and even just normal web sties (that others
may not like) are all equal targets of DDOS attacks. Granted there are always the
repeat offenders.
Most of the details I have been able to see from DDOS attacks are just logs of
Source/Destination IP and packet type, not an actual packet capture to be able to
examine the packet. I would be interested in knowing what article you read also.
Avleen Vig wrote:
I seem to have just found out that ACLs and sampled NetFlow can
both be configured concurrently on routers running IOS >= 12.0(18)S.
This is in theory, not something I have tried (yet), and may depend
on the specific LCs you have in your router.
I don't know if/where the feature has been implemented on other
release trains.
Joe
jabley@automagic.org disait :
> rate-limite and/or traffic filtering may be available on some
> box (GSR) but cannot run concurently with other feature (NetFlow).I seem to have just found out that ACLs and sampled NetFlow can
both be configured concurrently on routers running IOS >= 12.0(18)S.
All can be configured concurently .... but you have a message
from line card that Netflowx has been stopped because another feature
is activated.
Below is feedback i received from Cisco :
1. There is no incompatibilities on E0,1,3,4 but some features are not
available on some E
2. For E2 in 17S, here are the priorities:
ACLs
SNF
PIRC
IP Coloring
BGP Policy accounting
FR Traffic policing which is not FR traffic shaping
Beside, output ACL are done at ingress (before forwarding),
thus output ACL activate input filtering on all LC ...
Vincent.
jabley@automagic.org disait :
> > rate-limite and/or traffic filtering may be available on some
> > box (GSR) but cannot run concurently with other feature (NetFlow).
>
> I seem to have just found out that ACLs and sampled NetFlow can
> both be configured concurrently on routers running IOS >= 12.0(18)S.All can be configured concurently .... but you have a message
from line card that Netflowx has been stopped because another feature
is activated.
Right. That is the behaviour that I have been led to believe
no longer happens past 12.0(18)S; supposedly, on 12.0(18)S and
greater, ACL and SNF can both be configured concurrently such
that both features work concurrently.
If you know different, I would love to hear about it 
Below is feedback i received from Cisco :
1. There is no incompatibilities on E0,1,3,4 but some features are not
available on some E
2. For E2 in 17S, here are the priorities:
ACLs
SNF
PIRC
IP Coloring
BGP Policy accounting
FR Traffic policing which is not FR traffic shapingBeside, output ACL are done at ingress (before forwarding),
thus output ACL activate input filtering on all LC ...
That gels nicely with what I have been told; an input ACL on
an interface disables SNF on that interface, while an output ACL
on any interface disables SNF on the entire router.
Joe