Hello folks,
Here is a great move from one of the biggest NSPs, I'm
sure we will see L3, Sprint, UUNet and others will do
the same soon to gain more customers since DDoS is the
nightmare of the internet now.
http://biz.yahoo.com/prnews/040601/nytu051a_1.html
Thanks,
-J
Major providers such as Sprint and UUNet have had null route communities available for quite some time... Unless I am mistaken?
John Obi wrote:
Indeed.
However, the AT&T thing looks like a combination of Arbor PeakFlow:DoS for
automated DoS detection on the network, and what used to be Riverhead (and
now acquired by Cisco) for "traffic scrubbing" to allow normal traffic to
continue to be passed to nodes under attack.
COLT have been doing this exact same thing in the UK for a while now.
Simon
:: Major providers such as Sprint and UUNet have had null route communities
:: available for quite some time... Unless I am mistaken?
::
quoting the press release: "The mitigation option uses the principles
of analysis, filtering, scrubbing and diversion to protect against such
attacks."
So i would guess they are using riverhead and hoping they can scale
their mitigation clusters faster than DOS kids can scale their botnets.
That said, *golf clap* for doing something. Hope their system works as
well as their marketing. 
-jba
:: John Obi wrote:
::
:: > Hello folks,
:: >
:: > Here is a great move from one of the biggest NSPs, I'm
:: > sure we will see L3, Sprint, UUNet and others will do
:: > the same soon to gain more customers since DDoS is the
:: > nightmare of the internet now.
:: >
:: > http://biz.yahoo.com/prnews/040601/nytu051a_1.html
:: >
:: > Thanks,
:: >
:: > -J
:: >
:: >
:: >
:: >
:: > __________________________________
:: > Do you Yahoo!?
:: > Friends. Fun. Try the all-new Yahoo! Messenger.
:: > http://messenger.yahoo.com/
:: >
:: >
::
Great. So now we're going to see all the spam filtering issues (false positives...) for IP in general? Am I just being cynical or is the port-80-only-internet coming closer and closer?
John Obi wrote:
... since DDoS is the
nightmare of the internet now.
The sad fact is that simple ingress and egress filtering would
eliminate the majority of bogus traffic on the Internet -- including
(D)DoS attacks. If all ISPs would simply drop all outbound packets
whose source address is not a valid IP for the subnet of origin,
and all inbound packets that do not have valid source IP addresses,
the DDoS problem would be (for all intents and purposes) fixed. If
proper filtering was done, then any DoS attacks would have to have
either valid source IP addresses, or IP addresses that spoofed IPs
within their network of origin. In either case, identifying and
shutting down the attackers would become a greatly simplified task
compared to the mess it is today.
Why no filtering by ISPs? "Because it takes resources and only benefits
the other guy" -- unless your network is the one under attack.
Maintenance of the ACLs should not be the issue. A single ACL for each
subnet would be all that would be required for egress filtering. About
30 ACLs on an inbound border router would be required for ingress
filtering. Keeping the ingress ACLs current is a brain-dead task -- just
subscribe to the bogon mailing list at cymru.com.
ACLs have had a bad reputation for greatly slowing down routers. That
may have been true in the past, but properly written ACLs do not seem
to have a significant impact on most new routers. Yes, they may cut
peak through-put a few percent -- but if you are running that close to
the edge, it is time to upgrade anyway.
IMHO, there is absolutely no excuse for not doing ingress and egress
filtering. In fact, if you are an ISP, I would argue that you are
negligent in your fiduciary responsibilities to your customers and
shareholders if you are not filtering source IP addresses.
Fancy solutions may make great marketing, but simple proper router
filtering is a very workable lower-cost solution.
(Step down from soap box.) At least, that's my $0.02 worth.
Jon Kibler
Jon R. Kibler wrote:
Why no filtering by ISPs? "Because it takes resources and only benefits
the other guy" -- unless your network is the one under attack.
There you have the "operational" issue in a nutshell.
No dime, no do.
The sad fact is that simple ingress and egress filtering would
> eliminate the majority of bogus traffic on the Internet --
> including (D)DoS attacks. If all ISPs would simply drop all
> outbound packets whose source address is not a valid IP for the
> subnet of origin, and all inbound packets that do not have valid
> source IP addresses, the DDoS problem would be (for all intents
> and purposes) fixed.
The majority of the DDoS traffic that's been received here over the
past year has had 100% valid and accurate source IP addresses.
The sad fact is that simple ingress and egress filtering would
eliminate the majority of bogus traffic on the Internet -- including
(D)DoS attacks.
Couldn't agree more. It would probably cut hacked zombies (and that way
spam) by at least as much as DDoS traffic, in general we'd all have far
less problems if ISP's would stick to simple solutions where they're
needed. Although there are DoS's coming from valid IP's, 99 out of a 100
of these valid IP's are zombies hacked by using spoofed IP's so the
hacker isn't traceable. Good filtering will make this a lot harder to
pull off.
Why no filtering by ISPs? "Because it takes resources and only benefits
the other guy" -- unless your network is the one under attack.
And this is exactly the kind of ignorant thinking that prevents us from
solving the spam and DoS problems, while the exact same people can't
stop complaining about the spammers and script-kiddies ruining their
lunch.
Maintenance of the ACLs should not be the issue. A single ACL for each
subnet would be all that would be required for egress filtering. About
30 ACLs on an inbound border router would be required for ingress
filtering. Keeping the ingress ACLs current is a brain-dead task -- just
subscribe to the bogon mailing list at cymru.com.
If maintenance of ACLs was a problem for large ISPs, they'd be out of
business since that would imply they don't have the staff to keep their
networks running, let alone well enough to actually have customers on
it. I've probably heard the argument about the money it would cost and
the staff it would take a million times, but the fact is that if every
ISP did it's filtering, you'll see the need for troubleshooting,
spamfiltering, recovering from hackers, and mitigating DoS attacks drop
enormously. I'm 100% sure this would lead to lower maintenance costs,
not the other way around.
ACLs have had a bad reputation for greatly slowing down routers. That
may have been true in the past, but properly written ACLs do not seem
to have a significant impact on most new routers. Yes, they may cut
peak through-put a few percent -- but if you are running that close to
the edge, it is time to upgrade anyway.
Only very small ISPs relying on 36xx's or multilayer switching instead
of larger, more powerful might be still valid cases where ACL's are a
problem. But those aren't the ISPs generating 80% of all useless
traffic, it's the big boys that have plenty of hardware to burn that
refuse to do anything about it.
IMHO, there is absolutely no excuse for not doing ingress and egress
filtering.
Hear hear
While I mostly agree with your sentiment, one minor detail..
Based on recent observations of many folks, "spoofing is out of vogue".
So much so that some recent discussions I've had with several folks
lead me to believe that less than 1% of DDOS attacks today employ
source address spoofing. As such, the value of techniques such as
backscatter analysis and traceback decrease as well.
I suspect that [at least] the perception of wide-scale BCP 38/uRPF and
the sheer size and firepower of botnets today has resulted in a very
significant decline in source-spoofed attacks. Clever folks actually spoof
within the local (sometimes classful) subnet, making it slightly more difficult
to identify the concerned host (IF your traceback functions ever make
it to the "true Internet ingress" segment where a host resides, which
is more often than not unlikely).
I suspect this is largely because we do such a poor job fixing
compromised hosts that miscreants needn't worry much about losing
significant portions of their botnets to traceback and cleanup - as
Rob suggests, they're more concerned with losing them to other
miscreants.
This is also representative of the inversion in attack methods over the
past several years (i.e., the inversion from TCP-SYN type stuff to raw
UDP-fill-the-pipe style attacks).
Nonetheless, ingress filtering certainly helps significantly.
-danny
John Obi wrote:
> ... since DDoS is the
> nightmare of the internet now.
>The sad fact is that simple ingress and egress filtering would
eliminate the majority of bogus traffic on the Internet -- including
(D)DoS attacks. If all ISPs would simply drop all outbound packets
whose source address is not a valid IP for the subnet of origin,
and all inbound packets that do not have valid source IP addresses,
the DDoS problem would be (for all intents and purposes) fixed. If
proper filtering was done, then any DoS attacks would have to have
either valid source IP addresses, or IP addresses that spoofed IPs
within their network of origin. In either case, identifying and
shutting down the attackers would become a greatly simplified task
compared to the mess it is today.
Sorry to say this, but IMHO this is a naive view. It would only
marginally lessen the severity of attacks. The bulk of machines being
used for DOS attacks are compromised hosts and largely
intercontinental (from observations made from attacks against my
clients.) There are already machines sequentially opening HTTP
sockets, retrieving a particular URL, and repeating that process
thousands of times. These sorts of attacks can't be spoofed. And yet
when I attempt to contact the administrators of those machines (even
when I find them in the US under the auspices of major service
providers with "good" abuse departments), I get zero response to the
problem. So then if if the people writing this DOS software don't care
about hiding the addresses for this type of attack, why hide the
addresses from others? The same sort of damage will be done wether the
addresses are spoofed or not.
Filtering traffic isn't the principle issue (though it will help.) The
real problem is administrators who either don't care or flat refuse to
do anything about it. (Yes, the word "NO" has been said many times
when I've asked someone to investigate a possibly compromised host
even when supported by many hundreds of kilobytes of filter logs.)
And then of course, even if they DO respond, the end user is the one
who ultimately has to solve the problem and good luck getting THAT to
happen. (Yes, I know I'm a bit cynical about this but thats the result
of long and hard experience fending off such events.)
Why no filtering by ISPs? "Because it takes resources and only benefits
the other guy" -- unless your network is the one under attack.
Every one of my connections has rpf enabled unless there is a very
valid reson not to. (and thats done case by case.) Recent improvements
(I say recent, meaning over the last 5 years or so) have made such
efforts markedly more effective. The problem, as you state, is getting
the world at large to utilize these mechanisms.
Maintenance of the ACLs should not be the issue. A single ACL for each
subnet would be all that would be required for egress filtering. About
30 ACLs on an inbound border router would be required for ingress
filtering. Keeping the ingress ACLs current is a brain-dead task -- just
subscribe to the bogon mailing list at cymru.com.
For smaller networks, yes. For larger networks, they can have 2 or 3
hundred connections to a single border router with alloted IP space
varying daily. Meaning there would have to be frequent updates to an
upstream ACL (which may well be across an OC48) and lead to many human
caused outages. Simply not practicle for all networks.
-Wayne
Nonsense... While many more attacks are non-forged (see: hacked windows
machines in giant DDoS bot-nets that doesn't care about hiding the origin
because there are too many hosts to do anything about anyways) than they
were in the past, forged source attacks still make up huge portions of the
packets being thrown around.
What people may being seeing is that poorly randomized source attacks are
being automatically filtered by uRPF loose or other means before they ever
reach the target. I keep track of my network border filter counters, and
believe me spoofed attacks are not going out of style, especially from
foreign and certain smaller networks. As a customer of someone who does
this kind of filtering and maintains sufficient border capacity, you may
never see the gigabits of src bogons, protocol 0 or 255, port 0, 40 byte
syns w/no MSS option, etc, and assume that these attacks are out of style
because the only ones that get through are the WinXP MSS+SACK unforged
drone SYNs.
Interesting assertion. Care to support it?
--Jeff
What people may being seeing is that poorly randomized source attacks are
being automatically filtered by uRPF loose or other means before they ever
reach the target. I keep track of my network border filter counters, and
believe me spoofed attacks are not going out of style,
How do you discriminate *DDOS attacks employing source address spoofing*
from broken NATs, rampant worms, PMTU and other related misconfiguration
resulting in backscatter and similar garbage - with filter counters? Given,
tactically deployed filters in order to mitigate a specific attack to a particular
destination would likely glean some value WRT the validity of the source
distribution for a given attack, but not generally deployed filters for any
destination.
And exactly what represents "spoofed" by your definition? Note again that
I explicitly called out **DDOS attacks employing source address spoofing**,
which is non-inclusive of spoofing in general employed by worms and the
like, or common misconfigurations and brokenness that results in the slew
of random garbage floating about.
especially from foreign and certain smaller networks.
I'd be extremely interested in any empirical evidence you have to support
this, and in better understanding exactly how you determined "foreign and
certain smaller networks" were indeed the source of many of these spoofed
packets.
As a customer of someone who does this kind of filtering and maintains
sufficient border capacity, you may never see the gigabits of src bogons,
protocol 0 or 255, port 0, 40 byte syns w/no MSS option, etc, and assume
that these attacks are out of style because the only ones that get through
are the WinXP MSS+SACK unforged drone SYNs.
I agree, if it's filtered before someone observes it, it won't be
observed 
However, distinguishing between coordinated DDOS attacks that employ source
address spoofing and "run of the mill" spoofing (by worms and the like) or
simple misconfiguration of some sort resulting in "backscatter" is key.
-danny
Here is a great move from one of the biggest NSPs, I'm
sure we will see L3, Sprint, UUNet and others will do
the same soon to gain more customers since DDoS is the
nightmare of the internet now.Yahoo Finance - Stock Market Live, Quotes, Business & Finance News
Thanks,
Forgive me for being cynical, but couldn't this just AT&T putting an IDS in their POP (or virtually, in their POP) in front of the Customer connections? We are all talking about huge, all encompassing solutions to DDOS that we'd like to see... But what are the chances of that?
And is AT&T going to ignore customers that aren't subscribers of AT&T Protect -- even though their network "is monitoring & filtering malicious" traffic at its edges?
Deepak Jain
AiNET
We see approximately 60-70 spoofed DDOS attacks per day at our Network
Telescope:
http://noc.ilan.net.il/research/riverhead/
The volume of backscatter we see on our "dark space" /16 is about
50kb/sec:
http://noc.ilan.net.il/stats/TAU-GIGAPOP/riverbsc-gp1.ilan.net.il.html
I have no way of proving it but I assume we see only 10% of the daily DDOS
attacks that take place on the Internet since as you state - most of the
attacks these days are from "100% valid and accurate source IP addresses".
-Hank
>What people may being seeing is that poorly randomized source attacks
>are
>being automatically filtered by uRPF loose or other means before they
>ever
>reach the target. I keep track of my network border filter counters,
>and
>believe me spoofed attacks are not going out of style,How do you discriminate *DDOS attacks employing source address spoofing*
from broken NATs, rampant worms, PMTU and other related misconfiguration
resulting in backscatter and similar garbage - with filter counters?
Given,
tactically deployed filters in order to mitigate a specific attack to a
particular
destination would likely glean some value WRT the validity of the source
distribution for a given attack, but not generally deployed filters for
any
destination.
If it walks like a duck, and it sounds like a duck, it is probably a duck.
RFC1918 sourced space, most likely from misconfigured NATs and such,
account for only a very small amount of the bogon-source packets which go
splat.
Most of the DoS attempts by volume don't fall into the category of
questionable. When you see a 100Mbps stream (from a single ingress
interface, with consistant TTL's) of IP proto 0 or 255, or tcp port 0, or
classic SYN flooders (SYN w/no MSS) or stream (randomized seq# and fixed
ack# on a packet w/TH_ACK flag only) targetting a specific IP/port with a
source address of iph.ip_src.s_addr = random(), it is pretty easy to tell
those apart from the usual background noise of a worm.
> especially from foreign and certain smaller networks.
I'd be extremely interested in any empirical evidence you have to
support
this, and in better understanding exactly how you determined "foreign
and
certain smaller networks" were indeed the source of many of these
spoofed
packets.
Some days it helps to actually have an operational network, instead of
being a researcher. Even without interesting tools it isn't terribly hard
to look at your PNI graphs, match up the hundreds-of-meg spikes with
specific DoS incidents, and go from there. Not to point fingers at anyone
in particular, but it seems to be the same foreign networks who tend to
have little control over their spammers.
If it walks like a duck, and it sounds like a duck, it is probably a duck.
RFC1918 sourced space, most likely from misconfigured NATs and such,
account for only a very small amount of the bogon-source packets which go
splat.
But worms, OTOH, seems to be much more persistent.
Most of the DoS attempts by volume don't fall into the category of
questionable. When you see a 100Mbps stream (from a single ingress
interface, with consistant TTL's) of IP proto 0 or 255, or tcp port 0, or
classic SYN flooders (SYN w/no MSS) or stream (randomized seq# and fixed
ack# on a packet w/TH_ACK flag only) targetting a specific IP/port with a
source address of iph.ip_src.s_addr = random(), it is pretty easy to tell
those apart from the usual background noise of a worm.
Sure..
Some days it helps to actually have an operational network, instead of
being a researcher. Even without interesting tools it isn't terribly hard
to look at your PNI graphs, match up the hundreds-of-meg spikes with
specific DoS incidents, and go from there. Not to point fingers at anyone
in particular, but it seems to be the same foreign networks who tend to
have little control over their spammers.
Heh.. I certainly don't consider myself a researcher, or an
operator (any longer) for that matter (though I do have access
to a significant amount of both research and operational data
and tend not to call a duck a goose simply because I heard
a quack
-danny
A bogon packet is a bogon packet Filter them all and let the appropriate deity
sort them out (unless you bill by traffic volume 
"[att] which handles 1.3 petabytes of data per day"
woooo
"For example, in the case of the most recent Sasser worm, AT&T Internet Protect
identified precursors to the worm several weeks before it was fully launched.
AT&T immediately notified AT&T Internet Protect customers of the malicious
activity and provided recommendations for remediation through a secure Web
portal to help them proactively protect their networks."
oh? how did they do that then.. any at&t protect customers want to fwd the
notification... ?
Steve