Contrary to popular belief, I (not alone, of course) run,
manage, defend, and continually architect very large
networks. Very large. On none of them do we outsource
the protection of them -- because, in cases where we
have extended trust in the past, we have been screwed
(PC translation: disappointed).
So we protect ourselves.
It's been a business decision for my customers' networks
(ie. their network) not to outsource security, or rely on
an upstreampipedream, for protection of any sort.
Thus, I personally can't provide any insight here. Sorry.
- ferg
In this case it's a business decision. I understand that we could
simply weigh the costs of an attack with the costs of preemptively
detecting and mitigating an attack, but in our case we won't lose hard
dollars like an ecommerce site would. We have different reasons for
wanting to have some protection in place before we need it. I look at
it like it's an insurance policy, but I don't want to be ripped off.
It's like I'm getting estimates on building a protective dike around
my house. One contractor tells me that the floodwaters commonly reach
six feet so I should pay him $12,000 to build a wall at least that
high. Another contractor is telling me that he'll build a six-foot
wall for $6,000. Another contractor is telling me that the floodwaters
most likely won't go over two feet and he suggests that I pay him
$1,000 for a three-foot-high wall.
If it turns out that we really do need a six-foot-high wall then so be
it. I'm not the one who pays the bills so it isn't really my decision.
I just want to make sure I have a clearer picture of reality before I
make any suggestions to my boss.
Ferg,
Not everyone is in a position to have anetwork large enough to be "self-defending". I think he has clearly stated they are not in a position from a capacity standpoint to self-defend. If he has a few sites with some T1's or DS3's or whatever, his goal is to not stop the traffic at his router, but not ever allow the traffic onto his pipe.
I too have been involved in large, very large, networks and we used to see it happen everyday. Customers with OC12's getting smoked off the planet because of some kiddie made someone else mad in IRC. If the upstream offers a "value add" service such as DoS protection, why balk at it?
That's an understandable attitude given the nature of your networks.
In our case, I'm just talking about two or three T1s that provide
Internet connectivity to our website for our customers.
I appreciate your input, though. I will accept all advice and input if
it gets me closer to a better understanding of the realities of topic
at hand and if it helps weed out some of the marketing fluff that's
being heaped upon me by salespeople.
Thinking about this a bit, I subscribe to the theory that it somewhat
depends on how big of a target you are. People who have large
networks usually offer more services to more people, thus they have
more exposure. "Most of the time" when I see the large DoS attacks
they are to customers with nice fat pipes. I've rarely seen large
(2-3Gb) attacks to small customers with a few T1's. Not to say it
doesn't ever happen. But if you're not hosting a ton of sites there's
not that much reason for someone to DoS you. Botnets are a commodity
and when they are used, inevitably, a portion of the bots are found
and fixed. There's usually 2 reasons why people are DoS'd. You, or a
customer, has pissed someone else off or it's being done to you for
extortion reasons. If you're hosting a small site for your business
as mostly an informational purpose then you're likely not going to be
really pissing someone off. If you have a large ecommerce site then
you are a target for extortion purposes. Following this chain of
logic, if you have a large site, you have large pipes. I guess what
I'm getting at is this may be one area where security through
obscurity may actually pan out, speaking **only** to security of
bandwidth/packet rate DoS attacks.
As always, there are exceptions and anything can happen. I'm just
speaking from my personal experience. I've worked in a largeish ISP
NOC for about 3 years and this is mostly what I've seen.
Some things you can do that are free include making sure that if/when
you get attacked you have a plan in place of how to deal with it.
This includes having up to date contact information for your service
provider. Knowing what their capabilities are and how they deal with
attacks. Having circuit information, hardware information, and
hardware vendor contact information available to help all involved
parties aid in mitigating the attack. This can save huge amounts of
time when "bad things happen" and this applies no matter how large or
small you are.
Ok, so why not jump in with 1 foot atleast A note first though:
1) UUNET/MCI does sell this product (I don't sell it personally, I don't
sell anything actually)
2) UUNET/MCI's sales method for this product is 'confusing' (to me
atleast, but recall I'm a chemical engineer...)
3) UUNET/MCI has been providing this service for free for 6+ years, now
with special gear and a price for 'enhanced services'
now, down to business. The core of your question is two parts:
a) how much should you spend
b) how much protection do you need
For the 'a' part a few folks have said: "Pay what you are willing to part
with". That means you have to decide how much protection you want and how
much you'll need (see 'b'). For 'b' I can say, after 5+ years defending
UUNET's customers globally (well, the team I work on does this globally
it's not just me) and giving a talk here or there about this subject:
"Attackers will do just enough to be effective"
Keep in mind there is no way for them to know you have a 9600 baud modem
or a oc-48. I've seen 400mbps attacks against modem users, and a modem's
worth of 'attack' aimed at a oc-12 customer Normally the attackers aim
a weapon at the victim, shoot and add more weapons if required. They will
add more until they get their effect. This COULD mean that if you
purchased 60 gbps of attack mitigation capacity you'd get screwed in the
end... There is a trade off: "how much is realistic to expect", this has
nothing to do with your end-site connectivity. I'd aim at an average (high
average) attack size. I'd aim at 500mbps/1gbps, I'd also ask a few other
questions:
1) how does this mitigation get started? (phone call, ticket, call back?
or customer initiated bgp update? or prayers to the ddos-mitigation-god?)
2) how much capacity is available regardless of what is purchased?
3) how quickly can extra capacity be added if required? (days? hours?
seconds? at all?)
4) how much latency will be incurred if I have a /32 under mitigation?
what about a /24? a /16? does it matter?
5) how much granularity in the policy of said device(s) do I have?
6) how does reporting work for this service? (how do I know anything is
happening?)
7) are there dedicated individuals prepared to answer my questions at
0dark:30 on a Saturday Christmas night?
As I said, I do this for a living, I have a little bit of a bias but
I'm sure if you listen to Mr. Feger he's a smart guy as well, who knows
this problem as well as I do...
Good luck! If you want other info about this service (the mci version of
it) and don't want to jaw with a sales droid you can get me off-list.
Same goes for other folks, I'd just note I'm away from email a bit over
the next few days so I may be a little slow to respond
Some things you can do that are free include making sure that if/when
you get attacked you have a plan in place of how to deal with it.
This includes having up to date contact information for your service
provider. Knowing what their capabilities are and how they deal with
attacks. Having circuit information, hardware information, and
hardware vendor contact information available to help all involved
parties aid in mitigating the attack. This can save huge amounts of
time when "bad things happen" and this applies no matter how large or
small you are.
This sort of thing is very often overlooked. As with any emergency plan:
1) have a plan
2) test the plan
3) validate the plan
Also, chip didn't mention this, but... perhaps what is being attacked
doesn't HAVE to work. Your provider might also have the possibility to let
you blackhole things inside their network, so if something less important
is attacked, just make it go away 'free' don't pay for mitigation if it's
not required...
I'm very interested in technical solutions of ISP
based (D)DOS solutions. Where can I find
document/information on it?
design one yourself or buy mitigation capacity from someone who does it
already... or I think I may give a talk at the next nanog on this topic,
since it seems like the call for papers included a request to chat about
this topic
Has links to some interesting presentations, including - starting from
the basics
Barry Greene's templates for Egress and Ingress filtering on Cisco (oh
and RFC3704),
Several presentations on filtering bogons,
Chris Morrow on blackhole route servers [VERY useful, that]
Plus presentations on ddos mitigation boxes (Riverhead, Arbor etc) .
Oh yes - dig through the nanog / apricot etc archives for past
presentations on darknets and the team cymru darknet project
Normally the attackers aim