UDP port 80 DDoS attack

Mailman List Mirror (Read Only)
21

Since when are policers implemented in ram? You're talking FPGA if you
want to be able to make forwarding/filtering decisions assuming it's
possible which it isn't you're 1 million dollar boxes suddenly become
hundred million dollar boxes. Then there's v6 info..

Of course it's not possible ... if you use a crummy design. It's trivial
to come up with non-completely-crummy designs. For example, adding a
front-end where you take a hash of source-ip/dest-ip and run it through
a smallish hash table, you can use that as a filter to eliminate a lot
of traffic that's just normal and non-interesting. You want to take a
closer look at the traffic that's heaviest (read: most hits) or new and
significant (read: diff against an hour ago). You probably don't want
to do this just per-IP, but likely also per-network. And you probably
don't want to use just this one technique, you want to combine it with
others. And you probably need to consider the types of attacks that
are known, likely, etc., and design accordingly, because this one little
example I've provided is just one part of a comprehensive solution, but
it is capable of dealing with any amount of traffic and it would be a
very useful filter to start pulling out potentially interesting stuff.

This stuff isn't *easy*. Fine. But it certainly *is* possible.

... JG

22

Of course it's not possible ... if you use a crummy design. It's
trivial to come up with non-completely-crummy designs. For example,
adding a front-end where you take a hash of source-ip/dest-ip and run
it through a smallish hash table, you can use that as a filter to
eliminate a lot of traffic that's just normal and non-interesting. You
want to take a closer look at the traffic that's heaviest (read: most
hits) or new and significant (read: diff against an hour ago). You
probably don't want to do this just per-IP, but likely also per-
network.

I think one of the problems is that with modern bot-nets, traffic can be generated by a huge number of hosts and assuming your DoS traffic is coming from a source that can be blocked might be an unreasonable expectation. You can't assume that you are going to get a flood of traffic from some source that you can pin down and block. You might get flood of traffic from tens of thousands of source IPs from all over the world with each one sending only a very small amount AND the source IPs constantly changing. They might even be sending traffic that looks perfectly legitimate on the surface and might need to be profiled/fingerprinted in some manner at layer 4. It isn't as easy as just handling it at the router. And there is no guarantee the source IP of the traffic is really where it is coming from since there are still a good number of providers out there who don't install packet filters on their customer links. They accept any traffic their customer sends them even if the source IP isn't within the customer's network range. So that is part of the game, too. If you have 10,000 hosts sending packets with spoofed IP addresses where the goal is to get you to block the apparent source network, as soon as you block those source addresses, the DoS has succeeded.

And you probably don't want to use just this one technique,
you want to combine it with others.

I think "probably" is the wrong word here. The word "certainly" leaps to mind.

And you probably need to consider
the types of attacks that are known, likely, etc., and design
accordingly, because this one little example I've provided is just one
part of a comprehensive solution, but it is capable of dealing with any
amount of traffic and it would be a very useful filter to start pulling
out potentially interesting stuff.

The problem is that you have a game of cat and mouse with what amounts to an infinite supply of mice. It takes cooperation between the source and the provider networks. The "eyeball" heavy networks need to ensure they can't source bogus traffic. Having gear these days where the ACLs are in hardware has greatly reduced the CPU expense of filtering on edge ports but the human resource expense of maintaining those is still high unless automation is brought into the mix so that those filters are changed when the addresses served by a port change.

This stuff isn't *easy*. Fine. But it certainly *is* possible.

Of course it isn't easy. It is designed to be difficult. But there is plenty of "low hanging fruit" out there still.

23

Roland,

S/RTBH can be rapidly shifted in order to deal with changing purported source IPs, and it isn't limited to /32s.

The big drawback with S/RTBH is that it is a DoS method in itself.

Say eyeball provider X has implemented automated S/RTBH, and I have a
grudge against them.
I would simply DoS a couple of the subscribers with spoofed source IP
addresses from google, youtube, netflow and hulu.
The automated S/RTBH drops all packets coming from those IP addresses.
Presto; many angry consumers call the ISP's helpdesk.

The same goes for hosting networks, I just need to identify what kind
of service my intended victim is dependent on. (i.e. paypal).
Then DoS any part of the hosters network with spoofed source addresses
of paypal, the automated S/RTBH makes sure the entire hosting network
is not able to reach paypal anymore.
Presto, mission achieved.

Bas

24

Comes back to providers allowing "spoofed" traffic into their networks from customers. That seems to me to be the low-hanging fruit here.

25

I'm not an advocate of *automated* S/RTBH, and I am an advocate of whitelisting various well-known 'golden networks/IPs' via prefix-lists in order to avoid this issue in part; also, note that the concept of partial service recovery incorporates the notion of some legitimate traffic/users being blocked in order to maintain the availability of the targeted server/service/application for the majority of legitimate traffic/users.

Also note that S/RTBH isn't a universal panacea; it's just one tool in the toolbox. flowspec is more flexible due to its layer-4 granularity; and other types of tools such as IDMS are even more flexible and incorporate much richer classification technology.

It's important to understand that this isn't a theoretical discussion - I've personally utilized/helped others to utilize S/RTBH to successfully mitigate large-scale DDoS attacks, and it's a lowest-common-denominator in terms of a somewhat dynamic mitigation mechanism. Let's not make the perfect the enemy of the merely good.

;>

26

How do you stop it? Granted, traffic from 10/8 or 127.0.0.1 coming in via
an upstream is obvious, but that's about it. There's nothing in a packet
that will tell you where it came from compared to the source IP field in
the IP header. uRPF is a problem for anyone who's sufficiently multihomed
since it causes asymmetric routing.

27

From: Keegan Holley

How do you stop it?

A provider knows what destination IP traffic they route TO a customer, don't they? That should be the only source IPs they accept FROM a customer.

If you don't route it TO the customer, you shouldn't accept it FROM the customer unless you have made special arrangements with them and verified they are entitled to source the traffic from the desired IPs.

28

It works in theory, but to get every ISP and hosting provider to ACL their
edges and maintain those ACL's for every customer no matter how large might
be a bit difficult. Also, what about non-BGP customers or customers that
just accept a default route? Or even customers that just want return
traffic to come in a different link for some reason. ISP's would suddenly
become giant traffic registries.

29

From: Keegan Holley
Subject: Re: UDP port 80 DDoS attack

It works in theory, but to get every ISP and hosting provider to ACL their edges and maintain those ACL's for every customer no matter how large might be a bit difficult.

You don't have to ACL in most cases. RPF works for most. There will be a few, relatively darned few, that you will need to ACL, but RPF takes care of a large number of them.

Besides, I never meant to imply that this business was easy and not "difficult".

Also, what about non-BGP customers or customers that just accept a default route?

I don't follow. The ISP still knows what traffic gets routed TO them. You only accept FROM them what you route TO them, even if you hand them a default route.

Or even customers that just want return traffic to come in a different link for some reason.

Still don't follow. I am not talking about having a firewall that is stateful. I am talking packet by packet. If you don't route it to them, you don't accept it from them unless you have made arrangements otherwise, which will be a small percentage of your customers. Sure, some might be multihomed but it is easy enough to verify that they have the networks in question SWIPed to them or a call to the other provider can clear that up in a few minutes. It isn't THAT hard.

ISP's would suddenly become giant traffic registries.

No, we have registries to act as registries, the ISPs should be checking them, and double checking. It isn't something that is going to change every day or every week. Once you get it set up, it is going to be stable for a while. Sure, it means a little more work in setting up a customer, but it also means that if all your neighbors do the same thing, you field many fewer calls dealing with stupid DoS crap.

30

No, we have registries to act as registries, the ISPs should be
checking them, and double checking. It isn't something that is going
to change every day or every week. Once you get it set up, it is going
to be stable for a while. Sure, it means a little more work in setting
up a customer, but it also means that if all your neighbors do the same
thing, you field many fewer calls dealing with stupid DoS crap.

I'll put it another way. Any provider that does not police their customer traffic has no business whining about DoS problems.

31

Most of us prevent their customers from sending out spoofed traffic.

77% of all networks seem to think so.
http://spoofer.csail.mit.edu/summary.php

However the remaining networks allow spoofed traffic to egress their networks.

When that traffic enters my network, I have no method whatsoever to
differentiate it from any other traffic.
I could ask my upstream where they see it coming from, which will be
quite hard if they do not have pretty fancy systems.
But if they receive it from a peer, I am as good as lost in trying to
find the culprit.

Bas

32

So I would need to find out which networks you would have classified
as "golden" and use those as sources for my DDoS.

Either I can achieve DoS with S/RTBH, or I can abuse the "golden
networks" to circumvent S/RTBH.

As far as I see it S/RTBH is in no way a solution against smart
attackers, of course it does help against all the kiddie attacks out
there.

Bas

33

Once again, I've used S/RTBH myself and helped others use it many, many times, including to defend against attacks with shifting purported source IPs. flowspec, IDMS and other tools are very useful as well, but S/RTBH is supported on a lot of hardware, if operators choose to configure it.

It is not a panacea. It is one tool in the toolbox.

Folks can either choose to make use of it or choose not to do so; it is operationally proven, it does work, and it's certainly better than nothing. YMMV.

34

Providers don't even check the registries for bgp advertisements. See the thread on hijacked routes for proof. Not to mention how do you handle a small transit AS? Do you trust that they have the correct filters as well? Do you start reading their AS paths and try to filter based on the registry for folks down stream? Then there's the RLDRAM issue. Most edge boxes will just run out if ACL's. Lastly there's no contractual obligation to play traffic cop for the entire Internet so providers would be dropping traffic that they can legitimately bill for.

35

From: Keegan Holley
Subject: Re: UDP port 80 DDoS attack

It works in theory, but to get every ISP and hosting provider to ACL their edges and maintain those ACL's for every customer no matter how large might be a bit difficult.

You don't have to ACL in most cases. RPF works for most. There will be a few, relatively darned few, that you will need to ACL, but RPF takes care of a large number of them.

RPF on the whole Internet would pretty much lead to an instant outage. What happens when you have two upstreams and one has an incoming route to you but your out going route for which ever of their customers talking to you doesn't agree? Instant outage. Multiply that by the entire table and then add churn. I'd give it a week before everyone turned it off, if you could even get them to enable it to begin with.

36

to be fair: "Some Providers do not check registries for 'right to use'
information about prefixes their customers wish to announce to them
over BGP."

37

I agree. I think RTBH is a broadsword not a scalpel. It's a tool in the
tool box and there is a danger of dropping legitimate traffic with both
S/RTBH and D/RTBH. BGP isn't a security protocol. It's not even that
great of a routing protocol.

38

77% of all networks seem to think so.
http://spoofer.csail.mit.edu/summary.php

And it would be the remaining 23% that really need to understand how difficult they are making life for the rest of the Internet.

However the remaining networks allow spoofed traffic to egress their
networks.

When that traffic enters my network, I have no method whatsoever to
differentiate it from any other traffic.

I'm not really thinking about traffic coming from the Internet. I'm thinking about its originating location. Correct, once it gets into the Internet, you really have no way to tell.

I could ask my upstream where they see it coming from, which will be
quite hard if they do not have pretty fancy systems.

At that point the game is really hard, agreed. And if it is distributed, it could be coming from any number of places or from every single one of their upstreams.

But if they receive it from a peer, I am as good as lost in trying to
find the culprit.

Agreed. That's why it is important to stop it at the source.

39

23% of 4.29 billion addresses is still more than enough to ruin anyone's
day.

40

Stop paying transit providers for delivering spoofed packets to the edge of your network and they will very quickly develop methods of proving that the traffic isn't spoofed, or block it altogether. =)

-Drew