Global Blackhole Service

Mailman List Mirror (Read Only)
1

Hi,

in the last 24 hours we received two denial of service attacks with something
like 6-8GBit volume. It did not harm us too much, but e.g. one of our
upstreams got his Amsix-Port exploded.

With our upstreams we have remote-blackhole sessions running where we announce
/32 prefixes to blackhole at their edge, but this does not work with our
peers. Also our Decix-Port received something like 2Gbit extra-traffic during
this DoS.

I can imagine, that for some peers, especially for the once having only a thin
fiber (e.g. 1GBit) to Decix, it's not to funny having it flooded with a DoS
and that they might be interested in dropping such traffic at their edge.

Well I could discuss with my peers (at least the once who might get in trouble
with such issue) to do some individual config for some blackhole-announcement,
but most probably I'm not the only one receiving DoS and who would be
interested in such setup.

Therefore I had the following idea: Why not taking one of my old routers and
set it up as blackhole-service. Then everyone who is interested could set up a
session to there and

1.) announce /32 (/128) routes out of his prefixes to blackhole them
2.) receive all the /32 (/128) announcements from the other peers with the IPs
they want to have blackholed and rollout the blackhole to their network.

My questions to all of you:

- - What do you think about such service?
- - Would you/your ASN participate in such a service?
- - Do you see some kind of usefull feature in such a service?
- - Do you have any comments?

Thank you for telling me your opinions and best regards

- --

2

Ah. rbl.maps.vix.com from about a decade back when it was available as
a bgp feed. But only for ddos sources.

srs

3

would this itself not be a dos path?

randy

4

Hi Jens,

I think we are in the same boat.

We suffered the same problem often, on a lower magnitude, but if a project like this exists those DDoS could even be almost near zero.

This is somewhat similar to what Spamcop, and other folks do with SPAM today, but applied on a diferent scope, say, BGP Blackhole.

This service can span wide after just peers, opening the opportunity to edge-to-edge DDoS mitigation.

Say, a network in .pt or .de is beign attacked at large, and dst operators inject the dst attacked source on the blackhole bgp feed... say that 100+ other ops around the world use a cenário like this... this might be very useful.
concers: the "autohority" or the "responsible" for maintaining this project, must assure that OP A or OP B can *only* annouce chunks that below to him, avoiding any case of hijack.

We would be interested in participating in something like this.

So,

My questions to all of you:

- - What do you think about such service?

It will be great. We are available to help.

- - Would you/your ASN participate in such a service?

Yes.

- - Do you see some kind of usefull feature in such a service?

Yes, a few thoughts above, some more might come up.

- - Do you have any comments?

For starters, a few above.

Regards,

5

In that way, Spamcop and other folks are DOS'ing for years aswell. And in fact, by denying things around, they are just scrubing and filtering, to make our day happier, avoiding huge masses of spam and useless crap.

I don't see it the way you do.

A project like this, like also spamcop, are great paths to take out the scum and undesired things from the net.

regards,

6

How do you vet proposed new entries to make sure that some miscreant doesn't
DoS a legitimate site by claiming it is in need of black-holing? Note that
it's a different problem space than a bogon BGP feed or a spam-source BGP
feed - if the Cymru guys take another 6 hours to do a proper paperwork and
background check to verify a bogon, or if Paul and company take another day
to verify something really *is* a cesspit of spam sources, it doesn't break the
basic concept or usability of the feed.

You usually don't *have* a similar luxury if you're trying to deal with a
DDoS, because those are essentially a real-time issue.

Oh, and cleaning up an entry in a timely fashion is also important, otherwise
an attacker can launch a DDoS, get the target into the feed, and walk away...

7

How do you vet proposed new entries to make sure that some miscreant doesn't
DoS a legitimate site by claiming it is in need of black-holing? Note that
it's a different problem space than a bogon BGP feed or a spam-source BGP
feed - if the Cymru guys take another 6 hours to do a proper paperwork and
background check to verify a bogon, or if Paul and company take another day
to verify something really *is* a cesspit of spam sources, it doesn't break the
basic concept or usability of the feed.

Presumably, the route server would have to have the same guidelines as issued by service providers. ie, /32 networks injected should come from authenticated feeds and fall within the netblock range owned by the injector. So one extra set of ACL's for each injector to upkeep. I believe what is being suggested is just one step beyond what many providers give to BGP customers to extend blackholes out.

Oh, and cleaning up an entry in a timely fashion is also important, otherwise
an attacker can launch a DDoS, get the target into the feed, and walk away...

This also would be decided by the injecting provider. More of a "Hey, one of my IPs is being DDOS'd, please drop traffic to it to protect the rest of my network." The downside to widespread use, is that it makes tracking the problem on the other side of the blocks near impossible. In all cases, once a blackhole is initiated anywhere, the DDOS has been successful. We use automatic community changes to accept /32 blackholes from customers, verify them, then send them on to peers that also support /32 blackholes with appropriate communities.

Jack

Jack

8

Ok, however, what i am talking about is a competelly diferent thing, and i think that my thoughts are alligned with Jens.

We want to have a Sink-BGP-BL, based on Destination.

Imagine, i as an ISP, host a particular server that is getting nn Gbps of DDoS attack. I null route it, and start advertising a /32 to my upstream providers with a community attached, for them to null route it at their network.
However, the attacks continue going, on and on, often flooding internet exchange connections and so.

A solution like this, widelly used, would prevent packets to leave their home network, mitigating with effective any kind of DDoS (or packet flooding).

Obviously, we need a few people to build this (A Website, an organization), where when a new ISP connects is added to the system, a prefix list should be implemented, preventing that ISP to announce IP addresses that DON'T belong to him.

The Sink-BGP-BL sends a full feed of what it gots to Member ISP's, and those member ISP's, should apply route-maps or whatever they want, but, in the end they want to discard the traffic to those prefixes (ex: Null0 or /dev/null).

This is a matter or getting enough people to kick this off, to build a website, to establish one or two route-servers and to give use to.

Once again, i am interested on this, if others are aswell, let know. This should be a community-driven project.

regards,

9

In other words, a legitimate prefix hijacking service...

As Randy and Valdis have pointed out, if this isn't done very carefully
it's an open invitation to a new, very effective DoS technique. You
can't do this without authoritative knowledge of exactly who owns any
prefix; you also have to be able to authenticate the request to
blackhole it. Those two points are *hard*. I also note that the
scheme as described here is incompatible with more or less any possible
secured BGP, since by definition it involves an AS that doesn't own a
prefix advertising a route to it.

    --Steve Bellovin, http://www.cs.columbia.edu/~smb

10

@jack: sorry for duplicate ... pressed reply instead of reply-all :wink:

Jack Bates schrieb:

Presumably, the route server would have to have the same guidelines as
issued by service providers. ie, /32 networks injected should come from
authenticated feeds and fall within the netblock range owned by the
injector. So one extra set of ACL's for each injector to upkeep. I
believe what is being suggested is just one step beyond what many
providers give to BGP customers to extend blackholes out.

Exactly that's the way I intended. I know that it's a not to small thing to
maintain such a system, we are running it successfully for years with both,
downstream-bgp-customers and upstreams. Even with quiet a small number of
downstreams there are several changes each month (new IP-Space, drop-off of PI
moved away from the customer ...), but I think it would be a manageable thing
to keep it up2date when preparing some automatism. E.g. a automated
prefix-list-generator requesting the authorization (e.g. automated mail with
link including authorization-hash) for blackholing at the AS-Owner before
accepting prefixes ...

Oh, and cleaning up an entry in a timely fashion is also important,
otherwise
an attacker can launch a DDoS, get the target into the feed, and walk
away...

This also would be decided by the injecting provider. More of a "Hey,
one of my IPs is being DDOS'd, please drop traffic to it to protect the
rest of my network." The downside to widespread use, is that it makes
tracking the problem on the other side of the blocks near impossible. In
all cases, once a blackhole is initiated anywhere, the DDOS has been
successful.

Well, for that single IP the DDoS was sucessfull, but looking at the issue I
had yesterday, it's to protect other customers also getting into trouble due
to this DoS. The complete rack had 1GBit-Uplink, which is normally absolutely
sufficient for 20 servers. Well one single server was under attack, but 19
other "innocent" customers were not reachable. And, the even bigger problem
was, the AMSIX-Port of one of my upstreams was "filled to death" due to this
DoS and therefore several thousand customers had enormous packetloss due to
one single destination-ip. Therefore it's to decide what to prefer, one single
customer dead or thousands of angry customers. And I know that I prefer to
protect my own backbone under these circumstances.

We use automatic community changes to accept /32 blackholes
from customers, verify them, then send them on to peers that also
support /32 blackholes with appropriate communities.

That's what we currently also do and until now we never had any problem with this.

BR
Jens

Jack

Jack

- --

11

Jens,

I would be interested in participating with a destination blackhole service, so long as peers were authenticated and only authorized to advertise /32s out of space that they are assigned -- hopefully the same OrgID is used for the ASN as the IP allocations.

However, a blackhole service based on sources would be out of the question altogether in my book, unless paired with a number of third parties that could vet the "badness" of those source IPs, as is done with spam zombies. Even then I'd be very nervous about it from a "causes more [potential] problems than it fixes" standpoint, no matter how cool it would be to defang a DDoS.

As for the memory requirements / "oh no! too many routes!" issue, that would be a non-issue for me.

Feel free to contact me off-list if you're serious about starting this project. I think that it would be worth it to talk to the Team Cymru guys to see if they'd be interested in this.

-Tico

Jens Ott - PlusServer AG wrote:

12

Steven M. Bellovin schrieb:

Ok, however, what i am talking about is a competelly diferent thing,
and i think that my thoughts are alligned with Jens.

We want to have a Sink-BGP-BL, based on Destination.

Imagine, i as an ISP, host a particular server that is getting nn
Gbps of DDoS attack. I null route it, and start advertising a /32 to
my upstream providers with a community attached, for them to null
route it at their network. However, the attacks continue going, on
and on, often flooding internet exchange connections and so.

A solution like this, widelly used, would prevent packets to leave
their home network, mitigating with effective any kind of DDoS (or
packet flooding).

Obviously, we need a few people to build this (A Website, an
organization), where when a new ISP connects is added to the system,
a prefix list should be implemented, preventing that ISP to announce
IP addresses that DON'T belong to him.

The Sink-BGP-BL sends a full feed of what it gots to Member ISP's,
and those member ISP's, should apply route-maps or whatever they
want, but, in the end they want to discard the traffic to those
prefixes (ex: Null0 or /dev/null).

This is a matter or getting enough people to kick this off, to build
a website, to establish one or two route-servers and to give use to.

Once again, i am interested on this, if others are aswell, let know.
This should be a community-driven project.

In other words, a legitimate prefix hijacking service...

As Randy and Valdis have pointed out, if this isn't done very carefully
it's an open invitation to a new, very effective DoS technique. You
can't do this without authoritative knowledge of exactly who owns any
prefix; you also have to be able to authenticate the request to
blackhole it. Those two points are *hard*.

As described in my earlier mail, I'd suggest to run a prefix-list generator
updating informations from IRR on a regulary basis and, as soon as a new
"matching" route-object appears in IRR, an automated mail might be send to the
ASN-owner (address also taken from irr-records) with a confirmation-link.

That way you'd need to hijack IRR-database and/or tech-c/admin-c mailbox
before being able to have a prefix added to the list of prefixes accepted from
your peer.

I also note that the
scheme as described here is incompatible with more or less any possible
secured BGP, since by definition it involves an AS that doesn't own a
prefix advertising a route to it.

No, the router may work as Route-Reflector, so you see exactly the as-path as
is and the route-reflectors own asn isn't visible at all..

    --Steve Bellovin, Steven M. Bellovin

- --

13

Steven M. Bellovin wrote:

In other words, a legitimate prefix hijacking service...

Absolutely, NOT. The origin AS will still be the AS that controls the IP space. In fact, I think SBGP would be great for a layout like this to secure down the injections. That being said, prefix lists with md5 auth are probably the best we can hope for. Routing registry macro support or a hashed authorization link sent to whois contacts to automate modification of the prefix lists would be ideal (not much different that a provider is *supposed* to do with their BGP customers). Once the peers is established and limited in scope, they can then start advertising /32 networks into the blockhole server who will pass it on to others.

As Randy and Valdis have pointed out, if this isn't done very carefully
it's an open invitation to a new, very effective DoS technique. You
can't do this without authoritative knowledge of exactly who owns any
prefix; you also have to be able to authenticate the request to
blackhole it. Those two points are *hard*. I also note that the
scheme as described here is incompatible with more or less any possible
secured BGP, since by definition it involves an AS that doesn't own a
prefix advertising a route to it.

I would presume that md5 BGP peering with prefix lists developed based on public information (whois/routing registry) is about as good as any of us have it now. Granted, there are places that don't do that, and that is where we see route hijacking. A service like this would have to mandate it, to insure any /32 injected into it came from the peer that is authorized for the network the /32 belongs to. Since the AS_PATH can be maintained, I don't see an issue with secure BGP. Granted, the packets themselves won't be taking any path.

Jack Bates

14

Before everyone goes off and re-invents the wheel, please heed the advice
already provide by Randy, Steve, and Valdis. Community instigated RTBH is
used by a variety of Operational Security Communities. _Experience_ has
demonstrated caution. _Experience_ has pointed to the ways you use these
tools. _Experience_ has also demonstrated that you DO NOT let the bad guys
know about the details of what you do to fight them.

The people who DOS your network are most like know - if not already on
NANOG!

All of you what are getting fired up about a "Global Blackhole Service"
.....

1. Make sure you and your upstream have an agreement on how to work DDOS
incidents.
2. Make sure you and your peer have an agreement on how to work DDOS
incidents.
3. Establish a procedure for resolving DDOS incidents.
4. Get vetted and join Operational Security Communities.

Details for all of this are on the NANOG archives:

http://www.nanog.org/presentations/archive/

Keyword search "Security"

Get this done first, then consult with your peers in those Operational
Security communities. Not on a forum which every bad guy in the world who
has a clue about Networking is keeping their eye on.

Barry

15

* Valdis Kletnieks:

16

eventually, the rpki will give you the first half, authentication
of the owner of the ip space. this leaves, as smb hinted, securing
the request path from the black-hole requestor to the service and
of the service to the users.

smb:

You can't do this without authoritative knowledge of exactly who
owns any prefix; you also have to be able to authenticate the
request to blackhole it. Those two points are *hard*.

randy

17

Nuno et all,
Count me in for this..
Cheers,

--Ricardo
http://www.cs.ucla.edu/~rveloso

18

in the last 24 hours we received two denial of service attacks with
something like 6-8GBit volume. It did not harm us too much, but e.g.
one of our upstreams got his Amsix-Port exploded.

[...]

Therefore I had the following idea: Why not taking one of my old
routers and set it up as blackhole-service. Then everyone who is
interested could set up a session to there and

Hi Jens,

We do something similar globally with our bogon route server project.
We'd be happy to host and maintain a similar setup.

John

19

* Steven M. Bellovin:

As Randy and Valdis have pointed out, if this isn't done very carefully
it's an open invitation to a new, very effective DoS technique. You
can't do this without authoritative knowledge of exactly who owns any
prefix; you also have to be able to authenticate the request to
blackhole it. Those two points are *hard*.

If you want to run a public exchange point, you need to solve the same
announcement validation problem. Multiple organizations appear to do
it successfully, so it can't be that difficult.

20

No you don't.

And yes it is.

To be clear, I am not saying it should or should not be done, just that your comparison is invalid.