BCP38 - Internet Death Penalty

Ok, let's haul this up out of the other thread.

It seems consensus that the anti-source-address-spoofing provisions (at
least) of BCP38 have long since become critical to mitigating (and eventually
preventing) UDP attacks like DNS reflection and such, and that such attacks
are uniformly considered Bad Things.

It also seems that, with 13 years to get it done, even if equipment makers
have put usable working knobs into their edge routers and concentrators,
sufficient numbers of IAPs have not started turning them on.

The problem here is, of course, one of externalities and the Common Good,
hard sales to make in a business environment.

But have we reached the point where it's time to start trying?

Do we need to define a flag day, say one year hence, and start making the
sales pitch to our Corporate Overlords that we need to apply the IDP to
edge connections which cannot prove they've implemented BCP38 (or at very
least, the source address spoofing provisions thereof)? Put this in
contracts and renewals, with the same penalty?

Do the engineering heads at the top 10 tier-1/2 carriers carry enough water
to make that sale to the CEOs?

Cheers,
-- jr 'will rouse rabble for food' a

Unfortunately, no - else it would've come to pass quite some time ago.

How would one prove this? (In particular, consider the test "have them
download the spoofer code from SAIL and run it" - I'm positive there will
be sites that will put in a /32 block for the test machine so it "fails"
to spoof but leave it open for the rest of the net).

My observation is that a lot of people sometimes struggle to just hold their routing together at times, let alone to do something that is a second tier feature/capability.

- Jared

An excellent question. I suspect the largest collection of problem
networks are cable/DSL eyeball networks; certainly a cabal of network
ops types could be formed, anonymously to the carriers, who could run
test software from home...

I'm sure there are a bunch of ways that could reasonably give you a heads
up that it's time to investigate. Due process is certainly called for,
but clearly, lesser threats (if any have been made) aren't solving the
problem.

Are you conceding that BCP38 *will* solve the problem? Cause that's
Question One.

Cheers,
-- jra

(Mobile device)

I'll certainly say that I've worked for many an imperfect network when it comes to this. I've always tried to drop bogons (non routed space) including spoofed packets. While many a network is imperfect, there are places where we can improve. The high-speed servers in data centers or part of your VM infrastructure is one example of a place where:

a) They aren't running BGP for multihoming with 10k prefixes
b) have a fixed IP subnet for that edge host or management LAN
c) could quickly have unicast-rpf enabled with no impact.

If you have folks bringing in/out either known or unknown VMs, you must treat these as a possible risk. Same goes for any colocation environment.

Those T1 customers you still have? DS3 with one prefix and BGP on? Turn on unicast-rpf there too. You won't break anyone.

I recall going around and turning it on dozens of T1's at a time since they all had static routes. Just because the link size is now gig-e (or 10G or 100G) doesn't mean it's not worth doing.

Consider this my plea to the community. Ask the question: Can we spoof? What happens when law enforcement comes to the door to confiscate a machine because it was involved in a 10's of gigs of attack? what if it's 100's of gigs? At what size of attack for what duration will make things change? Are there simple places where unicast-rpf makes sense? In front of the firewall is a simple place to drop spoofed packets. You'd be surprised how many of them are broken and emit an internal IP anyways, so don't leak that.

I certainly see many places where simple steps like this will improve the overall ecosystem and make us safer.

Machines get compromised, but limit the scope of the possible abuse. If nothing is done, will udp/53 become blackholed just like tcp/25 is for many folks? Is that the type of network you want to debug and troubleshoot?

- Jared

I wish the Internet census people would try spoofing from their "botnet" and tell us which ISPs allow spoofing. I don't think this will fixed until there is a hall of shame or some kind of law requiring this and offenders would be fined.

Can't we get homeland security into this? Threat to US national security if people can spoof? :stuck_out_tongue:

I'm not sure you want this regulated.

Jared Mauch

But have we reached the point where it's time to start trying?

Yes.

Do we need to define a flag day, say one year hence, and start making the
sales pitch to our Corporate Overlords that we need to apply the IDP to
edge connections which cannot prove they've implemented BCP38 (or at very
least, the source address spoofing provisions thereof)? Put this in
contracts and renewals, with the same penalty?

Yes, but scope the problem a little differently.

1. The general IDP does not apply to stub networks which do not speak
BGP. It is for those stubs' ISPs to determine how they'll handle
mis-sourced traffic they receive from those networks.

2. A BGP origin-only network is required to secure its BGP-speaking
borders against source address spoofing. It may also secure spoofing
from downstream networks (and in fact it SHOULD do so) but it avoids
the IDP so long as its BGP-speaking borders are secured.

3. A BGP transit network is required to secure all components of its
network against source address spoofing, including all non-BGP stub
customers and all origin-only BGP customers. It is not expected to
prevent spoofed packets from arriving from neighboring transit BGP
networks.

It is also expected to promptly assist (24/7/365) with trace requests
from any individual presenting legitimate credentials as the assignee
of a particular source address and presenting with reasonable evidence
that packets with a spoofed address cross a specifically identified
system owned by the transit network. Where the packet stream
continues, these trace requests must promptly result in identification
of the actual source of the packet (if within the transit network's
system) or the identification of the neighboring system, the specific
entry point and high-level contacts within the neighbor system capable
of continuing the trace.

Some number of misconfigurations which permit spoofed packets from
components of the transit network's components are to be expected and
promptly corrected.

4. Applying the IDP _does not_ mean you cut off the network. That'll
piss of your customers as much or more than it pisses off theirs. The
position is untenable. Instead, the IDP consists of redirecting the
offender's public presence web sites to a web site which proclaims the
IDP, lists the causes of the IDP and lists the actions required to
lift the IDP.

5. IDP can't be a local decision. We should elect or empanel or
otherwise select a group of individuals who decide both when a network
gets the IDP and when the IDP is lifted. Compliance with the group's
decision to impose an IDP can be optional but a riot of RBLs like have
developed in the anti-spam community would cause at least as much
trouble as it fixes.

Do the engineering heads at the top 10 tier-1/2 carriers carry enough water
to make that sale to the CEOs?

To ask the CEOs to authorize cutting off access to a competitor's web
site with the full support and approval of a group of recognized
Internet luminaries?

Regards,
Bill Herrin

From: "William Herrin" <bill@herrin.us>

Yes, but scope the problem a little differently.

1. The general IDP does not apply to stub networks which do not speak
BGP. It is for those stubs' ISPs to determine how they'll handle
mis-sourced traffic they receive from those networks.

So, here, you mean customers of such as "Road Runner Business", who
have an office full of workstations and maybe some servers.

The goal, unless I badly misunderstood it, was to *drop forged traffic
coming from this sort of source (assuming you generalize "my PC at
home on a cablemodem" as the limiting example of this class, which I
do).

2. A BGP origin-only network is required to secure its BGP-speaking
borders against source address spoofing. It may also secure spoofing
from downstream networks (and in fact it SHOULD do so) but it avoids
the IDP so long as its BGP-speaking borders are secured.

This is the next size up of edge network; a buyer of transit.

This item does no good; you're expecting *the potential bad actor*
*not to act badly*.

3. A BGP transit network is required to secure all components of its
network against source address spoofing, including all non-BGP stub
customers and all origin-only BGP customers. It is not expected to
prevent spoofed packets from arriving from neighboring transit BGP
networks.

*This* is Road Runner. Also Comcast, Mindspring, and the other Top 40
eyeball networks. It is also, of course, larger carriers who sell access
in addition to more traditional transit and possibly peering.

AFAICT, this is the spot where source-address-spoofing needs to be
*enforced*; the people selling connections and transit here *know* what
addresses should be coming in those pipes, and can therefore -- if their
gear permits, and it damned well should by now -- force the dropping of
all packets coming in with bogus source addresses.

It is also expected to promptly assist (24/7/365) with trace requests
from any individual presenting legitimate credentials as the assignee
of a particular source address and presenting with reasonable evidence
that packets with a spoofed address cross a specifically identified
system owned by the transit network. Where the packet stream
continues, these trace requests must promptly result in identification
of the actual source of the packet (if within the transit network's
system) or the identification of the neighboring system, the specific
entry point and high-level contacts within the neighbor system capable
of continuing the trace.

Assuming they pass the packets at all, which is what I'm trying to prevent,
myself. Surely, special case handling will need to be done for customers
who are multihomed, and may originate packets from connection A out
connection B, but *this is the layer* where that must be done; you can't
do it any closer to the backbone, since the necessary administrative
knowledge isn't available there.

4. Applying the IDP _does not_ mean you cut off the network. That'll
piss of your customers as much or more than it pisses off theirs. The
position is untenable. Instead, the IDP consists of redirecting the
offender's public presence web sites to a web site which proclaims the
IDP, lists the causes of the IDP and lists the actions required to
lift the IDP.

Unless I misunderstand you there, you're suggesting that inbound HTTP to
public websites *operated by the spoofing entity* should be redirected
to a site that shames them? Yeah, that will piss them off less... :slight_smile:

5. IDP can't be a local decision. We should elect or empanel or
otherwise select a group of individuals who decide both when a network
gets the IDP and when the IDP is lifted. Compliance with the group's
decision to impose an IDP can be optional but a riot of RBLs like have
developed in the anti-spam community would cause at least as much
trouble as it fixes.

> Do the engineering heads at the top 10 tier-1/2 carriers carry
> enough water
> to make that sale to the CEOs?

To ask the CEOs to authorize cutting off access to a competitor's web
site with the full support and approval of a group of recognized
Internet luminaries?

The problem with that sub-approach is that luminaries (of the scale that
everyone will automatically listen to them), as Jon Postel has said, do
not scale.

Cheers,
-- jra

but they are paying attention....

/bill

...

Do the engineering heads at the top 10 tier-1/2 carriers carry enough water
to make that sale to the CEOs?

To ask the CEOs to authorize cutting off access to a competitor's web
site with the full support and approval of a group of recognized
Internet luminaries?

I'm not a lawyer, but I do work out with one at the gym.

I would strongly encourage people considering this to
discuss the following terms with their legal staff *first*:

Sherman Anti-Trust Act
Anti-competitive practice
Refusal to deal
restraint of trade
collusion
cartel

Once you've had a polite and cheerful discussion about
the feasibility of some set of companies in the public
space banding together to restrict access to a subset
of their competitors with your legal staff, we can
go back to discussing how best to configure our
routers.

Thanks!

Matt

From: "William Herrin" <bill@herrin.us>
1. The general IDP does not apply to stub networks which do not speak
BGP. It is for those stubs' ISPs to determine how they'll handle
mis-sourced traffic they receive from those networks.

So, here, you mean customers of such as "Road Runner Business", who
have an office full of workstations and maybe some servers.

Correct.

The goal, unless I badly misunderstood it, was to *drop forged traffic
coming from this sort of source (assuming you generalize "my PC at
home on a cablemodem" as the limiting example of this class, which I
do).

Indeed. But it isn't achievable. $Random_SOHO will continue to be
hacked on a regular basis. He doesn't have someone working for him
with the skill to prevent it. Further victimizing him with a game of
whack-a-mole helps nobody.

Besides, his failings aren't important to us. What's important to us
is that his failings don't impact us. We achieve that by insisting
that his ISP not leak his forged packets on to the public Internet. It
would be nice if his ISP didn't accept the forged packets at all, but
that's really not our problem and we don't need to make it our
business.

2. A BGP origin-only network is required to secure its BGP-speaking
borders against source address spoofing. It may also secure spoofing
from downstream networks (and in fact it SHOULD do so) but it avoids
the IDP so long as its BGP-speaking borders are secured.

This is the next size up of edge network; a buyer of transit.

This item does no good; you're expecting *the potential bad actor*
*not to act badly*.

At last count there are fewer than 45,000 such systems worldwide, not
millions upon millions like in group 1. This is a manageable number of
potential bad actors to whom the IDP would apply.

Also, we're not looking to catch bad actors here. We're looking to
catch sloppy actors. We catch bad actors at step 3 by spanking their
upstream transit ISPs for failing to prevent source spoofing.

3. A BGP transit network is required to secure all components of its
network against source address spoofing, including all non-BGP stub
customers and all origin-only BGP customers. It is not expected to
prevent spoofed packets from arriving from neighboring transit BGP
networks.

*This* is Road Runner. Also Comcast, Mindspring, and the other Top 40
eyeball networks. It is also, of course, larger carriers who sell access
in addition to more traditional transit and possibly peering.

Correct.

AFAICT, this is the spot where source-address-spoofing needs to be
*enforced*;

Unfortunately, it's also the spot where system complexity reaches a
point where as a purely technical matter, source address filtering
isn't always possible. You can filter your customers based on the
routes they say they plan send you and you can filter your own
internal networks based on the addresses you assign but filtering your
peers for falsely sourced packets can be as intractable as filtering
your upstream for falsely sourced packets.

Hence the additional expectations...

It is also expected to promptly assist (24/7/365) with trace requests
from any individual presenting legitimate credentials as the assignee
of a particular source address and presenting with reasonable evidence
that packets with a spoofed address cross a specifically identified
system owned by the transit network. Where the packet stream
continues, these trace requests must promptly result in identification
of the actual source of the packet (if within the transit network's
system) or the identification of the neighboring system, the specific
entry point and high-level contacts within the neighbor system capable
of continuing the trace.

4. Applying the IDP _does not_ mean you cut off the network. That'll
piss of your customers as much or more than it pisses off theirs. The
position is untenable. Instead, the IDP consists of redirecting the
offender's public presence web sites to a web site which proclaims the
IDP, lists the causes of the IDP and lists the actions required to
lift the IDP.

Unless I misunderstand you there, you're suggesting that inbound HTTP to
public websites *operated by the spoofing entity* should be redirected
to a site that shames them? Yeah, that will piss them off less... :slight_smile:

I don't care about about pissing them off. I care about pissing off my
customers. My customers will be pissed off if they can't reach their
customers and suppliers. Who will often be hosted by the target of the
IDP. But will much more rarely be the target of the IDP.

To ask the CEOs to authorize cutting off access to a competitor's web
site with the full support and approval of a group of recognized
Internet luminaries?

The problem with that sub-approach is that luminaries (of the scale that
everyone will automatically listen to them), as Jon Postel has said, do
not scale.

Which is A-OK because if we're applying more than 1 or 2 IDPs in a
year to folks who weren't intentionally bad actors then we're doing it
wrong. It shouldn't ever "scale."

This has already been tested vis a vis the anti-spam RBLs. There are
no new concepts here to challenge a court.

Regards,
Bill Herrin

If you are with a ISP that does not practice BCP 38 are you willing
to risk your neck that you won't be subject to a "aiding and abetting"
charge? All of us here know that spoofing address like this is a
criminal activity. We are all experts in the field and the courts
apply higher standards to us than they do to Joe Blogs. We know
machines get compromised. We know how to block spoofed traffic
from compromised machines.

From: "William Herrin" <bill@herrin.us>

> So, here, you mean customers of such as "Road Runner Business", who
> have an office full of workstations and maybe some servers.

Correct.

> The goal, unless I badly misunderstood it, was to *drop forged traffic
> coming from this sort of source (assuming you generalize "my PC at
> home on a cablemodem" as the limiting example of this class, which I
> do).

Indeed. But it isn't achievable. $Random_SOHO will continue to be
hacked on a regular basis. He doesn't have someone working for him
with the skill to prevent it. Further victimizing him with a game of
whack-a-mole helps nobody.

Besides, his failings aren't important to us. What's important to us
is that his failings don't impact us. We achieve that by insisting
that his ISP not leak his forged packets on to the public Internet. It
would be nice if his ISP didn't accept the forged packets at all, but
that's really not our problem and we don't need to make it our
business.

It's possible I badly misunderstand how things like unicast-rpf work,
Bill. I run much tinier networks than most people here.

But what I *do* understand of it is: you have to run it *at the edge
concentrator*, cause that's the only device which knows which packets to
accept... since *it assigned the address for the link*.

When I say "drop forged traffic coming from...", *who I mean is 'his ISP'*,
as you suggest in the next graf. I don't see that there's anyway to *know*
packets have a forged address anywhere north of the edge of the lowest tier
IAP the connection is served from.

Did I miss something? Or was I simply unclear?

>> 2. A BGP origin-only network is required to secure its BGP-speaking
>> borders against source address spoofing. It may also secure
>> spoofing
>> from downstream networks (and in fact it SHOULD do so) but it
>> avoids
>> the IDP so long as its BGP-speaking borders are secured.
>
> This is the next size up of edge network; a buyer of transit.
>
> This item does no good; you're expecting *the potential bad actor*
> *not to act badly*.

At last count there are fewer than 45,000 such systems worldwide, not
millions upon millions like in group 1. This is a manageable number of
potential bad actors to whom the IDP would apply.

Yes. These are the people to whom edge nodes and private non-BGP nets
speak; the tier 3 4 and 5 network providers who run edge concentrators
and assign addresses.

Also, we're not looking to catch bad actors here. We're looking to
catch sloppy actors. We catch bad actors at step 3 by spanking their
upstream transit ISPs for failing to prevent source spoofing.

...which you would detect ... how? *Those* aggregator networks have
no contractual reason to know what's a valid address coming to them,
unlike the networks to which end sites attach directly.

> *This* is Road Runner. Also Comcast, Mindspring, and the other Top 40
> eyeball networks. It is also, of course, larger carriers who sell access
> in addition to more traditional transit and possibly peering.

Correct.

> AFAICT, this is the spot where source-address-spoofing needs to be
> *enforced*;

Unfortunately, it's also the spot where system complexity reaches a
point where as a purely technical matter, source address filtering
isn't always possible. You can filter your customers based on the
routes they say they plan send you and you can filter your own
internal networks based on the addresses you assign but filtering your
peers for falsely sourced packets can be as intractable as filtering
your upstream for falsely sourced packets.

I don't believe that's what I said.

Filtering based on routes doesn't help; that's *destination address*, not
source, no?

Yes, filtering your peers, or even transit customers, is effectively
impossible; it has to be done where addresses are handed out.

>> 4. Applying the IDP _does not_ mean you cut off the network.
>> That'll
>> piss of your customers as much or more than it pisses off theirs.
>> The
>> position is untenable. Instead, the IDP consists of redirecting the
>> offender's public presence web sites to a web site which proclaims
>> the
>> IDP, lists the causes of the IDP and lists the actions required to
>> lift the IDP.
>
> Unless I misunderstand you there, you're suggesting that inbound
> HTTP to
> public websites *operated by the spoofing entity* should be
> redirected
> to a site that shames them? Yeah, that will piss them off less...
> :slight_smile:

I don't care about about pissing them off. I care about pissing off my
customers. My customers will be pissed off if they can't reach their
customers and suppliers. Who will often be hosted by the target of the
IDP. But will much more rarely be the target of the IDP.

Ok; I apologies; I have laid the bike down in the sandy corner at
this point. Huh?

>> To ask the CEOs to authorize cutting off access to a competitor's web
>> site with the full support and approval of a group of recognized
>> Internet luminaries?
>
> The problem with that sub-approach is that luminaries (of the scale that
> everyone will automatically listen to them), as Jon Postel has said, do
> not scale.

Which is A-OK because if we're applying more than 1 or 2 IDPs in a
year to folks who weren't intentionally bad actors then we're doing it
wrong. It shouldn't ever "scale."

Yes, but you can't measure such a panel on output, you have to measure
it on *input*, no?

Cheers,
-- jra

Careful: source address spoofing, like using a name you don't have on your
driver license *is not inherently a crime*. *Fraudulent behaviour which
is advanced thereby* makes it an additional crime.

SAS is sometimes necessary for testing.

Cheers,
-- jra

An argument could be made that "...fraud is fraud, is fraud, is
fraud..." and should vigorously discouraged. :slight_smile:

- ferg

Sure, Ferg. But my point was "be careful what you take as a proxy for
fraud; some behaviors have valid uses".

Cheers,
-- jra