BCP38 Deployment

Mailman List Mirror (Read Only)
1

Hi all,

I'm Bingyang Liu, a ph.d student in Tsinghua University. My thesis topic is
on "source address validation".

Although BCP38 was proposed more than ten years ago, IP spoofing still
remains an attack vector [MIT-Spoofer] [ARBOR-Annual-Report] [Presentation
on NANOG Meeting] [Discussion in NANOG ML].

I did a lot investigation, but still have no idea why so many ISPs haven't
deploy BCP38. I enumerate three reasons I found, and I'd like your comments
very much.

1. Stub ASes: They rely on their providers to filter, so they won't deploy
BCP38 on their own.
2. Low tier transit ASes: They are most likely to deploy BCP38 on the
interfaces towards their customers.
3. Large or tier1 ASes: Their peers and customers are also large. So uRPF
may have false positive and ACLs are too large to manage.

I also asked some ISP guys in IETF today, they all agreed that IP spoofing
is an issue, but they may haven't deployed it. One key issue, I think, is
about incentive. i.e. you can filter, but you'll still receive spoofing
from providers and peers who haven't enforced BCP38.

best
Bingyang

2

While those reasons are somewhat valid, they are not the main reasons.

#1) Money.
Whenever someone asks "why...?", the answer is usually "money". It costs money - CapEx if your equipment doesn't support RPF, and OpEx even if it does. Plus opportunity cost if your customers don't like it or you screw up, as those customers will find someone who doesn't filter and move.

#2) Laziness.
When the question is "why have [you|they] not...?", the second most common answer is laziness. Some call it "inertia", but reality is people are busy, lazy, etc.

Please note the complete lack of smilies or other indication I am kidding or being sarcastic.

There is also ignorance, stupidity, malice (yes, some people actually attack others or sell to those who do), etc.

3

In a message written on Wed, Mar 28, 2012 at 11:00:39AM -0400, Patrick W. Gilmore wrote:

#1) Money.
Whenever someone asks "why...?", the answer is usually "money". It costs money - CapEx if your equipment doesn't support RPF, and OpEx even if it does. Plus opportunity cost if your customers don't like it or you screw up, as those customers will find someone who doesn't filter and move.

#2) Laziness.
When the question is "why have [you|they] not...?", the second most common answer is laziness. Some call it "inertia", but reality is people are busy, lazy, etc.

While Patrick is spot on, there is a third issue which is related
to money and laziness, but also has some unique aspects.

BCP38 makes the assumption that the ISP does some "configuration"
to insure only properly sourced packets enter the network. That
may have been true when BCP38 was written, but no longer accurately
reflects how networks are built and operated.

To get source address validation widely deployed it needs to be
baked into consumer CPE. The requirement needs to be a "default
on" in the DOCSYS specs, for instance. Residential gateways need
to come from the factory with unicast RPF turned on. BCP38 needs
to be applied at the OEM level in equipment maufacturing, not at
the operational level with ISP's.

There are, simply, too many variations in CPE devices to expect
ISP's to _configure_ them. Even when the configuration is
"standardized" (like DOCSYS) ISP's have to think really hard about
the operational impact of turning on a feature; and one buggy
implementationc can scuttle an idea network wide.

Which really comes back to Patrick's point #2. If the people who
care about this want to see a positive change they need to stop
badgering ISP's to implement BCP38 and start badgering
Linksys/Netgear/D-Link/Motorola/Apple/Touchstone/SMC/Westtel to
make unicast RPF a default part of their gateway implementation.
More importantly, they need to get them to brand it as a _feature_,
protect your computer from being used by hackers, our router insures
they won't use up all of your data cap! Then it will be something they
can sell, and thus something they will implement.

As long as folks keep beating on (consumer) ISPs to implement BCP38,
nothing will happen.

4

Leo,

#1) Money.
#2) Laziness.

While Patrick is spot on, there is a third issue which is related
to money and laziness, but also has some unique aspects.

BCP38 makes the assumption that the ISP does some "configuration"
to insure only properly sourced packets enter the network. That
may have been true when BCP38 was written, but no longer accurately
reflects how networks are built and operated.

An interesting assertion. I haven't looked at how end-user networks are built recently. I had assumed there continue to be customer aggregation points within ISP infrastructure in which BCP38-type filtering could occur. You're saying this is no longer the case? What has replaced it?

BCP38 needs

to be applied at the OEM level in equipment maufacturing, not at
the operational level with ISP's.

I don't believe this is either/or. I agree that BCP38 features should be turned on by default in CPE, however I believe it really needs to be enforced at the ISP level.

As long as folks keep beating on (consumer) ISPs to implement BCP38, nothing will happen.

Optimist.

Actually, given the uptick in spoofing-based DoS attacks, the ease in which such attacks can be generated, recent high profile targets of said attacks, and the full-on money pumping freakout about anything with "cyber-" tacked on the front, I suspect a likely outcome will be proposals for legislation forcing ISPs to do something like BCP38.

Regards,
-drc

5

Hi David, Leo, Patrick and all,

Considering the reasons you raised, do you think the following two things
can happen?

1. Give BCP38 the only practical anti-spoofing technique, can an ISP well
protect its customers by implementing BCP38? I don't think so, because I
think BCP38 is accurate near the source but inaccurate near the
destination, i.e. if its customer is the target of spoofing attack, its
capability to filter is relatively low.

2. Even if ineffective near the destination, is an ISP willing to deploy it
if it becomes easy to adopt and risk-free (no false positive)?

Sorry for my stupid and naive questions.

best
Bingyang

6

In a message written on Wed, Mar 28, 2012 at 08:45:12AM -0700, David Conrad wrote:

An interesting assertion. I haven't looked at how end-user networks are built recently. I had assumed there continue to be customer aggregation points within ISP infrastructure in which BCP38-type filtering could occur. You're saying this is no longer the case? What has replaced it?

Well, RFC3704 for one has updated the methods and tactics since BCP38
was written. Remember BCP38 was before even "unicast RPF" as we know it
existed.

Some relevant points from 3704:

3. Clarifying the Applicability of Ingress Filtering

   What may not be readily apparent is that ingress filtering is not
   applied only at the "last-mile" interface between the ISP and the end
   user. It's perfectly fine, and recommended, to also perform ingress
   filtering at the edges of ISPs where appropriate, at the routers
   connecting LANs to an enterprise network, etc. -- this increases the
   defense in depth.

5. Security Considerations
[snip]
   The closer to the actual source ingress filtering is performed, the
   more effective it is. One could wish that the first hop router would
   ensure that traffic being sourced from its neighboring end system was
   correctly addressed; a router further away can only ensure that it is
   possible that there is such a system within the indicated prefix.
   Therefore, ingress filtering should be done at multiple levels, with
   different level of granularity

I'm not saying ISP's can't or couldn't do it, what I am saying, and
RFC 3704 is repeating, is that it is cheaper/easier/faster and more
reliable to do it as close to the edge as possible. "The edge" is
not the edge of the ISP network, it is the edge of the entire
network, that is the /last router in the topology/. Today that
last router is owned and operated by the customer in most cases.

So if a provider drops off a modem with your service that also does
WiFi and the customer simply uses it, the provider is 100% responsible
for doing BCP 38, in my estimation. But as soon as the consumer
buys a routing device they become 100% on the hook for now operating
the last mile, and it is that device where the primary filtering
should take place. ISP's may still filter, for a defense in depth,
but they are no longer the edge of the network and as such their
responsibility is greatly diminished in my view.

BCP38 was written when a point to point handoff to a single customer was
standard, and that's easy to filter. Today a shared medium (like a
cable modem network) is common and more importantly connects to more
routers (home gateways), rathern than PC's. That's a funamental change
since BCP38 was written.

I'll also point out that operating systems fill a role here as well.
Many OS's won't let you spoof a layer 2 MAC address (try to write
a packet with a raw interface and it overwrites the source address)
but are happy to let an application send a packet with source layer
3 address that is forged! Sure, malware could always hack the OS
too, but it raises the bar. The community should demand that all
OS's default to not allowing L3 sources that aren't configured on
the box from leaving that box.

7

While I'm a big fan of RFP, it does require that operators be "good
citizens" for it to be effective. Like most of the Internet, it's
built on a "web" of trust.

8

I think the concern of RFC3704/BCP84, i.e., multihoming, is the
primary reason we don't see ingress filtering as much as we should.

Almost any network worth its salt these days is multihomed, making
strict RPF nearly impossible to pull off. Despite this, to my
knowledge, Juniper is one of the only vendors that provides
feasible-path RPF to deal with it. On Cisco and Brocade for example,
you're stuck doing some dark voodoo magic with BGP weights &
communities + strict RPF (refer to the previous money and laziness
points) to accomplish something that SHOULD be basic.

9

Exactly.

Either do it voluntarily or it will be done for you involuntarily at the federal level and you will have nobody but yourselves to blame.

The choice is yours.

-Dan

10

I would be surprised if this were true.

I'd argue that today, the vast majority of devices on the Internet (and certainly the ones that are used in massive D(D)oS attacks) are found hanging off singly-homed networks.

Regards,
-drc

11

Yep, one way is to give economic penalty.

But how about giving the _good_ ISPs economic reward? Say, some
transit ISPs deploy anti-spoofing techniques (e.g. uRPF), but only
filter those spoofing packets whose destination is the ASes having
purchased their *anti-spoofing service* ?

Bingyang

12

In a message written on Wed, Mar 28, 2012 at 08:45:12AM -0700, David Conrad wrote:

An interesting assertion. I haven't looked at how end-user networks are built recently. I had assumed there continue to be customer aggregation points within ISP infrastructure in which BCP38-type filtering could occur. You're saying this is no longer the case? What has replaced it?

Well, RFC3704 for one has updated the methods and tactics since BCP38
was written. Remember BCP38 was before even "unicast RPF" as we know it
existed.

I'm not saying ISP's can't or couldn't do it, what I am saying, and
RFC 3704 is repeating, is that it is cheaper/easier/faster and more
reliable to do it as close to the edge as possible. "The edge" is
not the edge of the ISP network, it is the edge of the entire
network, that is the /last router in the topology/. Today that
last router is owned and operated by the customer in most cases.

Yeahbut, the CPE isn't trusted. It would be _nice_ for customers
to be bcp38 clueful as well, but I don't think it's _required_ for
successful deployment from the ISP's standpoint. Even with a
system like DOCSIS where the CPE is semi-trustworthy from a
provisioning/etc standpoint, I don't think I'd _count_ on them.

In any case, isn't RPF really cheap these days on edge aggregation
routers? It's not like it's a new innovation or anything.

BCP38 was written when a point to point handoff to a single customer was
standard, and that's easy to filter. Today a shared medium (like a
cable modem network) is common and more importantly connects to more
routers (home gateways), rathern than PC's. That's a funamental change
since BCP38 was written.

DOCSIS was standardized in the mid to late 90's which more or
less predates bcp 38, and it has always been able to handle multiple
endpoints/modem. As I recall, there were specs for cable modem
nics for individual machines, but they never took off.

Mike

13

in a note (which didn't go anywhere in particular) i pointed out that
contract may address the same issue for which legislation may be
proposed, at least for "contractual closures" (sorry, a term of my
own, defined below) which share the property some jurisdictions have
of a finite access provider universe.

i mean "contractual closure" to be the performance guarantee (or
non-performance guarantee) present in a set of contracts for a
particular service.

think "china", after first abstracting all the negatives associated
with policy as a property of a distributed, shared, public resource,
or "firewalls 4 (bcp defined) good".

-e

14

Yeah, "contractual closures" might be a way to force the providers to
deploy BCP38.

However, when the customers become the target of a spoofing attack,
the provider may not be able to protect its customers, because ingress
filtering (including uRPF) is inefficient when done near the
destination. In other words, an ISP can deploy BCP38 or whatever, but
still cannot well protect its customers from spoofing attacks from
other ASes.

15

Yes, but RPF can be implemented in places other than the customer
edge. In those places, lack of widespread, easy, and vendor-supported
feasible-path uRPF is what I believe really hurts things.

Granted, this is along a different line than what the OP was talking
about, but in terms of answering the question of "why don't we see
ingress filtering as much as we should?", I think it's a large factor.

16

The ASes which enable spoofing need to have some penalty imposed or they will never engage in correct behavior.

Something like throwing all their traffic into scavenger class.

If their customers start complaining to them, maybe then they will shape up.

-Dan

17

Hi Darius,

Yes, I agree that feasible RPF solves the problem in a lot of scenarios.

However, in some other cases, the asymmetric routing is caused by
static routing, traffic engineering, policy routing, etc., where the
lengths of forward path and reverse path may differ, so feasible RPF
may also fail (false positive).

Bingyang

18

Also,

Don't forget that transit providers currently bill their customers to carry that spoofed/DoS traffic, why would they filter it when it's $$$$ on their balance sheets?

-Drew

19

In a message written on Wed, Mar 28, 2012 at 09:52:49AM -0700, Michael Thomas wrote:

Yeahbut, the CPE isn't trusted. It would be _nice_ for customers
to be bcp38 clueful as well, but I don't think it's _required_ for
successful deployment from the ISP's standpoint. Even with a
system like DOCSIS where the CPE is semi-trustworthy from a
provisioning/etc standpoint, I don't think I'd _count_ on them.

None of the routers are "trusted" if your perspective is right.

It's easy to find a path like:

"Tier 1 ISP" - Regional ISP - Local Provider - Subscriber - User

Techologically it may look like:

Tier 1 T640 core network with 10GE handoff
Regional Cisco GSR network with 1GE handoff
Local 1006 to Arris CMTS
Subscriber Motorola Cable Modem to NetGear SOHO Gateway
User Patron with Airport Express sharing a wired connection to WiFi

I don't trust any of the people in that list. More interesting
from a BCP38 perspective who should be doing the filtering? If you
were going to write it into law/regulation, where would you require
it?

Maybe all of them should, but can they from a technologial perspective?
There's multi-homing in that chain somewhere. Do you require it
at the first single homed place? If the subscriber is using a
NetGear that does both ethernet and cell card backup and is thus
multi-homed does that mean the user must do it? It's not even in
my list, but re-asking my previous question why don't we go a step
further and require the Operating System to do unicast RPF on-box?

I think given the thorny set of issues that taking a step back and
saying, "rather than a perfect solution, what gets us most of the
way there the cheapest, and quick" is a good question to ask. I'm
going to point to the local boxes. In my example the Netgear and
Airport devices are in a posion to do super-cheap unicast RPF. They
have (generally) one network behind them, and one way out. They
are CPU based boxes for which this check requires no hardware
changes. They don't even have enough interfaces in most cases to
multi-home, so the chance of it breaking is nil. And yes, while
the user may control both the end PC and these devices and thus be
able to turn it off and circumvent all of this, that's really not
the problem. The problem is infected machines spewing crap their
owners don't know about, and just having a separate device upstream
that stops it will do the job.

The perfect is the enemy of the good in this case. Solving this at the
consumer CPE level would remove 90-95% of the problem at zero hardware
cost, a very small software cost, and a very small support cost and
probably make us stop talking about this issue all together.

20

None of the routers are "trusted" if your perspective is right.

It's easy to find a path like:

"Tier 1 ISP" - Regional ISP - Local Provider - Subscriber - User

Techologically it may look like:

Tier 1 T640 core network with 10GE handoff
Regional Cisco GSR network with 1GE handoff
Local 1006 to Arris CMTS
Subscriber Motorola Cable Modem to NetGear SOHO Gateway
User Patron with Airport Express sharing a wired connection to WiFi

I don't trust any of the people in that list. More interesting
from a BCP38 perspective who should be doing the filtering? If you
were going to write it into law/regulation, where would you require
it?

I wasn't talking about laws/regulation, just responding to your comment
that lack of RPF in CPE was the bigger problem which doesn't seem right
to me. If I'm the owner of the CMTS network, I really shouldn't be trusting
the CM to be doing filtering for me. Maybe it would be nice to have RPF
checks in the CM to nip a spoofed DDOS before it ever gets on the HFC
network, but I wouldn't count on the CM not being compromised too, which
means I should probably be running RPF on the aggregation router as well.

Maybe all of them should, but can they from a technologial perspective?
There's multi-homing in that chain somewhere. Do you require it
at the first single homed place? If the subscriber is using a
NetGear that does both ethernet and cell card backup and is thus
multi-homed does that mean the user must do it? It's not even in
my list, but re-asking my previous question why don't we go a step
further and require the Operating System to do unicast RPF on-box?

It's been a long time since I've read bcp 38, but I thought its
intent was if you can reasonably do it, you *should* do it. multipath
obviously makes RPF problematic, but the vast majority of the
edge networks aren't multi-homed, so they really should be running
RPF just as a matter of... best common practice.

I think given the thorny set of issues that taking a step back and
saying, "rather than a perfect solution, what gets us most of the
way there the cheapest, and quick" is a good question to ask. I'm
going to point to the local boxes. In my example the Netgear and
Airport devices are in a posion to do super-cheap unicast RPF. They
have (generally) one network behind them, and one way out. They
are CPU based boxes for which this check requires no hardware
changes. They don't even have enough interfaces in most cases to
multi-home, so the chance of it breaking is nil. And yes, while
the user may control both the end PC and these devices and thus be
able to turn it off and circumvent all of this, that's really not
the problem. The problem is infected machines spewing crap their
owners don't know about, and just having a separate device upstream
that stops it will do the job.

The perfect is the enemy of the good in this case. Solving this at the
consumer CPE level would remove 90-95% of the problem at zero hardware
cost, a very small software cost, and a very small support cost and
probably make us stop talking about this issue all together.

Except for the small problem that getting cheap home router box
manufacturers to do just about anything is a pushing on string exercise.
So if I want to a) protect my network and b) be a good netizen, I'm
still going to want to do BCP 38 regardless of whether others violate
a, b or both. Right?

Mike