TTL exceeded messages are not important. They're useful to diagnostics
but nothing breaks if they're lost. There's no need to broadly allow
misconfigured customers to originate packets from RFC1918 space nor to
allow them to originate ICMP type 11 (time exceeded) messages from
RFC1918 space. You should not do so.
The packets you're worried about here are fragmentation needed: ICMP
type 3 code 4. Fragmentation needed messages are used for Path MTU
discovery. When PMTUD breaks, TCP fails. Path MTU discovery is the one
place in the core TCP/IP protocol stack where the end-to-end principle
was abandoned. TCP requires the ability to receive this notification
from any midpoint node in order to shrink packets too large for that
midpoint node's next hop. It's the most broken part of the TCP/IP
design.
Unfortunately, your solution of allowing ICMP type 3 messages from
RFC1918 space is dysfunctional. Even if you accept the packets (and
tailor your acceptance so that it only applies to ICMP type 3 messages
with a source address in RFC1918 space) the packets are highly likely
to be discarded elsewhere in the path.
In the past, I've suggested that vendors implement a feature allowing
destination unreachables generated from a privately addressed
interface to be originated from a different, user-configured public IP
address. So far I haven't seen any takers.
Useful automated reverse path filtering can ONLY be used when there is exactly ONE valid path to which and from which packets can be received. This is where strict mode uRPF actually works.
This seems to be predicated on /strict/ uRPF enforcement.
As for loose mode, it's basically useless in a BCP38 discussion. Loose mode only filters bogons. It doesn't prevent impersonation of any IP address currently routed in the system and doesn't do anything at all on a router with a default route.
Okay. I didn't see how /loose/ uRPF could do any good save for the DFZ or other situation where there isn't a default route.
This thread has made me wonder if there isn't a need for a 3rd type of uRPF or comparable filtering wherein the incoming interface is a viable route in the RIB even if it's not the best route in the FIB.
You're thinking about it from the upstream perspective, where a route could be accepted but depreferenced and thus not actively used. Think about it from the downstream network's perspective, though. If you're my upstream, and I don't want to use your link for inbound traffic, but I'd like to be able to send out some traffic over the link, how can I advertise the prefix to you in a way that would both ensure that you have it in your table locally, so that uRPF is happy, but also to ensure no packets actually make *use* of that routing table entry? Sure, you could tag the routes with 'no-export', but that only prevents the prefix from propagating outward, it doesn't prevent traffic on that router from using the routing table entry. You can try adjusting your MED, and hope the upstream doesn't squash the MED back to a default value it applies to all its customers. For the most part, you're up against a wall. You don't know how your upstream's route selection process is stacked with respect to routes you advertise, so you have no certainty that if you announce a prefix to them, it won't potentially be used to carry all your inbound traffic.
Interesting points Matt. Hence why I asked.
The only way to be sure that you won't take inbound traffic on a link is to not advertise the prefix at all across that link.
I agree that's the only way to be absolutely certain.
I would naively wonder if AS Path pre-pends would help mitigate this.
However, based on the RIB vs FIB sub-threads in this larger thread, I'm beginning to doubt that such will work with uRPF because the route would be depreffed and thus not in the FIB thereby would be filtered by uRPF.
As Mr. Bush might say, "I recommend all my competitors implement this in their networks."
*nod*nod*
Thank you for the understandable explanation Matt.
This thread has made me wonder if there isn’t a need for a 3rd type of
uRPF or comparable filtering wherein the incoming interface is a viable
route in the RIB even if it’s not the best route in the FIB.
Hi Grant,
Two problems here:
uRPF reuses the existing FIB. Either the next hop matches the source (strict) or some entry in the FIB matches the source (loose). The hardware “fast path” to implement the per-packet FIB is already there; all it takes is a second lookup. Unless someone comes up with a really clever idea, that’s basically all the filtering you can squeeze out of the FIB. To do something more, you’d have to implement an additional “fast path” in the router that can act on every packet. That’s kinda expensive.
BGP is a distance-vector protocol, not a link-state protocol. The significance for filtering is that the BGP RIB on a given router does not describe the complete state of the network. It only knows about itself and its immediate neighbors. That’s not algorithmically guaranteed to be enough knowledge about the network to determine that a packet with a given source address can’t reasonably come from a particular interface.
Consider the following scenario:
A - B - C - D - E - A
F
B receives a packet from A to F via C. Is the packet legitimate? B can’t know because the only route to A in B’s RIB is the direct connection to A. C didn’t pass E’s route to A back to B because it was a longer, unpreferred path. C might not even have received the route from D.
So even if you go to the expense of building a fast path that can consider what’s in the RIB instead of just the FIB, you don’t have the information you need in the RIB either.
From: "Joel Halpern" <jmh@joelhalpern.com>
To: "Brian Turnbow" <b.turnbow@twt.it>
Cc: nanog@nanog.org
Sent: Tuesday, November 8, 2022 10:03:20 AM
Subject: Re: BCP38 For BGP Customers
There is work a tthe IETF on an addon to RPKI called ASPA. There is a
draft that describes how the combiantion of ASPA and RPKI can be used to
help with DDOS prevention.
There is also a working group at the IETF called SAVNET that is looking
at what technological additions can be made to address the shortcomings
in BCP 38. In fairness, there is distinct disagreement as to what those
shortcomings are, and whether the ideas being presented can help. Input
from more operators would be great. (For completeness, I am a co-chair
of that working group.)
Wait; people are actually trying to implement BCP38, still? :-}
Thank you for taking the time to reply and help me understand the shortcomings of uRPF better.
I wonder if Feasible Path uRPF or Enhanced Feasible Path uRPF might help the situation. However I suspect they both suffer from the FIB != RIB problem and associated signaling.
That's a fairly good way to think about it. BGP knows -a- path and
sometimes it knows more than one but it simply doesn't have signal on
the totality of feasible paths for a particular IP address. No
distance-vector protocol can.
There's more than this going on as well, because there's a
number of other things going on, the IETF has created a SAVNET working
group to see if it's possible to do something here, and there's also
work in the SIDROPS WG that isn't yet adopted but may be.
The intent would be to include things like the ASPA work with
the SIDR/RPKI work to permit a proof to exist for SAV purposes. This
may not include all the p2p IP space that would exist between the
networks, and if one doesn't publish ASPA data for things like all those
cloud on-ramp type services, you may end up with traffic blackholed or
other side-effects.
Simply put, SAV/BCP-38 et al is hard, and nearly impossible when
you get much further away from the subnet that traffic originates from.
We’ve seen Juniper EX9ks implement uRPF in such a way that if I have two (load-balanced) BGP connections to the EX9k, and uRPF is turned on facing me, I immediately experience ~50% outbound packet loss.
Methinks the EX9ks apply uRPF a little too close to the hardware and ignore the RIB.
