Is there a generally accepted method of automatically altering exit
policies within an AS?
I'd like to dynamically change from best-exit to a "hot potato" exit
policy when an internal DS3 fails. We fail over to a much lower
bandwidth link and would like to avoid sending anything but internal
traffic over that link. If it's not already clear, this change needs to
happen automatically.
I realize that there are two means of accomplishing this:
(1) Set a weight on all routes received from the eBGP peer at each
location so that it prefers the direct eBGP peer.
(2) Sever the iBGP session by tying the iBGP session to an interface
IP address rather than a loopback IP. When the DS3 goes down, so
will the knowledge of the remote exit point.
The devil's in the details however. I can't figure out how to make the
weight approach work on routes that were received prior to the circuit
failure or how to remove the weights once the circuit comes back up.
Severing the iBGP session seems drastic to me, and I'm worried that our
advertised routes will be dampened by peers if the internal DS3 starts
flapping.
Any input from wiser peers would be greatly appreciated.
You can "nail" down your announcements to external peers by tying their
network blocks to a route-of-last resort on one of your loopbacks. This
will prevent flapping externally.
Point taken, but it's actually difficult to nail down all of our
routes. We have some lone /24's that are not subnetted and thus cannot
be used with an 'ip route ... null0' statement. When WAN connectivity
drops, the routes flap if we don't have a stable iBGP session. Thus I'd
like to steer well clear of severing the iBGP session.
The weights can be added/removed automatically by using a route-map on
the routes that will be added/removed by the interface going down.
Only a single internal /30 route will be removed when an interface goes
down. I can't come up with a route-map implementation that would
add/remove the weights to the routes already received from our eBGP
neighbors. If I'm missing something, please let me know.
Normally, however, you wouldn't use iBGP for this and you'd use a
heavier, link-aware internal routing protocol like ISIS or OSPF.
We use OSPF internally, but it just carries internal infrastructure
addresses. I understand that OSPF is link-aware and can carry knowledge
of link bandwidth, but I don't see how it would fit into our exit path
policies.
We accept full routes from our eBGP neighbors and it's not advisable to
inject those into OSPF. Our normal policy must be best-exit, which
leaves iBGP as the decision-maker unless I'm missing something. Is
there a better IGP-based method of choosing a network exit path that
would solve these problems? I ask because I'm curious, not because I
know the answer.
Perhaps a drawing of your architecture might make your travails more clear?
Benjamin Howell wrote:
You can "nail" down your announcements to external peers by tying their
network blocks to a route-of-last resort on one of your loopbacks. This
will prevent flapping externally.
Point taken, but it's actually difficult to nail down all of our
routes. We have some lone /24's that are not subnetted and thus cannot
be used with an 'ip route ... null0' statement. When WAN connectivity
drops, the routes flap if we don't have a stable iBGP session. Thus I'd
like to steer well clear of severing the iBGP session.
Not subnetting them doesn't mean you can't
ip route a.b.c.d 255.255.255.0 null0 250
while still routing the /24s internally (with lower metric) or having them
connected on some interface.
Only a single internal /30 route will be removed when an interface goes
down. I can't come up with a route-map implementation that would
add/remove the weights to the routes already received from our eBGP
neighbors. If I'm missing something, please let me know.
...
I'd like to dynamically change from best-exit to a "hot potato" exit
policy when an internal DS3 fails. We fail over to a much lower
bandwidth link and would like to avoid sending anything but internal
traffic over that link. If it's not already clear, this change needs to
happen automatically.
Are you talking about a single internal DS3, or the more general case of "if any of our internal DS3s are down, we need to route differently"?
If it's a simple case of two DS3 connected routers which are iBGP peers and also have directly connected eBGP peers, could you use route-maps to set ip next-hop on iBGP exchanged external routes (setting the ip next-hop to be the IP of the other end of the internal DS3, with a second IP of an eBGP neighbor interface)? I haven't tried it, but it seems like it might do what you want.
(1) Set a weight on all routes received from the eBGP peer at each
location so that it prefers the direct eBGP peer.
(2) Sever the iBGP session by tying the iBGP session to an interface
IP address rather than a loopback IP. When the DS3 goes down, so
will the knowledge of the remote exit point.
Another possiblility (I've never tried) would be to configure multiple iBGP sessions...one using loopback IPs, the other using the DS3 interface IPs, exchanging internal routes over both sessions, while exchanging external routes over only the second. If the DS3 goes down, the session exchanging external routes dies. I'm not sure you can do this, but I think by having different peer/endpoint IPs (loopbacks for one session, serial interface IPs for the other), it may work.
It may be appropriate to move this thread to the *-nsp list appropriate for your brand of routers.
>>You can "nail" down your announcements to external peers by tying their
>>network blocks to a route-of-last resort on one of your loopbacks. This
>>will prevent flapping externally.
>
>Point taken, but it's actually difficult to nail down all of our
>routes. We have some lone /24's that are not subnetted and thus cannot
>be used with an 'ip route ... null0' statement. When WAN connectivity
>drops, the routes flap if we don't have a stable iBGP session. Thus I'd
>like to steer well clear of severing the iBGP session.
Not subnetting them doesn't mean you can't
ip route a.b.c.d 255.255.255.0 null0 250
while still routing the /24s internally (with lower metric) or having them
connected on some interface.
Whoops, some oversights make you feel like an idiot. You're right.
>Only a single internal /30 route will be removed when an interface goes
>down. I can't come up with a route-map implementation that would
>add/remove the weights to the routes already received from our eBGP
>neighbors. If I'm missing something, please let me know.
...
>>>I'd like to dynamically change from best-exit to a "hot potato" exit
>>>policy when an internal DS3 fails. We fail over to a much lower
>>>bandwidth link and would like to avoid sending anything but internal
>>>traffic over that link. If it's not already clear, this change needs to
>>>happen automatically.
Are you talking about a single internal DS3, or the more general case of
"if any of our internal DS3s are down, we need to route differently"?
If it's a simple case of two DS3 connected routers which are iBGP peers
and also have directly connected eBGP peers, could you use route-maps to
set ip next-hop on iBGP exchanged external routes (setting the ip next-hop
to be the IP of the other end of the internal DS3, with a second IP of an
eBGP neighbor interface)? I haven't tried it, but it seems like it might
do what you want.
Indeed, I'll give it some thought. That seems like it should work. In my
case, it is just two DS3 connected routers. I figured I'd leave the
question open-ended though for other readers' benefit.
Another possiblility (I've never tried) would be to configure multiple
iBGP sessions...one using loopback IPs, the other using the DS3 interface
IPs, exchanging internal routes over both sessions, while exchanging
external routes over only the second. If the DS3 goes down, the session
exchanging external routes dies. I'm not sure you can do this, but I
think by having different peer/endpoint IPs (loopbacks for one session,
serial interface IPs for the other), it may work.
Actually this suggestion seems to be a common theme. I hadn't considered
this possibility and it seems like it should work fine. David Burns also
suggested this in an email that wasn't directed to the list.
Thanks for everybody's input. I should have some workable options now.