Hello all,
We find ourselves trying to solve a requirement where we would like to test the viability of our paths to the internet and tear down the bgp session if it is determined to be faulty. We had an issue recently where we did not lose link or bgp but the carrier lost the ability to route traffic to the internet for us and our existing automatic detection and remediation strategies failed to detect this condition and we lost customer packets.
Conceptually, we have a pair of DCS7050-QX landing a fiber each from two ISPs with default routes on BGP at a dozen POPs around the US.
One of the ISPs is our primary transit, and one is predominantly for peered customers, but we can use it for transit during issues with the primary circuits.
I did some research on this and it seems like perhaps the on-boot event handler launching a python daemon to do this active probing out each isp circuit and then making config changes in response to transit failures might be the best option available to us.
However, I thought I’d reach out to the broader community to see if there’s a better way to solve this, has an example script, or if anyone has recommendations for methods of active monitoring for protecting against this sort of failure.
Thanks in advance for any insight and time.
Hi, Alex. If it helps, I’ve had a variant of this on our transit routers for enterprise purposes for a few years. We run DFZ and originate 0/0 and ::/0 internally, but because we follow them to the nearest egress (0/0 using NAT for path symmetry, ::/0 using conditional advertisement for path symmetry), we want to only originate the internal default routes if the external peering at that egress is “healthy”. For IOS-XR, it effectively looks like this:
route-policy PS-TRANSIT-UP-IPV4
if rib-has-route in TRANSIT-SUBNETS-V4 then
pass
endif
end-policy
!
route-policy PS-TRANSIT-UP-IPV6
if rib-has-route in TRANSIT-SUBNETS-V6 then
pass
endif
end-policy
prefix-set TRANSIT-SUBNETS-V4
212.123.212.184/30
end-set
!
prefix-set TRANSIT-SUBNETS-V6
2001:920:3815::64/127
end-set
neighbor-group EBGP-CRT-IPV4
…
address-family ipv4 unicast
…
default-originate route-policy PS-TRANSIT-UP-IPV4
neighbor-group EBGP-CRT-IPV6
…
address-family ipv6 unicast
…
default-originate route-policy PS-TRANSIT-UP-IPV6
We keep those stanzas simple — is the direct link to the peer up, and therefore the direct route is in our RIB — but depending on the platform you’re using, you may have more knobs to check things. For example, I can’t directly check if a BGP peer is up/down, though I can match on routes the peer has given us (or lack thereof).
-dp
Hi David,
There are several variants on Alex's problem. One is that there's an
upstream failure reflected in the BGP table but Alex doesn't see it
because he's only taking a default route. Your solution, or one like
it, should work for that. In a nutshell:
1. Take a full table
2. Filter everything but a selection of representative routes
3. Set static default routes tied to addresses within the representative routes.
If the representative routes disappear from the table, the static
defaults become invalid and leave the local routing table as well.
Or perhaps he has the reverse problem where he wants to advertise his
route only if the representative routes are there so that when his
anycast node has network problems it drops itself off the Internet and
allows others to take over.
Another variant is that BGP reports having the entire Internet table
but the packets don't get there. The upstream suffers from anything
between high packet loss to a misbegotten filtering rule that black
holes all his packets. He'd like to do some active polling via static
routes to the upstream and drop both advertised and received routes
when the polling indicates a path failure.
I thought the latter was what he was asking for, but on a second
read-through I see he talked about taking a default route via BGP
rather than a full table.
Regards,
Bill Herrin
I did some research on this and it seems like perhaps the on-boot event handler launching a python daemon to do this active probing out each isp circuit and then making config changes in response to transit failures might be the best option available to us.
Pretty much, yes. They don’t have any functionality like IP-SLA in EOS, but you can roll your own.
Ex:
https://github.com/arista-eosext/PingCheck