This afternoon at around 12:17 central time today we began learning the subnet for the Equinix IX in Chicago via a transit provider; we are on the IX as well. The subnet in question is 208.115.136.0/23. Using stat.ripe.net I can see that this subnet is also being learned by others, see the snip below. On our network this caused a nasty routing loop until we figured out what was wrong. My current best understanding is that because the route was learned via eBGP it trumped the OSPF learned route. As soon as I filtered the advertisement from my transit provider everything returned to normal. What am I doing that isn’t best practices that would have prevented this?
Thanks,
graham
RIPE Info
1 RRCs see 1 peers announcing 208.115.136.0/23 originated by AS32703
· ▼RRC00 in Amsterdam, Netherlands sees 1 ASN orginating 208.115.136.0/23.AS32703
o ▼**AS32703** is seen as the origin by 1 peer.192.102.254.1
§ ▼**192.102.254.1** is announcing route AS395152 AS63297 AS6327 AS36280AS32703.
you're setting the next-hop of the prefixes learned at the IXP to be your own IP address from the IXP subnet (i.e. 208.115.136.0/23).
When your routers learn this address from an external source, that is preferred to your internal OSPF route. Ergo your IX traffic is sent out via transit.
There are two things you should do:
1. change the bgp distance for ebgp to be higher than all your IGPs. On a cisco router, you would use something like:
router bgp xxx
address-family ipv4
distance bgp 200 200 200
address-family ipv6
distance bgp 200 200 200
2. use next-hop-self on internal ibgp sessions to ensure that when you redistribute the eBGP routes learned from your IX towards the internals of your network, the next-hop address is set to be the loopback address of your peering router. I.e. you remove the requirement for your internal network to know anything about the IXP address range.
Nick
Thank you Nick.
Graham Johnston
Manager, Network Services
Westman Communications Group
1906 Park Avenue | Brandon, MB | R7B 0R9
204-717-2829 |
johnstong@westmancom.com
This afternoon at around 12:17 central time today we began learning the subnet for the Equinix IX in Chicago via a transit provider; we are on the IX as well. The subnet in question is 208.115.136.0/23. Using stat.ripe.net I can see that this subnet is also being learned by others, see the snip below. On our network this caused a nasty routing loop until we figured out what was wrong. My current best understanding is that because the route was learned via eBGP it trumped the OSPF learned route. As soon as I filtered the advertisement from my transit provider everything returned to normal. What am I doing that isn¡¯t best practices that would have prevented this?
Thanks,
graham
RIPE Info
1 RRCs see 1 peers announcing 208.115.136.0/23 originated by AS32703
¡¤ ¡åRRC00 in Amsterdam, Netherlands sees 1 ASN orginating 208.115.136.0/23.AS32703
o ¡å**AS32703** is seen as the origin by 1 peer.192.102.254.1
¡× ¡å**192.102.254.1** is announcing route AS395152 AS63297 AS6327 AS36280AS32703.
I have a policy applied to my upstreams and peers to deny the IXP’s LANs were connected to. I don’t think of any reason to learn these routes from someone else’s network.
I’ve been bit by this in the past at two different exchanges. I too have a policy applied to deny IXP LANs from upstreams and peers. It would be nice if there was a list of all IXP LANs somewhere that we could generically add to all upstream and peers.
* christopher.morrell.nanog@gmail.com (Christopher Morrell) [Thu 28 Mar 2019, 14:35 CET]:
I've been bit by this in the past at two different exchanges. I too have a policy applied to deny IXP LANs from upstreams and peers. It would be nice if there was a list of all IXP LANs somewhere that we could generically add to all upstream and peers.
I like Nick Hilliard's posted solution much better than creating static bogon lists that people will eventually forget about.
-- Niels.
IXPs can use RPKI ROAs to signal to the world what their intentions are!
IXPs could either create a ROA with an Origin ASN of '0' to suggest to
the world that the peering lan prefix should never be visible in the
DFZ, or they can specify their own services ASN and simply not announce
the prefix. In either case IXPs should carefully specify the Max Length
value to be the same as the Prefix Length value of the peering lan
prefix.
Kind regards,
Job
There is two pieces to help prevent this type of failure:
1/ Equinix should have created a RPKI ROA for 208.115.136.0/23, with an
Origin ASN of 0 or one of their own ASNs, and a Max Length of 23.
2/ You should implement RPKI based BGP Origin Validation in your network
and honor those ROAs.
Kind regards,
Job