After a many month wait, we were ready to turn up our BGP peering sessions on a new Comcast fiber connection.
With our other providers (Level 3 and Verizon) we have edge routers that directly connect between the provider's on premise connection and our primary and a backup core routers. Each core router has a multihop BGP session with the provider's BGP router. The goal is to keep the single BGP router from being a single point of failure.
Comcast said they could not support two separate BGP peering sessions on the same circuit. Does anyone have any counter examples? We used to have this setup with Comcast 5+ years ago, but now they say they can't support it.
I started a thread around the same topic back on 10/16 of 2014. A Comcast engineer (who ultimately spoke to the national product manager) came back after discussing and said the same thing "We don't support that". I got a slightly longer explanation of:
Whenever we set up a bgp peer we do that to minimize downtime when doing
maint. It's hit or miss. HE required a second physicall connection NTT was
more than accommodating.
In a message written on Thu, Oct 13, 2016 at 05:48:18PM +0000, rar wrote:
The goal is to keep the single BGP router from being a single point of failure.
I don't really understand the failure analysis / uptime calculation.
There is one router on the Comcast side, which is a single point of
failure.
There is one circuit to your prem, which is a single point of failure.
To connect two routers on your end you must terminate the circuit
in a switch, which is a single point of failure.
And yet, in the face of all that somehow running two routers with
two BGP sessions on your end increases your uptime?
The only way that would even remotely make sense is if the routers
in question were horribly broken / mismanaged so (had to be?) reboot(ed)
on a regular basis. However if uptime is so important using gear
with that property makes no sense!
I'm pretty sure without actually doing the math that you'll be more
reliable with a single quality router (elminiation of complexity),
and that if you really need maximum uptime that you had better get
a second circuit, on a diverse path, into a different router probably
from a different carrier.
It comes down to sizing your failure domain. Any single upstream Transit
alone means the failure domain is the whole site (making assumptions about
your topology). As mentioned earlier, any single point of failure doesn't
reduce your failure footprint and gives little in terms of redundancy. Now
if you point that second router to a second provider, now you've reduced
the size of your failure domain to a single router/Transit, not the whole
site.
Steering clear of the failure domain conversation, if its of any benefit -
we can at least confirm that Comcast is willing to establish /29's for
multiple BGP connections at 56 Marietta/ATL.
These circuits are written on true wholesale/transit IP service contracts,
which may be the difference.
In our experience the Comcast Enterprise/Business groups have rather rigid
circuit provisioning profiles, and even if you are able to talk an engineer
into building a customer's configuration outside of their normal "scope",
it usually comes back to haunt you at some point in the future, even if
years later.
Will send a link to the Comcast enterprise ip transit profiles separately,
for reference, in the event you were not provided such previously...Or if
Comcast wholesale is on the list, of course feel free to chime in too!
It really seems like it's a grave oversight to *NOT* support multiple BGP sessions. I drop to two routers for that same reason, I can do maintenance on one, while the other carries traffic.
I completely concur. We spread our uplinks across separate boxes and we have /29 allocations. Get the best of all worlds. But if I only had one provider, I'd want to have multiple BGP sessions for this reason.
