I have seen age old discussions on single AS vs multiple AS for backbone and datacenter design. I am particularly interested in operational challenges for running AS per region e.g. one AS for US, one EU etc or I have heard folks do one AS per DC. I particularly don't see any advantage in doing one AS per region or datacenter since most of the reasons I hear is to reduce the iBGP mesh. I generally prefer one AS and making use of confederation.
Zaid
I have seen age old discussions on single AS vs multiple AS for backbone and datacenter design. I am particularly interested in operational challenges for running AS per region e.g. one AS for US, one EU etc or I have heard folks do one AS per DC. I particularly don't see any advantage in doing one AS per region or datacenter since most of the reasons I hear is to reduce the iBGP mesh. I generally prefer one AS and making use of confederation.
Zaid
If you have good backbone between the locations, then, it's mostly a matter of personal preference. If you have discreet autonomous sites that are not connected by internal circuits (not VPNs), then, AS per site is greatly preferable.
Owen
We disagree.
Single AS worldwide is fine with or without a backbone.
Which is "preferable" is up to you, your situation, and your personal tastes. (I guess one could argue that wasting AS numbers, or polluting the table with lots of AS numbers is bad or un-ashetically pleasing, but I think you should do whatever fits your situation anyway.)
Multiple AS, one per region, is about extracting maximum revenue from
your client base. In 2000 we had no technical reason to do it, I can't see
a technical reason to do it today. This is a layer 8/9 issue.
jy
We're with Patrick on this one. We operate a single AS across seventy-some-odd locations in dozens of countries, with very little of what an eyeball operator would call "backbone" between them, and we've never seen any potential benefit from splitting them. I think the management headache alone would be sufficient to make it unattractive to us.
-Bill
Quoting Zaid Ali <zaid@zaidali.com>:
I have seen age old discussions on single AS vs multiple AS for backbone and datacenter design. I am particularly interested in operational challenges for running AS per region e.g. one AS for US, one EU etc or I have heard folks do one AS per DC. I particularly don't see any advantage in doing one AS per region or datacenter since most of the reasons I hear is to reduce the iBGP mesh. I generally prefer one AS and making use of confederation.
Zaid
Hi Zaid,
What timing - this is fresh on my mind too as I am in the middle of doing this myself with three locations, all with independent edges with different transit providers. I actually do have a private Layer2 circuit between, with one site being in the middle. I only have one public AS, but I have selected doing the confederation approach (which some may consider to be overkill with only three edges).
Each site has their own set of IPs and would originate out of their respective edge, and using EIGRP metric changes at each core to get 0.0.0.0/0 from another edge if the local fails. Each edge is then announcing each others' subnets with an extra pad or two to keep the asymmetrical routing down (the private L2 isn't as fast as my transits).
Good luck with your deployment!
-graham
Latest is here (which still needs a few minor comments incorporated):
<http://tools.ietf.org/html/draft-ietf-grow-unique-origin-as-00>
And the operative bits relative to this discussion are provided in
the title: "Unique Per-Node Origin ASNs for Globally Anycasted
Services"
-danny
>>
>>> I have seen age old discussions on single AS vs multiple AS for backbone and datacenter design. I am particularly interested in operational challenges for running AS per region e.g. one AS for US, one EU etc or I have heard folks do one AS per DC. I particularly don't see any advantage in doing one AS per region or datacenter since most of the reasons I hear is to reduce the iBGP mesh. I generally prefer one AS and making use of confederation.
>>
>> If you have good backbone between the locations, then, it's mostly a matter of personal preference. If you have discreet autonomous sites that are not connected by internal circuits (not VPNs), then, AS per site is greatly preferable.
>
> We disagree.
> Single AS worldwide is fine with or without a backbone.
> Which is "preferable" is up to you, your situation, and your personal tastes.We're with Patrick on this one. We operate a single AS across seventy-some-odd locations in dozens of countries, with very little of what an eyeball operator would call "backbone" between them, and we've never seen any potential benefit from splitting them. I think the management headache alone would be sufficient to make it unattractive to us.
-Bill
Right. I think that a single AS is most often quite fine. I think our
problem space is rather about how you organise the routing in your AS.
Flat, route-reflection, confederations? How much policing between
regions do you feel that you need? In some scenarios, I think
confederations may be a pretty sound replacement of the multiple-AS
approach. Policing iBGP sessions in a route-reflector topology? Limits?
Thoughts?
Cheers,
mh
While it's a very interesting read and it's always nice to know
what Danny is up to, the concept is a pretty extreme corner
case when you consider the original question. I took the original
question to be about global versus regional AS in a provider
backbone.
On the other hand if we'd had this capability years ago the notion
of a CDN based on anycasting would be viable in a multi-provider
environment. Maybe time to revive that idea?
jy
with one site being in the middle. I only have one public AS, but I have
selected doing the confederation approach (which some may consider to be
overkill with only three edges).
There are really several issues to consider, one of which certainly is
"overkill," but the others are:
1) in your case, you have to run allowas-in *anyway* because if your
transport or your "middle POP" goes down, so will your network and its
customers; so confederating isn't really buying you anything unless
your backbone is really vendor L3VPN
2) confederating / clustering can add to MED headaches and similar
While this is not directed at your deployment specifically, it is a
common newbie mistake to confederate something that doesn't need to
be, or to otherwise complicate your backbone because you think you
need to turn knobs to prepare for future growth. Guess what, that
growth might happen later on, but if you don't understand emergent
properties of your knob-turning, your plan for the future is really a
plan to fail, as you'll have to re-architect your network at some
point anyway, probably right when you need that scalability you
thought you engineered in early-on.
List readers should be strongly discouraged from confederating unless
they know they need to, understand its benefits, and understand its
potential weaknesses. In general, a network with effectively three or
six routers should never have a confederated backbone. The number of
guys who really understand confederating / route-reflection within the
backbone is very small compared to the number of guys who *think* they
are knowledgeable about everything that falls under "router bgp," our
beloved inter-domain routing protocol which gives the operator plenty
of rope with which to hang himself (or the next guy who holds his job
after he moves on.)
On the other hand if we'd had this capability years ago the notion
of a CDN based on anycasting would be viable in a multi-provider
environment. Maybe time to revive that idea?
That draft doesn't identify any particular technical challenges to
originating a prefix from multiple discrete origin ASNs other than the
obvious fact that they'll show up in the various "inconsistent origin
AS" reports such as CIDR Report, etc. It of course does identify some
advantages to doing it.
I imagine Danny McPherson and his colleagues have spent some time
looking into this, and can probably say with confidence that there are
in fact no real challenges to doing it today besides showing up in
some weekly email as a possible anomaly. It seems to be a taboo
topic, but once a few folks start doing it, I think it'll quickly
become somewhat normal.
Note that in the current IRR routing information system, it is
possible to publish two route objects, each with the same prefix, and
each with a different origin ASN. This is by design.
I have seen age old discussions on single AS vs multiple AS for backbone and datacenter design. I am particularly interested in operational challenges for running AS per region e.g. one AS for US, one EU etc or I have heard folks do one AS per DC. I particularly don't see any advantage in doing one AS per region or datacenter since most of the reasons I hear is to reduce the iBGP mesh. I generally prefer one AS and making use of confederation.
If you have good backbone between the locations, then, it's mostly a matter of personal preference. If you have discreet autonomous sites that are not connected by internal circuits (not VPNs), then, AS per site is greatly preferable.
We disagree.
Single AS worldwide is fine with or without a backbone.
Which is "preferable" is up to you, your situation, and your personal tastes.We're with Patrick on this one. We operate a single AS across seventy-some-odd locations in dozens of countries, with very little of what an eyeball operator would call "backbone" between them, and we've never seen any potential benefit from splitting them. I think the management headache alone would be sufficient to make it unattractive to us.
Experience with a major backbone in the early 2000's that spanned 50 core sites and 4 continents - single AS is not really a problem. We chose IS-IS with wide metrics as the IGP, and one-layer of route-reflection for the bgp mesh control.
The only reason I could possibly see doing multi-AS in a general case is if your route policies are different in different regions (i.e. in one region a peer AS is a 'peer' and in another region the same AS is a 'transit' or 'upstream'). You CAN do it with a single AS, but it's more painful...
I always look at confederations as a longer term plan because you have some idea how your backbone is going to shape out. Knowing where you are going makes confederation planning easier. Start with RR's and then see if confeds make sense.
Zaid
>>>>
>>>>> I have seen age old discussions on single AS vs multiple AS for backbone and datacenter design. I am particularly interested in operational challenges for running AS per region e.g. one AS for US, one EU etc or I have heard folks do one AS per DC. I particularly don't see any advantage in doing one AS per region or datacenter since most of the reasons I hear is to reduce the iBGP mesh. I generally prefer one AS and making use of confederation.
>>>>
>>>> If you have good backbone between the locations, then, it's mostly a matter of personal preference. If you have discreet autonomous sites that are not connected by internal circuits (not VPNs), then, AS per site is greatly preferable.
>>>
>>> We disagree.
>>> Single AS worldwide is fine with or without a backbone.
>>> Which is "preferable" is up to you, your situation, and your personal tastes.
>>
>>
>> We're with Patrick on this one. We operate a single AS across seventy-some-odd locations in dozens of countries, with very little of what an eyeball operator would call "backbone" between them, and we've never seen any potential benefit from splitting them. I think the management headache alone would be sufficient to make it unattractive to us.
>>
>> -Bill
>>
>>
>
> Right. I think that a single AS is most often quite fine. I think our
> problem space is rather about how you organise the routing in your AS.
> Flat, route-reflection, confederations? How much policing between
> regions do you feel that you need? In some scenarios, I think
> confederations may be a pretty sound replacement of the multiple-AS
> approach. Policing iBGP sessions in a route-reflector topology? Limits?
> Thoughts?I always look at confederations as a longer term plan because you have some idea how your backbone is going to shape out. Knowing where you are going makes confederation planning easier. Start with RR's and then see if confeds make sense.
Yes, I agree. Confed's is a choice, in particular if you need elaborate
policies between regions of your network. Police sessions between
sub-ASes, keep iBGP simple...
Say, you start with RR... If or once your network is large, and having
customers connected to it, migrating to conferedrations may be a
challenge. Right? Thoughts? Experiences?
mh
Single AS worldwide is fine with or without a backbone.
Which is "preferable" is up to you, your situation, and your personal
tastes. (I guess one could argue that wasting AS numbers, or
polluting the table with lots of AS numbers is bad or un-ashetically
pleasing, but I think you should do whatever fits your situation
anyway.)
Depends on your routing requirements, for example, you need to rely on a
default or disable as_path checks (or re-write the path) to be able to
see your other clusters (if they do need to communicate)...
I have seen age old discussions on single AS vs multiple AS for backbone and datacenter design. I am particularly interested in operational challenges for running AS per region e.g. one AS for US, one EU etc or I have heard folks do one AS per DC. I particularly don't see any advantage in doing one AS per region or datacenter since most of the reasons I hear is to reduce the iBGP mesh. I generally prefer one AS and making use of confederation.
Zaid
If you have good backbone between the locations, then, it's mostly a matter of personal preference. If you have discreet autonomous sites that are not connected by internal circuits (not VPNs), then, AS per site is greatly preferable.
We disagree.
Single AS worldwide is fine with or without a backbone.
Only if you want to make use of ugly ugly BGP hacks on your routers, or, you don't care about Site A being
able to hear announcements from Site B.
Which is "preferable" is up to you, your situation, and your personal tastes. (I guess one could argue that wasting AS numbers, or polluting the table with lots of AS numbers is bad or un-ashetically pleasing, but I think you should do whatever fits your situation anyway.)
I don't see any significant downside to AS number consumption given a 32-bit AS Number space.
I do see significant downsides to disabling BGP loop detection.
Owen
To be clear, when I said backbone, I meant that if a packet arrives at site A destined for site B, it goes across
some form of internal path and not back out to the internet. That Site A and Site B learn each other's routes
via iBGP and not via third-party ASNs.
If A learns B's addresses from a third party ASN, then, it is highly desirable (IMHO) to have A and B in
separate ASNs.
Owen
You are highly confused.
Accepting default is not ugly, especially if you don't even have a backbone connecting your sites. And even if we could argue over default's aesthetic qualities (which, honestly, I don't see how we can), there is no rational person who would consider it a hack.
You really should stop trying to correct the error you made in your first post. Remember the old adage about when you find yourself in a hole.
Another thing to note is the people who actually run multiple discrete network nodes posting here all said it was fine to use a single AS. One even said the additional overhead of managing multiple ASes would be more trouble than it is worth, and I have to agree with that statement. Put another way, there is objective, empirical evidence that it works.
In response, you have some nebulous "ugly" comment. I submit your argument is, at best, lacking sufficient definition to be considered useful.
accepting default has one important drawback for smaller networks: it causes loose urpf not to work for transit connections, and this causes remotely triggered blackhole discards not to work as expected. Depending on your requirements, this may or may not be a concern to worry about.
Nick
sigh, dumbass and blindingly wrong postings like this provide the evidence of why it's best to stay in the garden reading a good book on a warm sunday afternoon, rather than distractedly replying to the occasional email on nanog.
Nick