Hi there,
as 2-byte ASNs are close to depletion (see NRO announcement this week),
we have come across a topic, that might influence the adoption of 4-byte
ASNs. First of all, we have no data or experience about 4-byte ASN
adoption and are therefore unable to evaluate, if we should rush for the
last available numbers.
So here's the thing: IXPs usually implement N:M filtering based on
standard community strings. As standard BGP communities support only 4
bytes, this only works for IXPs with 2-byte ASNs and peers with 2-byte
ASNs.
Extended communities to the rescue? We are not sure: While RFC4260
(Extended Communities) allows for <Global Admin,2bytes>:<Local Admin,
4bytes> community strings (3.1 Two-Octet AS Specific Extended
Community), this works as long as the IXP itself uses a 2-byte ASN.
What happens, if the IXP uses a 4-byte ASN? RFC5668 (4-Octet AS Specific
BGP Extended Community) defines <Global Admin,4bytes>:<Local Admin,
2bytes>.
I have been asking some IXP operators, about their practice and their
reply was "4-byte ASNs are supported by our RS". What's your experience?
Did you see IXPs, that do not support them? Do you think, the IXP
operators are aware of this limitation? Have you seen IXPs with 4-byte
ASNs or do RIRs reserve 2-byte ASNs for future IXPs? What other
operational problems did you experience while using 4-byte ASNs?
A lot of questions. I am very curious about your answers.
Cheers,
Sebastian
What happens, if the IXP uses a 4-byte ASN? RFC5668 (4-Octet AS Specific
BGP Extended Community) defines <Global Admin,4bytes>:<Local Admin,
2bytes>.
I have been asking some IXP operators, about their practice and their
reply was "4-byte ASNs are supported by our RS". What's your experience?
Did you see IXPs, that do not support them? Do you think, the IXP
operators are aware of this limitation? Have you seen IXPs with 4-byte
ASNs or do RIRs reserve 2-byte ASNs for future IXPs? What other
operational problems did you experience while using 4-byte ASNs?
Do you see an issue with future IXP or future IXP members ?
You can see at this list of members of one IXP that there are both 2-byte
and 4-byte ASNs living in harmony, with a large number of 4-byte ones:
http://ptt.br/particip/sp
The IXP itself is using a 2-byte public ASN, so if at some point
communities are used (although there are route-servers capable of granular
policy without communities), they could use standard 2-byte community
format.
And with a bit coordination, 2-byte private ASNs and communities could be
used to overcome limitations in member routers support of 4-byte ASNs.
Rubens
telnet lg.sp.ptt.br
Trying 200.160.1.3...
Connected to lg.sp.ptt.br.
Escape character is '^]'.
Dear Sebastian,
So here's the thing: IXPs usually implement N:M filtering based on
standard community strings. As standard BGP communities support only 4
bytes, this only works for IXPs with 2-byte ASNs and peers with 2-byte
ASNs.
There are various directions in which a workable solution may for IXP
Operators:
- Use a mapping table: 32 bit values to 16 bit values. Every
participant with a 4-byte ASN is represented with 2-bytes, there
are pro's and con's to this approach.
- Use an out-of-band mechanism which allows IXP participants to
signal the desired routing policy to the Route Server. (VIX.at
offers a web-interface where people can click and point to which
ASNs they want to export or not).
Another approach would be to signal the desired routing policy through
RPSL. I've been working on some extensions which would allow an IXP
participant to publish what the Route Server should do on their behalf,
in this example 6777 is a Route Server and AS4247483647 is an IXP
participant with a 4-byte ASN:
import-via: AS6777 from AS4247483647 accept AS4247483647
export-via: AS6777 to AS4247483647 announce AS-ATRATO
(read more here: draft-snijders-rpsl-via-03 )
Support for these extensions will be available in the next release of
the RIPE Whois Database. After that IXP Operators can consider
implementing support for RPSL-via.
All of the above approaches have disadvantages:
- Mapping tables add complexity
- Most Out-of-band methods would differ from each other
- RPSL-via is has yet to be implemented in the wild
Extended communities to the rescue?
Nope. Not as it stands today.
I have been asking some IXP operators, about their practice and their
reply was "4-byte ASNs are supported by our RS". What's your experience?
Most of them are lying through their teeth. 
Kind regards,
Job
I have over 100,000 servers located in routing diverse datacenters with
4byte ASN numbers and have not had 1 problem or complaint related to the
ASN for not able to communicate with the datacenter. The first 1 did make
me really nervous for all of the reasons already mentioned but turned out
to be a non-issue.
While it would be nice to see community string support increase to 4byte, I
think this is more of an educational challenge for the IXPs on how to setup
your community strings to work and not really a technical problem.
Bryan
I have over 100,000 servers located in routing diverse datacenters
with 4byte ASN numbers and have not had 1 problem or complaint related
to the ASN for not able to communicate with the datacenter. The first
1 did make me really nervous for all of the reasons already mentioned
but turned out to be a non-issue.
This thread is not about reachability of prefixes announced by 4-byte
ASNs. This thread is about prefix filtering on Route Servers at Internet
Exchanges.
While it would be nice to see community string support increase to 4byte, I
think this is more of an educational challenge for the IXPs on how to setup
your community strings to work and not really a technical problem.
Can you elaborate on how you would setup 'community strings'?
Kind regards,
Job
Re-reading, I was thinking of someone connecting to an IXP, not a new IXP
needing a 2Byte. This is an interesting situation and you are correct,
my comment was off topic.
*Bryan Socha*
Network Engineer
646.450.0472 | *bryan@serverstack.com <bryan@serverstack.com>*
*ServerStack* | Scale Big
Sorry for not mentioning the beef: Extended Communities effectively
leave 6 bytes to the user. So, it is not possible, that a 4-byte-ASN IXP
encodes its own 4-byte-ASN and a 4-byte-ASN customer in one extended
community string. This is needed, as the IXP RS usually interpret their
own ASN and a peer ASN as: "Send this prefix out to the peer ASN".
To make things worse: even if the IXPs ASN is 2-byte, I would assume,
that RS implementors chose to interpret extended community strings as
always being in the format 4-byte:2-byte (see RFC5668).
Best regards,
Sebastian
some ixp operators (e.g. me) are rather enthusiastic about the idea of a
modified form of draft-raszuk-wide-bgp-communities getting more traction.
This would solve this particular problem and many others.
Nick
2-byte ASN depletion - the other white meat....
To make things worse: even if the IXPs ASN is 2-byte, I would assume,
that RS implementors chose to interpret extended community strings as
always being in the format 4-byte:2-byte (see RFC5668).
some ixp operators (e.g. me) are rather enthusiastic about the idea of a
modified form of draft-raszuk-wide-bgp-communities getting more traction.
This would solve this particular problem and many others.
Wide communities is the wrong tool here. You want this:
-- Jeff
Martin,
> Wide communities is the wrong tool here. You want this:
> draft-ietf-idr-as4octet-extcomm-generic-subtype-06
This draft does not cater for the use case of describing a 32-bit ASN peering
with a 32-bit route server, which would require a 4-byte Global Administrator
as well as a 4-byte Local Administrator sub-field.
I think that's the first clear articulation I've read about why some people
want wide comms vs. a simple replacement for existing regular communities as
extended communities. Thanks.
Wide comms can do that, but they're intended to be a somewhat bigger hammer.
This case is probably worth chatting with the authors and others in IDR at
IETF to see if we should do something simpler.
-- Jeff
I suspect the operator confusion is that’s how they’ve been using 16-bit ASNs
all along, so how did the IETF end up with something different.
http://www.onesc.net/communities/ is a fairly comprehensive list of how they are used today.
- jared
Thanks for the list. Browsing the first several entries somewhat supports
the reason why I'm acting "surprised". While there are some cases where the
right hand side of the community is an AS number, a significant amount of it
was either an arbitrary value or something more structured.
The generic 4-byte draft I'm part of is intended to be "low hanging fruit".
We don't need to deploy a new attribute, the format specifier is already
present in code. All that was needed was a code point to say "go use this
like existing RFC 1997 comms".
The wide comms draft (and flex comms, where some of the ideas were pulled in
from) was intended to address the messier case where the meaning of a
community was already structured. To pick on one of the items in the list:
http://www.onesc.net/communities/as209/
Coding these using regexes is a PITA, both as an implementor of the
underlying policy and as a sender who has to remember what the magic value
means. Ideally the operator should end up with something simple:
Tell AS209, Do not announce to AS foo. Prepend N times to PST peers. Etc.
Right now, these things are magic values.
The last few rounds of wide comms were attempts to get encoding formats in
place that accommodated some amount of grouping logic
(peer-class CUSTOMER & North America, e.g.). We did manage to work out a
way to do that encoding correctly. But it turned out that the real killer
was transitive manipulation - you can't meddle with such a thing without
breaking logic. So, back to a simpler drawing board.
An update to the wide-comm draft should be out by end of this week. We'd
welcome some constructive criticism on it.
-- Jeff
When possible (e.g.: here at AS2914) we have used things like this:
65500:nnn do not announce to peer
where the NNN is the peer ASN. Using such literal numbering is easier for
the humans involved. The ability to see what route is learned from specific ASN
is also helpful, as things like AS_PATH are just a bit-string that can be arbitrarily
set and sent by a peer.
I will try to keep my eye open for the draft.
(perhaps see you in Atlanta or London).
- Jared
Food for thought:
- ASNs can be reused at different locations by IXPs, barring perhaps
certain business or administrative reasons. Ask Equinix.
- For IXPs that already have 16-bit ASNs for route servers, this saves additional
allocations from RIRs and mitigates concerns for the IXP getting potentially a 32-bit ASN,
thus having trouble with BGP communities as described. Having a single ASN
may raise other issues but it is an option.
- I believe it would help if RIRs reserved 16-bit ASNs
in addition to IPv4 micro allocations for IXPs, until a formal
solution can be finally found about the 32-bit ASN BGP communities issue.
Aris Lambrianidis