Peering Exchange Configurations

Mailman List Mirror (Read Only)
1

Hello All,

First, apologies for the change in email address.. my work account was getting a little busy so I've moved my lists to my Gmail account. But onward..

I'm interested in peering exchange design. We are not lucky enough to have access to a peering exchange so I have no direct experience. My questions are as follows:

1) Is a private AS typically used for the exchange side of the session?
2) Are RFC1918 IPs typically used for the p2p links into the exchange?
3) Do peering exchanges typically remove their AS from the path advertised to exchange participants?
     3a) If no: Do participants typically preference exchange-learned routes over other sources?
4) Do exchanges typically support the following address families?
       IPv4 Multicast
       IPv6 Unicast
       IPv6 Multicast

In exchanges where a route server is employed:
4) Do participants have a p2p link into a simple routing environment then multi-hop to a route server?
5) I see that Bird, OpenBDGd, and Quagga are all options for route server software. Does one of those packages stand out as the clear current choice for production peering exchanges?

I very much appreciate any responses.

2

1) Is a private AS typically used for the exchange side of the session?

No. Everybody uses his own AS number to establish sessions at peering
points.

2) Are RFC1918 IPs typically used for the p2p links into the exchange?

No. You usually get an IP address from the IX which pertains to the
IX's AS, and sits in a class that's specifically not announced to the
outside world.

3) Do peering exchanges typically remove their AS from the path
advertised to exchange participants?

This is the case that happens if you use a route server. Being at a
peering exchange point means you have the chance to sit on a switch
where other participants are directly connected. At this point you can
either establish direct peering relationships (configuring a session for
each peering agreement you get) or create a session with the local route
server, getting routes from all the other participants.

    3a) If no: Do participants typically preference exchange-learned
routes over other sources?

It depends. It's mostly a matter of economics more than a personal choice.

In exchanges where a route server is employed:
4) Do participants have a p2p link into a simple routing environment
then multi-hop to a route server?

The route server usually sits on the same LAN segment as the IXP
participants.

5) I see that Bird, OpenBDGd, and Quagga are all options for route
server software. Does one of those packages stand out as the clear
current choice for production peering exchanges?

I would say that OpenBGPd and BIRD are your best choices for this.
Quagga is getting better now, but suffered lots of problems with a high
number of peers in the past at many IXes. That's why many migrated
either to OpenBGPd, BIRD or both.

Ciao!

3

1) Is a private AS typically used for the exchange side of the session?

No. Also many exchange points do not run route servers at all, and expect participants to build bilateral BGP sessions directly between each other.

2) Are RFC1918 IPs typically used for the p2p links into the exchange?

No. Participants in an exchange typically number their exchange-facing interfaces out of a larger (non-p2p) subnet, e.g. an IPv4 /24 or /23, or an IPv6 /48 (or both).

3) Do peering exchanges typically remove their AS from the path advertised to exchange participants?

Some do, I hear. See above regarding route servers.

   3a) If no: Do participants typically preference exchange-learned routes over other sources?

Many people apply a higher LOCAL_PREF to routes learnt over an exchange in order to prefer cheap peering routes over more expensive transit routes. This is not universal, however. I know of networks who deliberately flatten LOCAL_PREF across peering and transit sessions in order to use different discriminators for exit selection (e.g. AS_PATH length).

4) Do exchanges typically support the following address families?
     IPv4 Multicast
     IPv6 Unicast
     IPv6 Multicast

I'm quite ignorant of multicast. IPv6 unicast peering is common.

In exchanges where a route server is employed:
4) Do participants have a p2p link into a simple routing environment then multi-hop to a route server?

In all exchange points I have seen where a route server was available, the route server appears on the shared fabric numbered just as any other participant would be.

5) I see that Bird, OpenBDGd, and Quagga are all options for route server software. Does one of those packages stand out as the clear current choice for production peering exchanges?

BIRD seems to be the choice du jour based on idle hallway chatter, but I have not compared them.

Joe

4

Re JOe,

jabley@hopcount.ca (Joe Abley) wrote:

> 1) Is a private AS typically used for the exchange side of the session?
No. Also many exchange points do not run route servers at all, and expect participants to build bilateral BGP sessions directly between each other.

...which is a shame. Routeservers in place gives you a nice benefit
upon hooking up to the exchange and before you have even found out
who is on the grid (anyone have a list for NOTA?).

Basically, the bigger exchange points (as in "european" mostly) all
have routeservers ready, but not everybody chooses to use them.

> 2) Are RFC1918 IPs typically used for the p2p links into the exchange?
No. Participants in an exchange typically number their exchange-facing interfaces out of a larger (non-p2p) subnet, e.g. an IPv4 /24 or /23, or an IPv6 /48 (or both).

Using RFC1918 for oft-traversed addresses is also not a good idea :wink:

> 3) Do peering exchanges typically remove their AS from the path advertised to exchange participants?
Some do, I hear. See above regarding route servers.

None of the routeservers I am peering with does insert their ASN.
On direct peering sessions, there is of course nobody in between.

> 4) Do exchanges typically support the following address families?
> IPv4 Multicast
> IPv6 Unicast
> IPv6 Multicast

I'm quite ignorant of multicast. IPv6 unicast peering is common.

Multicast is still seen as something special, sometimes even on
dedicated hardware, or on different VLANs. It's certainly possible,
but usually there are not so many participants...

> 5) I see that Bird, OpenBDGd, and Quagga are all options for route server software. Does one of those packages stand out as the clear current choice for production peering exchanges?

BIRD seems to be the choice du jour based on idle hallway chatter, but I have not compared them.

I was thinking "plat du jour"...and well, it's "du jour", so it can change
in an instant.

Cheers,
  Elmar.

5

1) Is a private AS typically used for the exchange side of the session?

No.

2) Are RFC1918 IPs typically used for the p2p links into the exchange?

No. In EU usually it is separate public /24, /23 or /22. The IPv6 range in RIPE region for exchanges is assigned from within special pool 2001:7f8::/32 (each IX gets /48).

3) Do peering exchanges typically remove their AS from the path
advertised to exchange participants?

The direct peering is between participants AS'es, there is nothing in between, including IX AS. Route-servers based on Cisco box put their AS number in between, but Quagga/Bird usually remove the IX as, however it may be configured per peer not to do so.

3a) If no: Do participants typically preference exchange-learned routes
over other sources?

Most do.

4) Do exchanges typically support the following address families?
IPv4 Multicast

no

IPv6 Unicast

Almost all.

IPv6 Multicast

No.

In exchanges where a route server is employed:
4) Do participants have a p2p link into a simple routing environment
then multi-hop to a route server?

Route-server is just like one of the members of the exchange. You get /23 or similar prefix of the exchange. You may have a BGP session with the route-server, but you are also free to have direct BGP sessions with other parties. Route-servers are mostly used by peers with open peering policies, but you still may steer your announcements basing on BGP communities.

5) I see that Bird, OpenBDGd, and Quagga are all options for route
server software. Does one of those packages stand out as the clear
current choice for production peering exchanges?

Quagga was used most often, but recently most biggest EU exchanges replaced it with Bird and it is much more stable.

6

I've never had a problem getting a participant list for NOTA from Terremark.

One down-side of route servers on a shared exchange fabric is that the layer-2 path through the exchange for the BGP sessions does not always match the layer-2 path through the exchange for traffic. This means that AS1 might continue to learn AS2's routes through the route server even though there's a layer-2 problem that prevents AS1 and AS2's peering routers from talking directly to each other. Hilarity may result.

I've never seen such a problem on small exchanges where the layer-2 fabric is simple, but I have seen it more than once on larger, more complicated exchanges.

My personal preference is to focus peering energy on bilats, and not to rely on a route server. But I understand the savings in opex that route servers can provide on busy exchanges.

Joe

7

1) Is a private AS typically used for the exchange side of the session?

Not in a typical public internet exchange. that said, there is no
reason why one could not build an exchange point that uses private ASNs.
One might do this for a specialised application of PPVPN peering
partners but that is beyond what you are asking.

2) Are RFC1918 IPs typically used for the p2p links into the exchange?

No. Public IXPs will provide an address space for its participants to
use. That said, once again, there might be some very specific
non-public applications of exchange points which may use RFC1918 address
space.

3) Do peering exchanges typically remove their AS from the path
advertised to exchange participants?

I'm assuming you are talking about routes exchanged via a route-server.
Most moderm deployments of RSes do act transparently but in the past
there have been cases where some exchange point participants did want to
see the RS' ASN in the ASPATH. Merit's old RSes had the capbility to
turn on or off transparency on a peer-by-peer basis. I am not sure
about modern RS implementations.

     3a) If no: Do participants typically preference exchange-learned
routes over other sources?

That is a matter of personal reference and religion. But in my
experience at larger exchanges with bigger players, the RSes' routes are
considered secondary and less preferred.

4) Do exchanges typically support the following address families?
       IPv4 Multicast
       IPv6 Unicast
       IPv6 Multicast

IPv4 multicast and IPv6 unicast seems standard in modern IXPs but I'm
not sure about IPv6 multicast.

In exchanges where a route server is employed:
4) Do participants have a p2p link into a simple routing environment
then multi-hop to a route server?

Not in modern exchange points. In some older ones this was the case but
I think that's largely historic.

5) I see that Bird, OpenBDGd, and Quagga are all options for route
server software. Does one of those packages stand out as the clear
current choice for production peering exchanges?

I wouldn't say clear but BIRD seems to be gaining ground on the other
two. There were some talk given at the last NANOG meeting in the route-
servers track...

http://www.nanog.org/meetings/nanog48/abstracts.php?pt=MTUxMyZuYW5vZzQ4&nm=nanog48

8

Some Research&Education type peering exchanges, like Pacific Wave http://www.pacificwave.net/ , support ipv4 multicast forwarding. As an exchange operator you'd want to support PIM-Snooping and the ability to disable DR-Flooding to control those flows just to the networks that joined them.

Chris

9

I operate the exchange point in the Kansas City area so I'll answer your
questions based on how we do it.

1) Is a private AS typically used for the exchange side of the session?

No. Each participant uses their own ASN.

2) Are RFC1918 IPs typically used for the p2p links into the exchange?

No. Exchanges typically have their own IPs assigned by their RIR and pass
them out to the members for connections to the exchange.

3) Do peering exchanges typically remove their AS from the path
advertised to exchange participants?

There is no peering directly with the exchange in a private link. In the
case of peering with a route server on the exchange then it is considered
best practices to do so.

     3a) If no: Do participants typically preference exchange-learned
routes over other sources?

Yes. As far as I know all our members set routes learned through the
exchange fabric higher than anything else. That's kind of the point as
exchange traffic is free so you always want to use it first.

4) Do exchanges typically support the following address families?
       IPv4 Multicast
       IPv6 Unicast
       IPv6 Multicast

No
Yes
No

In exchanges where a route server is employed:
4) Do participants have a p2p link into a simple routing environment
then multi-hop to a route server?

No. The route server is typically accessed like any other peer on the
fabric.

5) I see that Bird, OpenBDGd, and Quagga are all options for route
server software. Does one of those packages stand out as the clear
current choice for production peering exchanges?

We use Quagga. It's what we we're most familiar with and we haven't had any
issues.

I very much appreciate any responses.

No Problem. Feel free to stop by and check out our fabric for yourself.

Aaron

10

    3a) If no: Do participants typically preference exchange-learned
routes over other sources?

Yes. As far as I know all our members set routes learned through the
exchange fabric higher than anything else. That's kind of the point as
exchange traffic is free so you always want to use it first.

Actually, the order of preference is usually:

1. Private Interconnects (direct private peering)
2. Non-metered paid peering/transit
3. Exchange Points
4. Metered paid peering/transit

Owen

11

Where 'usually' here is rather nebulous.

I am not trying to say Owen is wrong, just don't think the way any network uses interconnectivity is somehow standard. Every network is different, and even similar links in the same network are different.

IXPs are standard ('usually' :), networks are not.

12

Where 'usually' here is rather nebulous.

I am not trying to say Owen is wrong, just don't think the way any network
uses interconnectivity is somehow standard. Every network is different, and
even similar links in the same network are different.

IXPs are standard ('usually' :), networks are not.