Internet Exchange Point(IXP) questions

Yaoqing_Joey_Liu · February 18, 2011, 2:03am

I'm doing some research on multiple origin AS problems of IXPs. As I know,
generally there are two types of IXPs
type 1: use exchange routers, which works in layer 3
type 2: use switches and Ethernet topology, which works in layer 2.
So I have a couple of qustions:
1. For type 1, the exchange routers may use several IP prefixes for routing,
how often does the IP prefixes have their own AS?
2. For type 2, all peers connected to the IXP must work in the same subnet
required by Ethernet rules. Is possible that the subnet IP prefixes belong
to some private IP address space, such as 192.168.x.x? How often does this
happen? If the subnet only contains public IP addresses, how are the
addresses announced?

Thanks,
Yaoqing

Michael_K_Smith · February 18, 2011, 2:17am

Hello:

On the Seattle Internet Exchange (SIX) we have ARIN-assigned addresses that we use on the Layer 2 fabric (your type 2 above). Hopefully the addresses aren't being announced at all, although we sometimes have to chase down people that announce it. Those addresses aren't the destination for any traffic, they are merely part of the transport to a destination, so there is no need for them to be in the DFZ.

Regards,

Mike

Leo_Bicknell1 · February 18, 2011, 2:31am

In a message written on Fri, Feb 18, 2011 at 02:17:48AM +0000, Michael K. Smith - Adhost wrote:

On the Seattle Internet Exchange (SIX) we have ARIN-assigned addresses that we use on the Layer 2 fabric (your type 2 above). Hopefully the addresses aren't being announced at all, although we sometimes have to chase down people that announce it. Those addresses aren't the destination for any traffic, they are merely part of the transport to a destination, so there is no need for them to be in the DFZ.

I've had to deal with exchanges like this in the past, and frankly
they have always been a pain for the support organization.

You see, customers use tools like mtr or Visual Traceroute that do
a traceroute and then continuously ping each hop. Many of these
customers don't have a default route, or default to their _other_
provider. These tools end up showing 100% loss at the exchange,
as they get the traceroute response and then can't ping it.

They then open a ticket, and your support organization has to explain
to them how all of this works and why it isn't the real cause of
their problem.

My preference is that the exchange get an ASN, peer with everyone
(e.g. from the route server) and announce the exchange prefix.
That way it's consistently announced. For exchange that don't do
this, I've always put the prefix into BGP in such a way that I will
announce it but only to my customers to work around this problem.

Please get your own ASN and announce the route, for the sake of all of
your members.

Bandy_Rush1 · February 18, 2011, 2:40am

On the Seattle Internet Exchange (SIX) we have ARIN-assigned
addresses that we use on the Layer 2 fabric (your type 2 above).
Hopefully the addresses aren't being announced at all, although we
sometimes have to chase down people that announce it.

I've had to deal with exchanges like this in the past, and frankly
they have always been a pain for the support organization.

You see, customers use tools like mtr or Visual Traceroute that do
a traceroute and then continuously ping each hop. Many of these
customers don't have a default route, or default to their _other_
provider. These tools end up showing 100% loss at the exchange,
as they get the traceroute response and then can't ping it.

They then open a ticket, and your support organization has to explain
to them how all of this works and why it isn't the real cause of
their problem.

<aol>

My preference is that the exchange get an ASN, peer with everyone
(e.g. from the route server) and announce the exchange prefix.

i do not like route servers or peering with strange things.

treat the exchange as an internal route and announce it within your
net and to your customer cone.

randy

Yaoqing_Joey_Liu · February 18, 2011, 3:04am

> From: Yaoqing(Joey) Liu [mailto:joey.liuyq@gmail.com]
> Sent: Thursday, February 17, 2011 6:03 PM
> To: nanog@nanog.org
> Subject: Internet Exchange Point(IXP) questions
>
> I'm doing some research on multiple origin AS problems of IXPs. As I
know,
> generally there are two types of IXPs
> type 1: use exchange routers, which works in layer 3
> type 2: use switches and Ethernet topology, which works in layer 2.
> So I have a couple of qustions:
> 1. For type 1, the exchange routers may use several IP prefixes for
routing,
> how often does the IP prefixes have their own AS?
> 2. For type 2, all peers connected to the IXP must work in the same
subnet
> required by Ethernet rules. Is possible that the subnet IP prefixes
belong
> to some private IP address space, such as 192.168.x.x? How often does
this
> happen? If the subnet only contains public IP addresses, how are the
> addresses announced?
>
> Thanks,
> Yaoqing

Hello:

On the Seattle Internet Exchange (SIX) we have ARIN-assigned addresses that
we use on the Layer 2 fabric (your type 2 above). Hopefully the addresses
aren't being announced at all, although we sometimes have to chase down
people that announce it. Those addresses aren't the destination for any
traffic, they are merely part of the transport to a destination, so there is
no need for them to be in the DFZ.

But I just checked the IXP prefix list, and found SIX owns prefix
206.81.80.0/23. And it has been announced by three ASNs, AS11537(Internet
2), AS3130(RGnet, LLC) and AS25973(Mzima Networks, Inc). I'm not sure if my
info is correct. Does SIX own its own ASN other than the three above?

Yaoqing

_Bill_Woodcock · February 18, 2011, 3:16am

As I know, generally there are two types of IXPs

This is incorrect.

type 1: use exchange routers, which works in layer 3

This is not an IXP. This is a router. That router would be owned by someone, who would have some sort of policy in the router, which would make it an Internet service provider, not an Internet exchange point.

type 2: use switches and Ethernet topology, which works in layer 2.

This is an IXP. Routers belonging to Internet service providers, communicating with each other across a switch fabric, which is an Internet exchange point.

1. For type 1, the exchange routers may use several IP prefixes for routing,
how often does the IP prefixes have their own AS?

Since this is not an IXP, I think the question is irrelevant to your research.

If an ISP wants to participate in BGP routing, and originate an IP prefix, that ISP must have an AS.

2. For type 2, all peers connected to the IXP must work in the same subnet
required by Ethernet rules.

Generally, yes, though some IXPs are not that prescriptive, and would allow a subset of the ISPs to peer on a different subnet if they wished.

Is possible that the subnet IP prefixes belong to some private IP address space, such as 192.168.x.x?

It is possible, but it does not follow best-practices, because it breaks traceroute and other diagnostic tools.

How often does this happen?

Very very rarely.

Only two IXPs out of more than three hundred are using FRC1918 space at this point: Maputo and Santiago de Compostela.

This used to be a more common mistake, but as communications with the operators of new IXPs has improved over time, it's become very rare.

If the subnet only contains public IP addresses, how are the addresses announced?

They are generally not announced. Occasionally they're announced by one or more participating ISPs at the IXP. Sometimes that's purposeful, other times it's accidental. Some IXPs have rules prohibiting the announcement of the exchange subnet, others actively seek out sources of transit for the exchange subnet.

                                -Bill Woodcock
                                 Research Director
                                 Packet Clearing House

Bandy_Rush1 · February 18, 2011, 3:24am

type 1: use exchange routers, which works in layer 3

This is not an IXP. This is a router. That router would be owned by
someone, who would have some sort of policy in the router, which would
make it an Internet service provider, not an Internet exchange point.

this from the guy who pushed "layer three exchange points" for years?
rofl!

_Bill_Woodcock · February 18, 2011, 5:17am

I was one of the people who built one in 1994, and used it quite happily for a few years, until it had outlasted its need.

Do you have something else in mind? Or are you just trying to keep your blood pressure up?

-Bill

Michael_K_Smith · February 18, 2011, 6:39pm

From: Yaoqing(Joey) Liu [mailto:joey.liuyq@gmail.com]
Sent: Thursday, February 17, 2011 6:03 PM
To: nanog@nanog.org<mailto:nanog@nanog.org>
Subject: Internet Exchange Point(IXP) questions

I'm doing some research on multiple origin AS problems of IXPs. As I know,
generally there are two types of IXPs
type 1: use exchange routers, which works in layer 3
type 2: use switches and Ethernet topology, which works in layer 2.
So I have a couple of qustions:
1. For type 1, the exchange routers may use several IP prefixes for routing,
how often does the IP prefixes have their own AS?
2. For type 2, all peers connected to the IXP must work in the same subnet
required by Ethernet rules. Is possible that the subnet IP prefixes belong
to some private IP address space, such as 192.168.x.x? How often does this
happen? If the subnet only contains public IP addresses, how are the
addresses announced?

Thanks,
Yaoqing

Hello:

On the Seattle Internet Exchange (SIX) we have ARIN-assigned addresses that we use on the Layer 2 fabric (your type 2 above). Hopefully the addresses aren't being announced at all, although we sometimes have to chase down people that announce it. Those addresses aren't the destination for any traffic, they are merely part of the transport to a destination, so there is no need for them to be in the DFZ.

But I just checked the IXP prefix list, and found SIX owns prefix 206.81.80.0/23<http://206.81.80.0/23>\. And it has been announced by three ASNs, AS11537(Internet 2), AS3130(RGnet, LLC) and AS25973(Mzima Networks, Inc). I'm not sure if my info is correct. Does SIX own its own ASN other than the three above?

Yaoqing

Christopher_Morrow · February 18, 2011, 7:34pm

why is it a good idea to send this to your customers? the next-hop
info is surely only useful to your local network? done right it's even
only relevant to the IX connected router, right? it seems wholely
unusful to your customers. (to me at least)

Michael_K_Smith · February 18, 2011, 9:24pm

I was thinking about what Leo said about tools that test each hop through a path. At least my downstream customers will be able to test through the SIX connection if I announce the /23 to them.

Mike

Joe_Abley2 · February 18, 2011, 9:28pm

Well, except for the reason that Leo mentioned.

The NEXT_HOP in the exchange point subnet will not make it to the customer router. It's not a transitive attribute. The customer will see a NEXT_HOP corresponding to the provider router (or whatever they decide to re-write it as). See RFC 4271 section 5.1.3.

Joe

Leo_Bicknell1 · February 18, 2011, 9:29pm

In a message written on Fri, Feb 18, 2011 at 02:34:21PM -0500, Christopher Morrow wrote:

why is it a good idea to send this to your customers? the next-hop
info is surely only useful to your local network? done right it's even
only relevant to the IX connected router, right? it seems wholely
unusful to your customers. (to me at least)

If by "done right" you mean perhaps a feature like returning ICMP's from
a loopback IP rather than the interface IP, there are two issues with
that:

The far end ISP controls this feature. If they don't enable it you must
work around by announcing the prefix to your customer. One person doing
it wrong at the exchange is enough that you have to work around it.

I at least find it useful when traceroute shows the interface. I
believe it saves time for your NOC, and burning IP's for interfaces
makes a lot of sense in terms of speeding troubleshooting. Even if all
of my gear allowed me to send ICMP's from the loopback it's quite likely
I would not use that feature.

Christopher_Morrow · February 18, 2011, 9:36pm

sorry, I was only talking|thinking about routing bits, I missed your
point about people being able to ping an IX interface... I'd submit
that in many networks the path to the nexthop may be a vastly
different one than the path to 'the broken thing' through the
isp/ixp/isp set of routers.

I meant: "Is the nexthop in your (the ixp connected isp) network the
IXP interface IP, or the loopback of your IXP connected router?"

'Done right' (I agree that this is an individual perspective) here
meant, to me, that the IXP prefix wasn't necessary in the IXP
connected ISP's network, reset to loopback in ibgp policy and never
send the IXP prefix (connected route) off the IXP connected router.

leaking the IX prefix to customers, to me, seems like a recipe for
much wider/unintended leakage

Christopher_Morrow · February 18, 2011, 9:37pm

hopefully the path to the IXP prefix is the same as to the item they
are testing failure of?

Leo_Bicknell1 · February 18, 2011, 9:44pm

In a message written on Fri, Feb 18, 2011 at 04:36:28PM -0500, Christopher Morrow wrote:

leaking the IX prefix to customers, to me, seems like a recipe for
much wider/unintended leakage

Oh, it is. I remember when MAE-EAST was injected by at least 50 people
into the DFZ because back then people weren't careful enough to just
send such things to customers.

AMS-IX (and others) have the better solution. They have AS1200,
announce the exchange LAN from AS1200 (195.69.144.0/22). They will
peer with you if you are at the exchange, see
AMS-IX Amsterdam. I believe, but can't find
a reference really quick that they get transit for it from a couple
of providers so those that don't peer still have the route.

I mean really, you have a block. If your IXP matters it's already
taking up space in all of the largest ISP's tables anyway, so there's no
"saving a route argument". Get an ASN, which since your multi-homed is
trivial, announce the block from there and peer with your exchange
participants. Everyone is happy, the route is consistent, and life is
good.

Leo_Bicknell1 · February 18, 2011, 9:46pm

In a message written on Fri, Feb 18, 2011 at 04:37:05PM -0500, Christopher Morrow wrote:

On Fri, Feb 18, 2011 at 4:24 PM, Michael K. Smith - Adhost
> I was thinking about what Leo said about tools that test each hop through a path. At least my downstream customers will be able to test through the SIX connection if I announce the /23 to them.

hopefully the path to the IXP prefix is the same as to the item they
are testing failure of?

Of course it isn't. Perhaps you missed my implication in the original
mail I wrote.

The customers cloging up your help desk with this sort of stuff are
idiots. Unfortunately that's where the majority of your helpdesk time
goes...

Christopher_Morrow · February 18, 2011, 9:48pm

i admit to missing it but yes, now with the explanation, I get your point

Martin_Pels2 · February 19, 2011, 6:26pm

Hi,