IXP

  I'll get off my soap-box now and let you resume your observations that
  complexity as a goal in and of itself is the olny path forward. What
  a dismal world-view.

No-one is arguing that complexity is a goal. Opportunities to
introduce gratuitous complexity abound, and defending against them
while recognizing that the opportunity that represents genuine
progress (trading outhouses for indoor plumbing, for example) is quite
a challenge.

I'm all for using the cleanest, simplest, and most reliable means to
meet requirements. Not all IXPs have the same requirements driving
their business, though - an IXP that operates a distributed metro-area
fabric has additional concerns for reliability and cost-efficient use
of resources than an IXP that operates a single switch. If
requirements were such that I needed to buy and *use* a partial mesh
topology for a distributed IXP fabric in the most reliable fashion
possible, I'd much rather go the route described earlier than try to
cobble something together with PVST/MST L2 technologies, but that's
just me.

You can assert that the status quo gives you solid predictable
performance, but the reality is that you occasionally get sucked into
a vortex of operational issues arising from L2's failure modes. To
continue with my bad plumbing analogy, open sewers were a reliable
means of moving waste material, easy to see when they were failing,
but occasionally produced outbreaks of disease. Are open sewers still
in use in the world today? You bet.

The underlying hardware layer that IXPs use is capable of more than
IXPs use. Whether to turn on those features is driven by requirements,
from customers and from the economics of the business. I would argue,
though, that at today's level of robustness and penetration of the
technologies that we've been discussing, the customer "requirement" to
peer on a shared VLAN is much more about complacency than avoiding
risk (as you seem to be arguing).

When we were turning PAIX from a private interconnect location into a
public IXP, we questioned every assumption about what role IXPs played
in order to ensure that we weren't making decisions simply to preserve
the status quo. One of the things we questioned was whether to offer a
peering fabric at all, or simply rely on PNIs. Obviously we opted to
have a peering fabric, and I don't regret the decision despite the
many long nights dealing with migration from FDDI to Ethernet (and the
fun of translational bridge MTU-related issues during the migration),
and the failure modes of Ethernet L2 - so your assertion that Ethernet
L2 provides solid predictable performance needs to be modified with
"mostly". I'll counter with an assertion that some L2.5/L3 networks
are built and operated to more 9s than some IXP L2 networks that span
multiple chassis. Whether that additional reliability makes business
sense to offer, though, is a different question.

If lack of complexity was a *requirement* that trumped all others,
there would still be a DELNI at PAIX.

Stephen, that's a straw-man argument. Nobody's arguing against VLANs. Paul's argument was that VLANs rendered shared subnets obsolete, and everybody else has been rebutting that. Not saying that VLANs shouldn't be used.

"Bill Woodcock" <woody@pch.net> writes:

... Nobody's arguing against VLANs. Paul's argument was that VLANs
rendered shared subnets obsolete, and everybody else has been rebutting
that. Not saying that VLANs shouldn't be used.

i think i saw several folks, not just stephen, say virtual wire was how
they'd do an IXP today if they had to start from scratch. i know that
for many here, starting from scratch isn't a reachable worldview, and so
i've tagged most of the defenses of shared subnets with that caveat. the
question i was answering was from someone starting from scratch, and when
starting an IXP from scratch, a shared subnet would be just crazy talk.

In a message written on Fri, Apr 24, 2009 at 01:48:28AM +0000, Paul Vixie wrote:

i think i saw several folks, not just stephen, say virtual wire was how
they'd do an IXP today if they had to start from scratch. i know that
for many here, starting from scratch isn't a reachable worldview, and so
i've tagged most of the defenses of shared subnets with that caveat. the
question i was answering was from someone starting from scratch, and when
starting an IXP from scratch, a shared subnet would be just crazy talk.

I disagree.

Having no shared subnet renders an exchange switching platform
useless to me. If I have to go to all the work of configuring both
ends in a exchange point operator provisioning system (and undoubtly
being billed for it), assigning a /30, and configuring an interface
on my router then I will follow that procedure and order a hunk of
fiber. Less points of failure, don't have to deal with how the
exchange operator runs their switch, and I get the bonus of no
shared port issues.

The value of an exchange switch is the shared vlan. I could see
an argument that switching is no longer necessary; but I can see
no rational argument to both go through all the hassles of per-peer
setup and get all the drawbacks of a shared switch. Even exchanges
that took the small step of IPv4 and IPv6 on separate VLAN's have
diminished value to me, it makes no sense.

It's the technological equvilient of bringing everyone into a
conference room and then having them use their cell phones to call
each other and talk across the table. Why are you all in the same
room if you don't want a shared medium?

Because you don't want to listen to what others have to say to you.

Adrian
(The above statement has network operational relevance at an IP
level.)

Leo Bicknell wrote:

The value of an exchange switch is the shared vlan. I could see
an argument that switching is no longer necessary; but I can see
no rational argument to both go through all the hassles of per-peer
setup and get all the drawbacks of a shared switch. Even exchanges
that took the small step of IPv4 and IPv6 on separate VLAN's have
diminished value to me, it makes no sense.

Cost. Shared port/ports versus port per peer, no physical cross connects to be made for each new peer. For a medium sized network, an IXP can provide cheap connectivity to many peers saving on transit costs.

I'll admit, my knowledge is limited given I exist in the non-existent Oklahoma infrastructure, but I count the days (years?) until I can afford to light a 10Gb ring down to Dallas and hopefully minimize the number of ports and size of hardware I need down there to interconnect my ring (and thus me) to everyone else. Hopefully with as few physical interconnects as possible, as my Junipers ports are expensive for my size. I'll never be transit free, but perhaps I can get peering through an IXP and save some transit costs.

Jack

I like to disagree here, Paul.

Best regards,
Arnold

Leo Bicknell wrote:

In a message written on Fri, Apr 24, 2009 at 01:48:28AM +0000, Paul Vixie wrote:

i think i saw several folks, not just stephen, say virtual wire was how
they'd do an IXP today if they had to start from scratch. i know that
for many here, starting from scratch isn't a reachable worldview, and so
i've tagged most of the defenses of shared subnets with that caveat. the
question i was answering was from someone starting from scratch, and when
starting an IXP from scratch, a shared subnet would be just crazy talk.

I disagree.

Having no shared subnet renders an exchange switching platform
useless to me. If I have to go to all the work of configuring both
ends in a exchange point operator provisioning system (and undoubtly
being billed for it), assigning a /30, and configuring an interface
on my router then I will follow that procedure and order a hunk of
fiber. Less points of failure, don't have to deal with how the
exchange operator runs their switch, and I get the bonus of no
shared port issues.

The value of an exchange switch is the shared vlan. I could see
an argument that switching is no longer necessary; but I can see
no rational argument to both go through all the hassles of per-peer
setup and get all the drawbacks of a shared switch. Even exchanges
that took the small step of IPv4 and IPv6 on separate VLAN's have
diminished value to me, it makes no sense.

It's the technological equvilient of bringing everyone into a
conference room and then having them use their cell phones to call
each other and talk across the table. Why are you all in the same
room if you don't want a shared medium?

I second that.

We got to go through all the badness that was the ATM NAPs (AADS, PacBell NAP, MAE-WEST ATM).

I think exactly for the reason Leo mentions they failed. That is, it didn't even require people to figure out all the technical reasons they were bad (many), they were fundamentally doomed due to increasing the difficulty of peering which translated to an economic scaling problem.

i.e. if you make it hard for people to peer then you end up with less peers and shared vlan exchanges based on things like ethernet outcompete you.

Been there done that.

We've already experienced the result of secure ID cards and the PeerMaker tool. It was like pulling teeth to get sessions setup, and most peers plus the exchange operator didn't believe in oversubscription (can you say CBR? I knew you could), so you end up with 2 year old bandwidth allocations cast in stone because it was such a pain to get the peer to set it up in the first place, and to increase bandwidth to you means your peer has to reduce the bandwidth they allocated to somebody else.

Mike.