Patrick W. Gilmore wrote:
As a big ra-ra guy around peering, I thought this might be interesting, but I do not think I agree with the numbers.
I think you read this thinking I meant something I didn't mean; perhaps I should have used a different set of prepositions.
[deleted the discussion about utilization and CIRs, that is up to everyone to engineer and negotiate, my point shouldn't be so fragile as to require quoting spot pricing in markets @ various commits].
Assuming 10G PNI model: Assuming even nominal cross-connect fees of $100-$300/month per fiber pair, plus router port costs for each private peer (assuming you aren't at >10% utilization on the port) at a commercial exchange, you are eating a pretty significant cost per megabit you are actually moving. (plug in your numbers here). Assumption: Above 1Gb/s utilization, this makes sense or you are counting on growth.
Define "significant"?
If you are running << 1Gb/s per PNI it is "expensive".
Taking 1 Gbps (10%), and assuming even 20K per 10G port over 2 years, adding in $300/month for the x-conn, you are still looking at barely over $1/Mbps. If you have more than 10% utilization, that number goes down. Is that significant? Compared to what? Transit?
You have costs. Below 1Gb/s I'm stating they are "significant". The assumption above says it "automatically makes sense" above 1Gb/s or you are counting on growth with a starting point below 1Gb/s. The idea of presenting the PNI case was to avoid this sort of response. Obviously I didn't draw enough attention to the assumptions.
I would say a 75% price reduction is pretty significant. Plus you haven't considered CapEx cost for the transit ports.
Depends on the business. 75% may not be enough if a network's opex costs (PP&E) are high enough that this doesn't help. Again, I was trying to avoid painting the picture for any particular network, but more of the industry's interested parties as a whole.
Below 10% you would normally go to a paid peering fabric where you are paying cross connect + a flat port charge + router port for 1->N peers and hoping that enough utilization occurs that you get >10% utilization (to recover capex, opex, etc) and then whatever additional utilization you need to cover the flat port charge or you are counting on growth.
Here we agree. The port fee even on european IXes is measured in 1000s of $$ per month. And don't get me started on US or Japanese ports....
I was thinking of US ports, but modeling based on LINX pricing.
A "coop", best-effort switch fabric colo'd at a few sites would allow participants to peer off traffic at a price of the order of a single cross-connect (~$500/month per 10G port is possible, maybe less), private-VLANs all-around, or to only-mutually approved peers (e.g. via an automated web interface, prior art) to avoid many of the /old/ issues. No requirement for multi-lateral peering. You could peer, sell transit, buy transit, multicast, etc.
The way I figure it, it removes approximately an order of magnitude from the operational cost of peering with more than a handful of your largest single talkers. Especially as 100G LAN Ethernet becomes production before 100G WAN connections become commonplace. Economic theory (assuming that worked on the Internet) suggests this would allow for the increase in number of peers by approximately an order of magnitude (maybe more).
Sorry, I can't get there.
First, the "largest single talkers" would not be on a shared fabric, they'd be on dedicated ports, so this idea doesn't help.
I excluded "largest single talkers" by saying "more than a handful of your largest single talkers. Semantically, the assumption in PNI was that at 1Gb or above PNI makes sense [or you'd soon get there]." The question I asked was really related to the idea. If you have a few sensible PNIs, presumably you have enough traffic that you could conceivably have many potential peers at levels below the PNI case, but above the degenerate traffic case. (0, maybe 20mb/s, some number that isn't interesting enough to engineer for [or even consider] unless you have a nearly zero marginal cost to approach it)
For the medium to small guys, I think it's a great idea. Look at SIX, TorIX, PaNAP, etc. But shaving an _order of magnitude_ off? No, I don't see it. CapEx alone is more than 10% of your cost. (Well, unless you get Japanese IX ports or the most expensive US IX ports.)
Perhaps I'm lost or confused? Can someone help me understand?
You and Woodcock make a good point (re: SIX @ zero cost). However, ~20Gb/s aggregate is at most saving $80K/month between all participants for the additional traffic, which is pretty academic given the costs of operating networks of any sufficient scale [without looking at the constituency of the participants or the traffic]. If *each* network were saving $80K/month through the use of a few of these in multiple cities, that would be interesting to me.
I guess they would be more interesting deployed in Ashburn or some place similar because you could exclude the cost of "bringing" traffic to the exchange if the equipment (and bits) are already transported through that facility.
That said, I can't get with "Capex alone is more than 10% of your cost". I see 4 port Cisco WS-6704s with 4 XENPAKs on Ebay for like $3K/port, but hey, YMMV. There is no real reason to use deep buffers as an interface to a low-cost, low latency fabric, especially when you (and the fabric) can just add ports cheaply.
So I guess this is now meandering. I can present it differently.
Take the most rudimentary part of the SIX model, put it in a few cities where more traffic is exchanged then Seattle, bake, then taste. Wouldn't this be far more preferable (with scale) than the "expensive" US IX ports -- especially for new [rather than existing] traffic -- and as Woodcock mentions, anyone can run it, just requires some scale to be valuable enough; and more importantly, since the effective price of coop IX traffic would be lower than current major IX traffic, wouldn't this encourage more exchange to all participants benefit?
Or (back to my original post) are these costs essentially insignificant to the modern business case given the current set of market dynamics?
Thanks,
Deepak
