Peering and Network Cost

Hi,

As you all know, transit costs in the wholesale market today a few percent of what it did in 2000. I assume that most of that decline is due to a modified version of Moore's Law (I don't believe optics costs decline 50% every 18 months) and the advent of maverick players like Cogent that broker cozy oligopoly pricing.

But I also wondering whether the advent of widespread peering (promiscuous?) among the Tier 2 players (buy transit and peer) has played a role. In 2000 peering was still an exclusive club and in contrast today Tier 2 players often have hundreds of peers. Peering should reduce costs and also demand in the wholesale IP market. Supply increases and demand falls.

I thank you in advance for any insights.

Regards,

- R.

Roderick Beck
Sales Director/Europe and the Americas
Hibernia Networks

This e-mail and any attachments thereto is intended only for use by the addressee(s) named herein and may be proprietary and/or legally privileged. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this email, and any attachments thereto, without the prior written permission of the sender is strictly prohibited. If you receive this e-mail in error, please immediately telephone or e-mail the sender and permanently delete the original copy and any copy of this e-mail, and any printout thereof. All documents, contracts or agreements referred or attached to this e-mail are SUBJECT TO CONTRACT. The contents of an attachment to this e-mail may contain software viruses that could damage your own computer system. While Hibernia Networks has taken every reasonable precaution to minimize this risk, we cannot accept liability for any damage that you sustain as a result of software viruses. You should carry out your own virus checks before opening any attachment.

Hi Roderick,

transit cost is lowering close to peering cost, so it is doubghtful
economy on small channels. If you don't live in
Amsterdam/Frankfurt/London - add the DWDM cost from you to one of major
IX. That's the magic.

In large scale peering is still efficient. It is efficient on local
traffic which is often huge.

Hi,

As you all know, transit costs in the wholesale market today a few percent of what it did in 2000. I assume that most of that decline is due to a modified version of Moore's Law (I don't believe optics costs decline 50% every 18 months) and the advent of maverick players like Cogent that broker cozy oligopoly pricing.

But I also wondering whether the advent of widespread peering (promiscuous?) among the Tier 2 players (buy transit and peer) has played a role. In 2000 peering was still an exclusive club and in contrast today Tier 2 players often have hundreds of peers. Peering should reduce costs and also demand in the wholesale IP market. Supply increases and demand falls.

I thank you in advance for any insights.

Regards,

- R.

Roderick Beck
Sales Director/Europe and the Americas
Hibernia Networks

This e-mail and any attachments thereto is intended only for use by the addressee(s) named herein and may be proprietary and/or legally privileged. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this email, and any attachments thereto, without the prior written permission of the sender is strictly prohibited. If you receive this e-mail in error, please immediately telephone or e-mail the sender and permanently delete the original copy and any copy of this e-mail, and any printout thereof. All documents, contracts or agreements referred or attached to this e-mail are SUBJECT TO CONTRACT. The contents of an attachment to this e-mail may contain software viruses that could damage your own computer system. While Hibernia Networks has taken every reasonable precaution to minimize this risk, we cannot accept liability for any damage that you sustain as a result of software viruses. You should carry out your

own virus checks before opening any attachment.

It also depends on traffic makeup. Huge amounts of eyeball traffic go to (well, come from) NetFlix (a third) and Google, FaceBook, Hulu, Amazon, etc. (another third). It's comparable price to peer off those few huge sources of traffic and buy better transit than you would have than to just buy cheap transit.

A lot of people tend to forget there are thousands of independent ISPs out there, usually in areas where there aren't a breadth of providers in the first place. Most could get buy with a single GigE (or even less).

Not actually Facebook net, but Akamai CDN. Not a Google (peer), but GCC
node :wink:

It is varying from location to location. For example here in Ukraine we
(still) have 1st place for traffic amount from Vkontakte (mostly music
streams), second from EX.ua (movie store), but almost none NetFlix, Hulu
or Amazon. And you can't get both of them in a good quality neither at
IXes, nor at Tier1.

I think in another locations, for example in India, traffic profile will
be different from both of us, and have some local specific as well.

Very true. I left it as I did given that I expect a similar profile from others in North America... on NANOG.

Basically, wherever your region's streaming video or application updates come from. :wink:

(Reply to thread, not necessarily myself.)

If you can pull a third of your traffic off at the cost of a cross connect and another third at the cost of an IX port, now you can spend a buck or two a meg on what's left. Yes, I understand the cost of a cross connect or IX port is the $/megabit you're actually using and not $/megabit of capacity.

Most cost models select a capacity figure that represents typical high-watermark utilization before the next cash outlay is triggered. By using your actual utilization, you might be penalizing your cost if you have low utilization and that low utilization is expected to be a temporary situation given the state of your business. That way your cost doesn't increase (for example) as a function of losing a large customer or other traffic shifting event. The only reason I would see some intentionally pick a lower figure is if the dynamic of their specific business suggests that low-utilization interconnect ports are typical for them.

Hi,

As you all know, transit costs in the wholesale market today a few
percent of what it did in 2000. I assume that most of that decline is
due to a modified version of Moore's Law (I don't believe optics
costs decline 50% every 18 months) and the advent of maverick players
like Cogent that broker cozy oligopoly pricing.

But I also wondering whether the advent of widespread peering
(promiscuous?) among the Tier 2 players (buy transit and peer) has
played a role. In 2000 peering was still an exclusive club and in
contrast today Tier 2 players often have hundreds of peers. Peering
should reduce costs and also demand in the wholesale IP market.
Supply increases and demand falls.

You might find https://www.nanog.org/meetings/nanog53/presentations/Tuesday/valancius.pdf and the concomitant http://conferences.sigcomm.org/sigcomm/2011/papers/sigcomm/p194.pdf interesting.

-Scot

Transit cost is down but IX cost remains the same. Therefore IX is longer
cost effective for a small ISP.

As an (non US) example, here in Copenhagen, Denmark we have two internet
exchanges DIX and Netnod. We also have many major transit providers,
including Hurricane Electric and Cogent.

Netnod price for a 1 Gbps port is 40000 SEK = 4500 USD / year
http://www.netnod.se/ix/join/prices. DIX is 40000 DKK = 5700 USD / year
http://dix.dk/serviceinformation/

HE.net is offering 1 Gbps flatrate for 450 USD / month list price = 5400
USD /year.

Cogent can match that.

So why would a small ISP pay 4500 USD for a service with no guarantee of
how much traffic they will be able to peer away?

You need to get a 10 Gbps port and be able to peer at least 2-3 Gbps before
it is even break even with the deals you get from the transit providers.
But then you will notice that all the traffic is only with a few peers and
you can just peer directly with those and skip the middleman.

Regards,

Baldur

Even in the three cities you mentioned peering on small scale is usually not cheaper at all or only very little.

Please keep in mind that some companies peer despite it offers no savings for them and at the end of the day it might be even more expensive. They do it because of performance and reliability reasons.

https://www.youtube.com/watch?v=Y43Fy4oU2XE

There are reasons to peer other than cost reduction.

* Baldur Norddahl <baldur.norddahl@gmail.com>

Transit cost is down but IX cost remains the same. Therefore IX is longer
cost effective for a small ISP.

As an (non US) example, here in Copenhagen, Denmark we have two internet
exchanges DIX and Netnod. We also have many major transit providers,
including Hurricane Electric and Cogent.

Netnod price for a 1 Gbps port is 40000 SEK = 4500 USD / year
Netnod IX pricing | Netnod. DIX is 40000 DKK = 5700 USD / year
http://dix.dk/serviceinformation/

HE.net is offering 1 Gbps flatrate for 450 USD / month list price = 5400
USD /year.

Cogent can match that.

So why would a small ISP pay 4500 USD for a service with no guarantee of
how much traffic they will be able to peer away?

We're in a similar situation here; transit prices has come down so much
in recent years (while IX fees are indeed stagnant) that I am certain
that if I were to cut all peering and buy everything from a regional
tier-2 instead, I'd be lowering my total MRC somewhat, without really
reducing connectivity quality to my (former) peers.

For us, the primary reason that keeps us peering is DDoS prevention. Our
traffic is mostly regional, so if a customer of mine gets hit with a
volumetric DDoS attack that would saturate my IP transit lines and cause
collateral damage, that's no big deal as we can just RTBH the customers
prefix towards our transit providers. The customer is only mildly
inconvenienced by this as, say, 90% of his traffic goes to our peers.
Without peering the attack would succeed because my RTBH would
completely offline my customer.

Tore

Even though you get more bandwidth at an exchange point for the price
you pay a transit provider for less bandwidth, the value you extract
from the more cost-efficient peering port is directly proportional to
how much peering you can get off of it.

If you are unable to offload enough of your traffic on to peering that
an incremental transit service cost can justify, you're likely better
off with a transit provider if budget is your constraint.

Mark.

And also to reduce AS hops. If you and your competition are pushing BGP
routes to your multi-homed customers, the customers are "more likely" to
choose paths with fewer AS hops. Traffic to you means $$ to you. In the
long run, it "could" pay off, or at the very least, get you more customers.

Mark.

I wouldn't say exchange point prices are stagnant, per se. They may
remain the same, but what goes up is the port bandwidth. It's not
directly linear, but you get my point.

Again, the burden is on the peering members to extract the most out of
their peering links by having as much peering as possible. Route servers
at the exchange points have played a huge role in facilitating this, but
the final stretch involves getting in touch with a bunch of members to
setup bi-lateral sessions, with no guarantee they will agree, or if they
do, may not peer their entire network, not taking into account whether
they will do this for free or not.

Perhaps the next bunch of exchange point operators will be those who
play a more active role in facilitating peering across all members, so
as to keep traffic on their switch fabric and away from transit
providers. But that is probably a zero-sum game, as their involvement
will just mean more salaries that need to get paid, leading to an
increase in membership fees.

Mark.

* Mark Tinka <mark.tinka@seacom.mu>

> We're in a similar situation here; transit prices has come down so
> much in recent years (while IX fees are indeed stagnant) that I am
> certain that if I were to cut all peering and buy everything from a
> regional tier-2 instead, I'd be lowering my total MRC somewhat,
> without really reducing connectivity quality to my (former) peers.

I wouldn't say exchange point prices are stagnant, per se. They may
remain the same, but what goes up is the port bandwidth. It's not
directly linear, but you get my point.

Again, the burden is on the peering members to extract the most out of
their peering links by having as much peering as possible.

You appear to be assuming that an IP transit port is more expensive
then an IXP port with the same speed. That doesn't seem to always be
the case anymore, at least not in all parts of the world, and I expect
this trend to continue - transit prices seems to go down almost on a
monthly basis, while the price lists of the two closest IXPs to where
I'm sitting are dated 2011 and 2013, respectively.

Even if the transit port itself remains slightly more expensive than
the IXP port like in the example Baldur showed, the no-peering
alternative might still be cheaper overall because even if you're
peering most of your traffic you'll still need to pay a nonzero amount
for a (smaller or less utilised) transit port anyway.

Tore

IX port pricing in the Chicago area has plummeted in the past two or three years. It's gone down... maybe 2/3.

Pricing at LINX here in the UK has definitely dropped over the past few years.

Back in 2011, the membership fee was £1500/year and it's now £1200/year.

1G ports were £391/month on the first London LAN and £335/month on the second London LAN. They're now free on both LANs for the first port and then £270/month and £180/month respectively for additional ports.
You can also get a free 1G port on each of the Manchester UK, Cardiff UK, Edinburgh UK and North Virginia/Washington DC USA LANs as part of the same membership fee (none of these additional LANs existed in 2011).

10G ports were £1463/month on the first London LAN and £1250/month on the second London LAN. They're now £1030/month and £785/month respectively.

So that's what, a 20% reduction in membership fees and a 30% or higher (depending on the service) reduction in port fees in 4 years?

I don't have any quantifiable data on what has happened to IP transit costs over the same period, but for a point comparison I'd say that off the top of my head you can get a 1G CDR on a 10G port from a tier-1 provider in London for approximately the same cost as a 10G port at LINX these days, maybe slightly cheaper.

Edward Dore
Freethought Internet

You appear to be assuming that an IP transit port is more expensive
then an IXP port with the same speed. That doesn't seem to always be
the case anymore, at least not in all parts of the world, and I expect
this trend to continue - transit prices seems to go down almost on a
monthly basis, while the price lists of the two closest IXPs to where
I'm sitting are dated 2011 and 2013, respectively.

Agreed.

Even if the transit port itself remains slightly more expensive than
the IXP port like in the example Baldur showed, the no-peering
alternative might still be cheaper overall because even if you're
peering most of your traffic you'll still need to pay a nonzero amount
for a (smaller or less utilised) transit port anyway.

Agreed again.

Mark.

If you have so much difference in price of IX connectivity (in general,
including cabling, DWDM to one of major IX, colo, etc) - this only mean
you should have a long talk with your current IP transit sales. Or just
change it to another one.