routing around Sprint's depeering damage

There are at least three ways to address this Sprint / Cogent partition:

1. Send Vint Cerf back up to Capitol Hill with a doomsday
scenario of what would happen to the economy if anyone else
gets as stupid as Sprint has been, begging for laws that any
tier-1 or tier-2 who wants to de-peer needs to provide all their
customers and peers with 90 day notice or face stiff fines.
Send John Schnizlein along with him to get the House
Communications Director an Akamai hosting account.
Repeat the "eyeballs-or-data, which is more valuable" mantra
whether or not there are still forty Republicans in the Senate.

2. Pick up some more fiber, dust off the router manuals, and
allow and recommend that tier-1s transit any third party
tier-1-to-tier-1 traffic.

3. Both.

Which is the best way?

Dave Blaine wrote:

Which is the best way?

More regs and more laws is certainly not in the running.

How about: If there is a need, somebody will provide at a suitable price? If no body steps up, we don't need it.

Dave Blaine wrote:

There are at least three ways to address this Sprint / Cogent partition

I'd be fairly reluctant to allow the government to get involved in peering relationships too deeply. Australia has some very wierd consquences of our government doing so almost ten years on. One of those is which is that the "Gang of Four" have effectively set a floor price on domestic transit that is much higher than it should be - meaning that much content is delivered to us from overseas because the cost of delivering in Australia to those networks is too high to do so economically. Even a lot of Australian content is hosted overseas for this reason.

Consider that this is in a land where the broadband providers don't have to deliver unlimited for a fixed price.

(Would things have been different without this government directive? Hmm. Dunno. Feel free to discuss).

Dave Blaine wrote:

There are at least three ways to address this Sprint / Cogent partition:

1. Send Vint Cerf back up to Capitol Hill with a doomsday
scenario of what would happen to the economy if anyone else
gets as stupid as Sprint has been, begging for laws that any
tier-1 or tier-2 who wants to de-peer needs to provide all their
customers and peers with 90 day notice or face stiff fines.
Send John Schnizlein along with him to get the House
Communications Director an Akamai hosting account.
Repeat the "eyeballs-or-data, which is more valuable" mantra
whether or not there are still forty Republicans in the Senate.

2. Pick up some more fiber, dust off the router manuals, and
allow and recommend that tier-1s transit any third party
tier-1-to-tier-1 traffic.

3. Both.

Which is the best way?

4. Multihome.

~Seth

More regs and more laws is certainly not in the running.

Why?

How about: If there is a need, somebody will provide at a suitable price?
If no body steps up, we don't need it.

There seems to be ample evidence, in many arenas, that naked
capitalism can have disastrous results.

Folks -

At some point, a society decides that X is important enough to the
society as a whole, that something official is in the overall
interest. Roads, immigration, whatever. That it's necessary to require
that some things be done (or not be done).

Peering may very well not be in that category, however I think it's
worth discussion.

- Marc

Marc Farnum Rendino wrote:

More regs and more laws is certainly not in the running.

Why?

That is the way government works, too much, too late, in the wrong place.

How about: If there is a need, somebody will provide at a suitable price?
If no body steps up, we don't need it.

There seems to be ample evidence, in many arenas, that naked
capitalism can have disastrous results.

There is no evidence whatever that "naked" capitalism has ever been allowed to operate.

Marc Farnum Rendino wrote:

Folks -

At some point, a society decides that X is important enough to the
society as a whole, that something official is in the overall
interest. Roads, immigration, whatever. That it's necessary to require
that some things be done (or not be done).

Peering may very well not be in that category, however I think it's
worth discussion.

- Marc

My response would be to point out the behavior of the incumbents in the
telcom industry vs. the upstarts; which are the ones responsible for
most of the innovation that actually got delivered to the market?

Regulations raise barriers to entry; which favors the incumbent over the
upstart.

Roads are a bad analogy and will only serve to obscure the discussion.

--Patrick

I'll make one comment before 'Alex the Hammer' closes this discussion for straying into politics.

Clearly regulating the incumbents to unbundle local loops has worked very well in some European countries (France and possibly others). Clearly US financial deregulation has cost the world dearly.

So regulation is the appropriate response in some cases (I hope that is clear given the world financial system almost went under a few weeks ago).

However, it is not clear what a well crafted peering regulation would do that is different than what the market has achieved already.

Sensible and hence pragmatic government mandated peering would require companies having equal bilateral traffic flows to peer or buy transit from each other. That would not necessarily preclude the current peering dispute.

Forcing companies to peer when it is not in their interest is unlikely to be supported by the courts in any country, even the French and German courts.

Sooner or later these two companies in conflict will either return to peering or one of them will buy transit to reach the other.

It is a short term issue that probably doesn't merit government intervention

Regards,

Roderick S. Beck
Director of European Sales
Hibernia Atlantic
13-15, rue Sedaine, 75011 Paris
http://www.hiberniaatlantic.com
Wireless: 1-212-444-8829.
French Landline: 33+1+4355+8224
French Wireless: 33-6-14-33-48-97.
AOL Messenger: GlobalBandwidth
rod.beck@hiberniaatlantic.com
rodbeck@erols.com
``Unthinking respect for authority is the greatest enemy of truth.'' Albert Einstein.

> How about: If there is a need, somebody will provide at a suitable
price?
> If no body steps up, we don't need it.

There seems to be ample evidence, in many arenas, that naked
capitalism can have disastrous results.

And there are lot of examples and ample evidence in history, in many areas,
that complete regulation, complete socialism can have disastrous results as
well.

If you want to have a good idea on how the internet will look like in the US
after regulation, simply look at Australia. The government imposed
regulation early on in internet infrastructure market caused nothing but
raising the entry barrier for small ISPs, only creating government-approved
monopoly for major telcos/carriers. Only such regulation creates a
situation where it is cheaper and affordable for a smaller ISP to route
traffic from .AU to .US, then back to .AU than interconnect directly with
incumbent carrier in their own country. So yes, more regulations definitely
help the internet indeed (by adding extra 300ms into the process).

Instead of calling for socialist/communist policies to regulate the transit
industry, the single-homed networks can simply multihome. Because of
Cogent, the cost of transit has come down to single-digit per megabit that
even after adding transport costs, it's now affordable to add a 2nd internet
connection for practically most organizations out there, especially in the
continental US (the same capitalism that you call 'disatrous results' is the
same capitalism that brought cheap dollars/meg pricing, allowing smaller
companies to multihome now when they couldn't afford to do so in the past).

As much as we blame Cogent and Sprint for breaking the internet, I also have
no sympathy for individual single-homed downstream customers on either
networks. If you are complaining about Sprint<->Cogent depeering and have
customers demanding for your mission-critical services, then you are just as
negligent to not have multihomed before all of this happened. If you need
that 100% uptime guarantee, you shouldn't rely on single carrier, nor should
you rely on government for more regulation. No one can help you but
yourself in ensuring your uptime-- so perhaps look at your own setup and
decide that you need that 2nd connection to back you up when first one
fails. This is a simple business logic.

James

The only government intervention I can imagine as being productive would be to mandate what the "Internet" is, and if someone is selling access to it, mandate that customers can demand a refund in case the "Internet Access" doesn't provide access to enough a big part of it in a well enough working manner.

If two parties were both to lose enough money over cutting off peering, then perhaps they'd make sure there was a backup path that at least worked well enough to enable them to refuse to refund their respective customers.

I see lots of problems with definitions in this model, but it's worked (for some definition of "worked") in other areas and it might work here as well.

Mikael Abrahamsson wrote:

The only government intervention I can imagine as being productive would be to mandate what the "Internet" is, and if someone is selling access to it, mandate that customers can demand a refund in case the "Internet Access" doesn't provide access to enough a big part of it in a well enough working manner.

In some parts of the US, we already have that. We call it "Contract Law" where I live.

You "make a deal" with somebody, with notes on paper about what each of you think is to be delivered in each direction, then when everybody agrees the notes accurately reflect the agreement, everybody signs it.

If somebody reneges, the lawyers get rich and maybe a repair is worked out, maybe not.

But if that doesn't work, probably nothing else was going to either.

Precisely the issue I am concerned about. End consumers cannot go off and multihome easily. Comcast got in trouble for altering traffic flows to its residential customers. Sprint has broken access to its EVDO customers. Does it make sense for end customers to be protected from companies providing access to only parts of the Internet?

Sprint could, in response to this partitioning, buy some transit to provide complete connectivity to its EVDO users. But unless they're willing to allow termination of contracts for cell phones and data cards without penalty, consumers are NOT free to switch carriers, and they are not getting unfettered access to the Internet as was sold to them. The other carriers in the space aren't much better. Verizon got in trouble for selling "unlimited" access via data cards, then cutting people off who used it heavily.

Is it worthwhile for the government and/or the courts to set rules for such? As a consumer, I would prefer the government protect me from large businesses selling me one thing, then delivering another. Consumer protection is a valid and useful function of government, IMO.

James Jun wrote:

As much as we blame Cogent and Sprint for breaking the internet, I also have
no sympathy for individual single-homed downstream customers on either
networks. If you are complaining about Sprint<->Cogent depeering and have
customers demanding for your mission-critical services, then you are just as
negligent to not have multihomed before all of this happened. ...

Ah yes, I suspect we can get all the network operators here to agree that any customer of another ISP should buy a second connection "just in case". Maybe this breakage will turn out to be the best way for everyone to double their customer base overnight.

But seriously, it shouldn't be necessary to have two connections at work, two connections at home, two connections for each mobile device, just to ensure that when large providers stop working together you can still reach what you need to reach.

Matthew Kaufman
matthew@eeph.com
http://www.matthew.at

James Jun wrote:

How about: If there is a need, somebody will provide at a suitable
      

price?
    

If no body steps up, we don't need it.
      

There seems to be ample evidence, in many arenas, that naked
capitalism can have disastrous results.
    
And there are lot of examples and ample evidence in history, in many areas,
that complete regulation, complete socialism can have disastrous results as
well.

If you want to have a good idea on how the internet will look like in the US
after regulation, simply look at Australia. The government imposed
regulation early on in internet infrastructure market caused nothing but
raising the entry barrier for small ISPs, only creating government-approved
monopoly for major telcos/carriers. Only such regulation creates a
situation where it is cheaper and affordable for a smaller ISP to route
traffic from .AU to .US, then back to .AU than interconnect directly with
incumbent carrier in their own country. So yes, more regulations definitely
help the internet indeed (by adding extra 300ms into the process).

Instead of calling for socialist/communist policies to regulate the transit
industry, the single-homed networks can simply multihome. Because of
Cogent, the cost of transit has come down to single-digit per megabit that
even after adding transport costs, it's now affordable to add a 2nd internet
connection for practically most organizations out there, especially in the
continental US (the same capitalism that you call 'disatrous results' is the
same capitalism that brought cheap dollars/meg pricing, allowing smaller
companies to multihome now when they couldn't afford to do so in the past).

As much as we blame Cogent and Sprint for breaking the internet, I also have
no sympathy for individual single-homed downstream customers on either
networks. If you are complaining about Sprint<->Cogent depeering and have
customers demanding for your mission-critical services, then you are just as
negligent to not have multihomed before all of this happened. If you need
that 100% uptime guarantee, you shouldn't rely on single carrier, nor should
you rely on government for more regulation. No one can help you but
yourself in ensuring your uptime-- so perhaps look at your own setup and
decide that you need that 2nd connection to back you up when first one
fails. This is a simple business logic.

James

If things were truly operating as designed the internet would be able to automatically route around this depeering..the problem is not only do these two depeer but they also totally block any other traffic coming in from the other side. This is not how things should be done..disconnect the peering but let the traffic get automatically route around the disruption as it should.

Well, selling you an "unlimited" account and them terminating that contract if you use "to much" is one thing, that is a stated lack of a limit in your contract.

There is no delivery guarantee of your IP packets in your contract, adding one would be a rather bad idea since there is no delivery guarantee in IP that your service is based on and that would open a carrier to liabilities if someone was using a firewall for instance since that is effectivly limiting your delivery to that machine.

What you are buying is access to Sprints network, and transit effectivly on Sprints view of the Internet, and that is what they deliver really..

But seriously, it shouldn't be necessary to have two connections at
work, two connections at home, two connections for each mobile device,
just to ensure that when large providers stop working together you can
still reach what you need to reach.

I think you're misinterpreting what I'm trying to say.

The consumers/end-users don't necessarily have to multihome. The problem is
the content providers/web hosters sitting single-homed on either networks,
when most of them are physically sitting in better environment to multihome
(i.e. a datacenter) than consumers.

A consumer can be single homed to Sprint or Cogent, but when the other side
(the content) is multihomed, you'll simply take new route to get to that
content. My point is, any business providing services over internet (this
excludes mobile devices, end-user/consumers) should be multihoming
themselves if they are serious about uptime.

James

Matthew Kaufman wrote:

James Jun wrote:

As much as we blame Cogent and Sprint for breaking the internet, I also have
no sympathy for individual single-homed downstream customers on either
networks. If you are complaining about Sprint<->Cogent depeering and have
customers demanding for your mission-critical services, then you are just as
negligent to not have multihomed before all of this happened. ...

Ah yes, I suspect we can get all the network operators here to agree that any customer of another ISP should buy a second connection "just in case". Maybe this breakage will turn out to be the best way for everyone to double their customer base overnight.

I have a probably dumb question. Even if a company were of large enough
wallet to have, say, a single redundant connection, how could it
evaluate the partition problem in order to choose the "best" connection
(where "best" is a function of overall connectivity, say) ? It seems to
me that that's a really, really hard problem. And surely isn't a static
one-off kind of calculation, right?

    Mike

* Seth Mattinen:

4. Multihome.

Or get upstream from someone who does, and who is small enough to be
able to get additional upstream upon short notice. I know that this
solution isn't always cost-effective. 8-/

(Multihoming alone isn't a solution because it's hard to figure out
how independent your peering partners are.)

William Warren wrote:

If things were truly operating as designed the internet would be able to automatically route around this depeering..the problem is not only do these two depeer but they also totally block any other traffic coming in from the other side. This is not how things should be done..disconnect the peering but let the traffic get automatically route around the disruption as it should.

Some of us gun-clingers think involuntary servitude is a seriously bad idea.