Sending vs requesting. Was: Re: Sprint / Cogent

Mailman List Mirror (Read Only)
1

Why does everyone keep referring to traffic flows as sendng?
In this case it's not as if Cogent just randomly sends data to Sprint.

Sprint customers are requesting content from Cogent customers right?
So Sprint depeers Cogent because Sprint customers are requesting to
much content from Cogents customers?

I've heard eyeball networks refer to traffic flows as sending too..
"You content hosters are sending us too much traffic, we want money to
upgrade ports and transport all that traffic" Complete reverse logic
imho. It is always eyeball network customers that request data.
(except for a small portion of iphone/blackberry push email, but that
can't account for much.)

Bastiaan

2

it makes little to no difference how you skin that cat... the traditional
model still plagues so called "content rich" networks and has been used,
shamelessly, by the eyeball networks with no end in sight. i am by no means
defending cogent, nor do i claim to know that ratios are the only item on
the list of violated peering agreement clauses.

my particular complaint is that with the upswing of broadband in this
country it is continually less and less to do with "how many direct eyeballs
do i have" and more to do with "to which cable/dsl providers do i provide
transit." the former was used as a cost-model basis for the eyeball
networks requiring ratios as it was far more expensive to establish a broad
presence to provide eyeball connectivity... the latter does not match that
logic and has yet to filter it's way into fair settlement-free peering
agreements.

/jer

3

Once upon a time, bas <kilobit@gmail.com> said:

I've heard eyeball networks refer to traffic flows as sending too..
"You content hosters are sending us too much traffic, we want money to
upgrade ports and transport all that traffic" Complete reverse logic
imho. It is always eyeball network customers that request data.
(except for a small portion of iphone/blackberry push email, but that
can't account for much.)

Traffic sources tend to be concentrated in large data centers (easier to
service), while traffic sinks (DSL, cable, wireless) are widespread and
costly to upgrade. The sink customers don't want to pay more (and
there's at least some competition), so the sink providers look to see
where else they get income to pay for their needed network upgrades.

4

Combined with hot-potato routing, the first part of that paragraph is a fancy way of saying "I have to carry the large packet a long way, you have to carry the small packet a long way". It is not "fair". This is almost a good reason, but not quite. (It can also be offset by moving the source next to the sink, through cold-potato routing / MEDs, anycast, CDNs, etc.)

The second part is a good business reason. Profitable revenue is good, costs are bad.

There are good business reasons not to pay the sink as well. But neither decision is obvious or the same for everyone.

Peering is complicated, people should stop trying to generalize it.

Peering is a business tool. For years & years many people have claimed that to "peer" you must be equal. Bullshit. If I can make more or spend less by peering, I should do it. If not, I should not. Full stop. Notice the complete lack of regard for how big you are, how much capacity your backbone has, how many ASes are downstream of you, etc.? When I go to buy routers or hire employees or any other business transaction, I don't say "that router vendor is making more money than I am, so I won't buy from him". If people applied "peering" logic to anything else, they'd be laughed out of a job.

Don't know about you, but I am in business to make money, not measure my anatomy. How big the next guy is doesn't enter into my equation - other than how it affects my bottom line.

To be clear, it is entirely possible that peering does not save you money. Vijay is right, most people can't measure their COGS to save their life. And if the network in question cannot, there's no way in hell the prospective peer can. If you are a huge point source of traffic and want to peer, I may save money by saying no and paying a transit provider to deliver the packet to me where I want it (especially at today's prices). Fiber, routers, colo, NOC employees, engineers, etc., are all not free ya know.

You can claim my customers asked for the data and therefore I have a requirement to peer, but you would be deluded. What my customer and I have agreed has _nothing_ to do with you or your needs. You don't tell me how to run my network, and I won't tell you how to run yours. Deal?

On the flip side, saying "you are not on 3 continents so I will not peer" is stupid of not peering costs you millions a year. Stupid decisions abound in the peering ecosystem.

There are tons of other _business_ reasons to peer, or not to peer. But "we're equal" or "your customer asked for it" are not reasons, stop using them.

5

Patrick,

To further your point about the dynamics of peering:

Not to sound overly altruistic, but nowhere in there did I see, "it's good for the Internet". If peering was less of a raw business decision, the Internet would be a better place. In this case, if they left it status quo and congested, at least users could send smoke signals across the two networks and could at least communicate.

The implications of this depeering could be pretty severe. If you dive into business ethics, there's some pretty serious moral dilemmas involved with cutting communications between two major networks. This could be taken to an extreme if it causes VoIP calls to fail and ultimately disrupts someone's 911 calling. In less reaching situations, someone can't SMS with their wife to know what to bring home from the grocery store. Regardless of how stupid relying on the Internet is (I know it's sad we can't) I do feel networks have a moral responsibility to provide continuity.

As you know, beyond business, peering can decrease latency, increase throughput, and provide better network engineering, thus increasing the scale of the overall Internet. As a technologist and entrepreneur I try to do what's best for the Internet in my peering decisions rather than what Bill Lumbergh would say, "umm yeah, do what's best for the company".

In this case, it's very clear that customers are impacted and the Internet as a whole suffers, which is really unfortunate. The end result of a business decision has been to sacrifice the customer's needs, trust, and ability to communicate. It's a bad maxim to subscribe to! I really hope that other networks do apply more thinking into peering than just what's best for business -- it sure shows off an ugly underbelly.

-Barrett

6

...

In this case, it's very clear that customers are impacted and the Internet
as a whole suffers, which is really unfortunate. The end result of a
business decision has been to sacrifice the customer's needs, trust, and
ability to communicate. It's a bad maxim to subscribe to! I really hope
that other networks do apply more thinking into peering than just what's
best for business -- it sure shows off an ugly underbelly.

-Barrett

Unfortunately, as I'm sure you're all too aware, for public companies, it's
very hard to get away with saying "I was doing what was right for the
Internet, not what would make my business the most money" at a
shareholder meeting, or during an earning's call with Wall Street
analysts; they tend to be very unforgiving of actions that aren't in
line with the short-term profit-making goal, to the point where CEOs
have been ousted and class-action lawsuits threatened when it
seems the actions being taken weren't geared to optimize profits
for the shareholders.

Converging two threads together, I think the same pressure affects
IPv6 deployment and will affect IPv6 peering; while it would be *best*
for the Internet if everyone put the time and resources forward to get
dual-stacked now, bring up widespread peering, and get IPv6 to a
point where it's a viable transport mechanism, the real fact of the
matter is that there's no profit motive in moving to IPv6 at the
moment, so people just aren't going to do it, no matter how much
it may be "best for the Internet".

Fix the market drivers for public companies, and then we can
fix the Internet; otherwise, we'll always be steering towards
that which makes sense for the business, regardless of which
customers of other networks it hurts, or which resource exhaustion
cliff it hurtles us towards.

Putting on a devil's advocate hat for a moment...if various international
regulatory bodies and government agencies mandated universal
connectivity via both IPv4 and IPv6, depeering would cease to be
an issue; regardless of what the business side said, companies
would not be allowed to partition the Internet, and widespread adoption
of IPv6 would be forced, rather than being a "maybe someday" case.
Downside is that prices would go up, and expansion into new regions
would slow down, as the costs associated with developing new areas
would bring a much higher price tag. It would be better for the Internet
as a whole, but worse for most of the individual users of it, from a cost
perspective. Would you still say things were better for the overall
Internet at that point, if it meant everyone had to pay more in order
to ensure that universal connectivity?

I'm cheap, so I'm leaning towards the side of letting the market
work these issues out; but I'm always willing to listen to other
thoughts on the matter. :slight_smile:

Matt

7

True... however this depeering may have created more of a mess for Sprint's marketing and their customers than they predicted, which has a negative impact on business and would not be fun to explain at a board meeting.

I guess it's hard for sweater vests to understand that until it smacks them in the face.

-Barrett

8

bas wrote:

Why does everyone keep referring to traffic flows as sendng?
In this case it's not as if Cogent just randomly sends data to Sprint.
  

I think it's a really odd reinterpretation of telephony concepts. In telephony interconnects are typically settlement based, sender pays receiver, in the settlement based world it seems to have gotten confused.

I'm still trying to come to terms with what Sprint is trying to achieve here. I can only assume it's (and I'm stealing from Vijay here) to raise Cogent's cost of doing business by forcing them to do settlement based or paid peering and thus trying to force the cost of their transit to rise. Maybe it's to damage Cogent's reputation as well? The cost of doing this seems to be high (ie. upsetting high paying (single homed) transit and mobile customers) and getting negative media coverage.

Is this really going to make a substantial kind of difference?

MMC

9

Matthew Moyle-Croft wrote:

I think it's a really odd reinterpretation of telephony concepts. In telephony interconnects are typically settlement based, sender pays receiver, in the settlement based world it seems to have gotten confused.

"in the settlement FREE world it seems to have gotten confused."

MMC

10

I really doubt Sprint's purpose here is to "hurt the Internet" or to harm
Cogent either in terms of costs or reputation.

Here are my views on the topic:

  Every time Cogent gets de-peered (at least 5 times now since
2003), this discussion comes up again and it seems that some people forget
(or don't know) how many times it's happened to them before. There must
be a reason it keeps happening, right? Are there any other large ISPs
that have had this type of problem 5 times?

  As someone was saying earlier, in the PSTN world carriers
generally pay for every call terminated to another carrier's network and
pay each other back and forth. In IP peering, these types of costs are
"eliminated" by settlement-free peering relationships where carriers feel
there is a benefit to do so.

  These are relationships or contracts between the two carriers, and
most of us have no idea how these are written or what clauses are included
about how and when one carrier can end that contract. Regardless of the
exact terms, there will certainly be actions or other situations that
would be viewed as a breach of contract, resulting in ending or changing
the relationship.

  In the case of Cogent, they seem to want to be a Tier 1 carrier
(usually loosely defined as an carrier that does not pay for transit or
access from/to any other carrier), but they are not usually considered one
by many in the industry. Technically at this point they are not since
they are believed to pay Level 3 and Sprint.

  Now I really can't speak to exactly why each carrier that has
de-peered Cogent in the past has done so, but based on conversations I've
had with higher-ups at one of these ISPs, their major issue with Cogent
was a huge discrepancy in the volume of inbound vs. outbound traffic. To
that carrier, based on the traffic patterns, they believed that Cogent
should be paying for their connections and was not keeping to the spirit
of their relationship or breaking the contract if there was one. They
supposedly attempted for some time to resolve the issue amicably, but when
that failed they chose to take action as a last chance to resolve the
dispute to their liking.

  Now as to the "harmful" effect to Cogent's customers, that effect
would be easily mitigated if Cogent would choose to buy transit from any
other ISP. Instead, they try to avoid that by offering affected customers
free circuits for some period of time, which hopefully turn into paying
customers at a later date. Also, anyone running any important site or
network knows never to be single-homed, and therefore should not be
effected in the long run. Anyone single homed accepts the risks
associated with that by not having redundant connections, especially if
that single home is Cogent based on their history of peering arguments.

  So based on that the only "difference" I'd expect this to make is
in the relationship between Sprint and Cogent in the future. I doubt this
will change Sprint's, Cogent's, or any other ISP's corporate
views/policies on peering in the long term.

  Just my 2 cents,

  -Scott

11

Well put. The etymology of the whole mindset around peering is a legacy
from the academic/socialist roots of the Internet. There are still a
great number of people who think this is some kind of social engineering
experiment, as opposed to a communications infrastructure run by, and
for the benefit of, businesses.

If that craws anyone's hide, then go build community networks and peer
with each other, no-one's going to stop you. By the same token, you
don't have the right to force anyone else to pay for what you want.

From: Patrick W. Gilmore [mailto:patrick@ianai.net]
Sent: Saturday, November 01, 2008 9:46 AM
To: NANOG list
Subject: Re: Sending vs requesting. Was: Re: Sprint / Cogent

Once upon a time, bas <kilobit@gmail.com> said:

I've heard eyeball networks refer to traffic flows as sending too..
"You content hosters are sending us too much traffic, we want money
to
upgrade ports and transport all that traffic" Complete reverse

logic

imho. It is always eyeball network customers that request data.
(except for a small portion of iphone/blackberry push email, but

that

12

The next Great Compromise could be over multihoming of endusers (whether the
enduser is a content consumer or a content provider is irrelevant; IP at
Layer 3 is peer-to-peer anyway, between end systems). After all, if
providers are not willing to 'budge' a little 'for the good of the Internet'
then why should endusers reduce or give up multihoming aspirations 'for the
good of the Internet?'

After all, if access to the 'Internet' is a metric for making money, and money
is being lost by access to content consumers being cut due to depeering
events outside the enduser's control, then it makes business sense for an
enduser to multihome, to bring connectivity events closer to the enduser's
control. Want to prevent multihoming exploding routing tables? Prevent the
desire to multihome by limiting 'partitioning' events.

13

Lamar Owen wrote:

  

Unfortunately, as I'm sure you're all too aware, for public companies, it's
very hard to get away with saying "I was doing what was right for the
Internet, not what would make my business the most money" at a
shareholder meeting, or during an earning's call with Wall Street
analysts; they tend to be very unforgiving of actions that aren't in
line with the short-term profit-making goal, to the point where CEOs
have been ousted and class-action lawsuits threatened when it
seems the actions being taken weren't geared to optimize profits
for the shareholders.
    
The next Great Compromise could be over multihoming of endusers (whether the enduser is a content consumer or a content provider is irrelevant; IP at Layer 3 is peer-to-peer anyway, between end systems). After all, if providers are not willing to 'budge' a little 'for the good of the Internet' then why should endusers reduce or give up multihoming aspirations 'for the good of the Internet?'

After all, if access to the 'Internet' is a metric for making money, and money is being lost by access to content consumers being cut due to depeering events outside the enduser's control, then it makes business sense for an enduser to multihome, to bring connectivity events closer to the enduser's control. Want to prevent multihoming exploding routing tables? Prevent the desire to multihome by limiting 'partitioning' events.
  
So now I'm going to join Randy and say this is rehashing. We know the "interwebz" are a collection of AS. We know they can partition at any time.
We know that certain players have a history of causing this to happen more then others.

What I haven't seen discussed in any great detail, is how to limit those events.

Some are asking for regulation. Ok. What will that regulation look like? As the technical network operations community, what
do we want to see? What stipulations etc? Are a few interruptions that affect a small subset of IP addresses and AS on the net
really worth going through the effort to pass regulation around the world? If so, then let's take advantage of a long and rich history
of reaching consensus (see the RFC process which I think has work pretty well) and hash out what we want to see.

Otherwise let's move on.

Charles