Level 3 Communications Issues Statement Concerning Comcast's Actions

Netflix pays someone to get access to "the internet" and that someone has
some sort of relationship with Comcast, or gets to Comcast through a third
party who has that relationship. No one is getting anything for free.

I don't think it's unreasonable to expect customers to bear the cost of
their provider doing business. If that business calls for the buildout of
additional infrastructure to remain competitive then so be it. Comcast
customers pay their provider, Netflix pays its provider.

I think what this really boils down to is an effect of shoddy marketing.
Access providers want to offer "unlimited" everything and don't want to have
to go back to their customer base and say, "oh, sorry, we didn't really mean
unlimited. We didn't think you'd really use that much." So they are looking
for ways of making up for the increased costs without having to look like
idiots to their customers.

My problem is, what happens if this becomes the new model? What if Comcast
comes to me and says, "Oh, we've noticed X Mbits originating from your
network coming through ours. Here's the bill of $X per bit." What happens
when I counter with, "Ok, and I see X bits originating from your network.
Here's my bill, too." Do they agree to an exchange of money for an exchange
of bits or do I get an "F you. Pay your bill to us and we're not giving you
crap."

Aaron

I hope Level3's acquiescence is temporary or the FTC puts a stop to

it.

It is sort of like FedEx owning the freeway and charging UPS to use

it.

Note that I would have no problem with Comcast's demand for payment if
they didn't also own NBC/Universal. If they were to divest themselves
of NBC/Universal and charge NBC/Universal the same distribution fee,
then I say fine.

The problem here is the fact that Comcast isn't charging more to pay for
the cost of viewing streaming content, they are charging more for
viewing COMPETING streaming content.

As a Comcast customer, I find it very interesting that they think they have
the right to reject bits from other providers I may request them from.

The first time I encounter Comcast actually blocking bits I want as a
result of this policy, it will result in a technical support call. If the bits
don't start flowing within a reasonable time after that call, you can
bet that I will be pursuing regulatory and judicial relief.

Ordinarily, I would simply vote with my feet and switch to another
broadband provider, but, if you want more than 1.5Mbps/384kbps
in my neighborhood, Comcast is currently the only game in town.

I encourage other Comcast customers to make it clear to Comcast
that this attempt to extort money from other providers at the potential
cost of degraded service to Comcast's own customers is deplorable
and certainly violates the spirit if not the letter of the service agreements
for their high speed internet service products.

Owen

In a message written on Mon, Nov 29, 2010 at 03:34:52PM -0800, Seth Mattinen wrote:

My take on this is that settlement free peering only remains free as
long as it is beneficial to both sides, i.e. equal amounts of traffic
exchanged. If it becomes wildly lopsided in one direction, then it
becomes more like paying for transit.

When you have users and no content how can the traffic be equal?

When you have content and no users how can the traffic be equal?

Ratio is horribly outdated. Cable and DSL providers enforce out
of ratio at the edge with technology and policy. My cable modem
is 8 down 2 up, yet my traffic profile is supposed to be equal? I
can't host any "servers" by my TOS, but aggregated up the ratio is
supposed to be 1:1?

No one will ever be in ratio compliance with an eyeball dominant
network. Ever. Period. It's not possible via technology and
TOS. Enforcing it as an eyeball network just forces content providers
to aquire eyeballs, e.g. compete with you. That's bad business.

But this isn't a technology problem, or a ratio problem. Peering
spats like this are ego problems. It's one VP/SVP/CTO/CFO deciding
that "my sandbox is more important than your sandbox", or "I'm going
to get revenue even if the world hates me for it and I'm going to
burn all my bridges in the process." If they actually wanted to
equalize the costs, they could do that. Decide on better peering
locations, use cold potato routing, locate caching/cdn things inside
the other network, etc.

From: Rettke, Brian
Sent: Monday, November 29, 2010 2:41 PM
To: Patrick W. Gilmore; NANOG list
Subject: RE: Level 3 Communications Issues Statement Concerning
Comcast'sActions

Essentially, the question is who has to pay for the infrastructure to
support the bandwidth requirements of all of these new and booming
streaming ventures.

From a cultural standpoint, we in the US are used to a model where the

sender pays the postage for unsolicited content and the requestor pays
the shipping for requested content. So asking an ad network to pay
Comcast for shipping their ads is valid but in a request where the end
user specifically asked for a movie, the user should be expected to pay
for that. What Comcast appears to be doing is subsidizing "flat rate"
customer rates by charging the providers "metered" access (assuming the
fee charged the provider isn't also a "flat rate"). If so, that really
isn't fair because:

On the internet, how would you tell these apart?

First, as I have said in a previous message, I think Comcast's actions
here are deplorable. However...

Generally speaking, most (legitimate) ads are delivered as content on
web pages at various sites. As a result, they are "requested" specifically
by the user's browser right alongside all the other content the end user
actually asked for.

Perhaps part of the problem here is that Comcast comes from a culture
where advertisers pay to deliver their advertising which is delivered
directly alongside the content that the consumer actually wants.
Perhaps Comcast is having a hard time realizing that the internet
is not broadcast television, or, perhaps they think they can convert
the rest of us to thinking of the internet along those lines.

Certainly the ad-delivery method on the web is actually more in line
with that of broadcast television than it is in line with postal delivery
of advertising.

1. The provider has no control over the size of or number of requests
that are made. The provider is essentially agreeing to an unknown
quantity in advance.
2. There is no way to ensure that a request is legitimate and not a
request generated simply to generate revenue (sort of like click fraud
... stream fraud).
3. It opens the provider up to a "denial of sustainability" attack
where a bot net requests many copies of various streams, sends them to
the equivalent of /dev/null and the provider is presented with a huge
bill.
4. The only way a provider could mitigate increases in fees is to meter
access causing a sub-optimal user experience.

All valid points. In fact, this points out that while the approach
Comcast is taking appears to resemble the "sponsor pays"
model of broadcast television, the metered aspect is a major
difference which changes the game significantly.

I suspect that Comcast is operating from the assumption that
every request for content to the provider is money flowing to
the provider so that the money collected by Comcast for the
corresponding traffic is merely "their cut" of the revenue.
Of course we all know this is a flawed assumption on Comcast's
part, but, if you look at it from that mind set the belief can
be rationalized, even if it is misguided.

Shouldn't Level3 turn around and charge Comcast for distributing
NBC/Universal content?

Seems like a reasonable response, but, hopefully Level 3 will not
surrender the high ground since they have it for the moment.

The whole thing is like a movie theater charging the studios to show
movies while selling tickets to the public to watch them. Actualy, it
is like Universal opening their own movie theaters and charging
competing studios to show their movies while still charging the public
to see them. Comcast is simply imposing a tariff on competing content.
If I were level3, I would have denied the request. Customers on Comcast
would then discover they have sub-optimal Internet service and gone to a
competitor (AT&T Uverse or Verizon FIOS, for example).

That's great in areas where that is an option. There are many areas served
by Comcast where Uverse, FIOS, etc. are not available. I live in San Jose,
California, the so called Capitol of Silicon Valley. In my neighborhood,
Comcast is the only cost effective alternative for more than 1.5mbps/384kbps.
(A residential DS-3 circuit is not cost effective).

As owner of NBC Universal, Comcast is a producer as well as a
distributor of content. That puts them in direct competition with other
producers regardless of the distributor. Level3 should deny the request
and Comcast users will have "Internet" access instead of Internet
access. Comcast doesn't have the captive audience they once had in many
places and when customers discover their choices are limited when they
choose Comcast, they will go elsewhere.

I hope Level3's acquiescence is temporary or the FTC puts a stop to it.
It is sort of like FedEx owning the freeway and charging UPS to use it.

I hope so, too. Unfortunately, Comcast is very experienced at these
kinds of bullying tactics and they have used them very successfully
with broadcasters and others for many years. If there is hope, it likely
lies with the regulators.

Owen

The NSPs generally don't do non-transit peering unless traffic loads are high enough to justify it. That said, CDNs are the same. Google doesn't want to peer privately with someone who doesn't do enough traffic to justify the cost of the port, haul, support, etc.

The ratio of which way bits are flying are really irrelevant when peering, and as you say, tends to be more ego than anything. The key, and what everyone wants is "Someone paying me talks to someone I don't have to pay." Doesn't matter if it's CDN talking transit to an eyeball network or eyeballs paying for transit to access a privately peered CDN. What you don't want is 2 entities talking to one another through you without you making a dime.

Jack

Now we know what "Xfinity" means :slight_smile:

Jeff

I find it helpful to distinguish "participant neutrality" from "service neutrality". The first says that you and I pay the same rate. The second says the my email costs the same as my voip.

As described, it appears that Level3 is being singled out, which makes for participant non-neutrality. On the other hand, if Comcast were charging itself for xfinity traffic, this might qualify as service non-neutrality (assuming there is a plausible meaning to "charging itself"...

d/

Equal *value* of traffic exchanged. A network that has a lot of eyeballs
may be willing to accept some imbalance to connect to a popular source
of content, and that content source is equally motivated to cut some
slack on the ratio to get good access to more eyeballs.

Oddly enough, cable channels like ESPN asking for a per-subscriber fee
from cable delivery networks like Comcast has been a mostly-scalable
model for the cable-TV arena for three or four decades now....

http://blog.comcast.com/2010/11/comcast-comments-on-level-3.html

Okay's let's say L3 gives in to Comcast and pays them. L3 then turns around and charges us (providers) more to cover the additional money they have to pay Comcast now. In the meantime Comcast continues to undercut the market it sells into making it harder for me as a service provider to compete...that just isn't right. Maybe Comcast should raise their prices to their customers to cover the cost of upgrading there network, but then they wouldn't be able to undercut me anymore...monopolies are a dangerous thing!

L3 gave into Comcast and paid them already according to a press release
they issued.

William.

In summary:
  Level3 is crying foul while their CDN competitors have quietly bought
into Comcast's racket.

  I applaud Level3 for calling attention to this matter.

Owen
(Speaking strictly for myself)

I think what this really boils down to is an effect of shoddy marketing.
Access providers want to offer "unlimited" everything and don't want to have
to go back to their customer base and say, "oh, sorry, we didn't really mean
unlimited. We didn't think you'd really use that much." So they are looking
for ways of making up for the increased costs without having to look like
idiots to their customers.

Unlimited access is already NOT unlimited access. A transfer cap isn't unlimited..while Comcast has a generous cap..it's still a transfer cap.

Comcast's blog specifically mentions unbalanced ratios as an issue.

~Seth

Is L3 hosting content for Netflix?

You bet.

http://blogs.barrons.com/techtraderdaily/2010/11/11/level-3-signs-deal-to-be-a-primary-netflix-cdn-shares-rally/

• NOVEMBER 11, 2010, 9:13 AM ET

Level 3 Signs Deal To Be A Primary Netflix CDN; Shares Rally

Regards
Marshall

Trying to follow this - so, if I have followed it correctly, L3 hosts
high-bandwitdh services (namely NetFlix) to which an abundance of Comcast
users subscribe? And Comcast is crying foul, and claiming a portion of L3's
revenue is rightfully theirs, for being "last mile" to a significant portion
of the CDN/NetFlix customer base? Does L3 even service a home user market,
in the same vein as Comcast or Verizon?

Hi,

This is nothing to do with technology, it never is, it is about cap-ex, money, sales, market share, dominance, internal products, hurting the competition.

While I have delusions about technical people putting agenda aside to work in a co-ordinate fashion on IPv6 I have no such delusions the second a commercial interest enters the fray.

Cogent made Level3 bend over years ago, it will be interesting to see if Comcast can do the same or if Level3 will grow a pair and refuse to be bullied despite the commercial loss.

Ben

And what happens when the content providers have multicast to the BGP edge and the access provider has to carry it from there on in their network.

This is solely about money and the brokenness of the current ISP / access / carrier / content provider commercial model. This has been coming for years once access speed (long since) got upto a sufficient speed to sustain 1 to 2 Mbit and they sorted out their copyright issues on the content.

Now all the access providers who spoke big in marketing and delivered little in service are being exposed and trying to fudge the issue. This has been coming for at least five years with video, and the next one is SIP with call revenues.

Show me the money!