RE: C&W Peering Problem?

Mailman List Mirror (Read Only)
1

Since on the Internet the sender pays for sent traffic, and the receiver
pays for received traffic, I've never understood the argument advanced by
BBN/Genuity, UUNET and now apparently C&W that unbalanced traffic means
someone is getting a free ride.

If the customer pays flat-rate, you collect the same amount of money no
matter how little traffic they send or receive. The 95% charging
used by some providers is the greater of *either* inbound or outbound
traffic. So imbalanced traffic to or from your customers is paid by your
customers.

So, can anyone explain why C&W, UUNET or Genuity care about traffic
balance, other than to limit competition by providers who are better
at attracting particular types of customers than them? If you are good
at being a webhoster, your traffic will have one profile. If you are
good at being an access provider, your traffic will have another profile.

If you are mediocre at everything, I guess your traffic will be balanced.

2

From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu]On Behalf Of
Sean Donelan
Sent: June 1, 2001 10:28 PM
To: nanog@merit.edu
Subject: RE: C&W Peering Problem?

If the customer pays flat-rate, you collect the same amount of money no
matter how little traffic they send or receive. The 95% charging
used by some providers is the greater of *either* inbound or outbound
traffic. So imbalanced traffic to or from your customers is paid by your
customers.

I know this was beaten to death a few threads ago, but there are ISPs (eg:
one of ours) that bill 95th percentile on the TOTAL of inbound and outbound.

So, can anyone explain why C&W, UUNET or Genuity care about traffic
balance, other than to limit competition by providers who are better
at attracting particular types of customers than them? If you are good
at being a webhoster, your traffic will have one profile. If you are
good at being an access provider, your traffic will have another profile.

The real reason is probably $$$ plain and simple, but...

My understanding, based on talking to some people who run networks like
@Home which are totally access providers, is that the theory they use it
this. Let's say you have network A, a big access network, and network H, a
hosting network.
If the two networks peer in San Jose, Dallas, Chicago, New York, and
Washington, DC, and network H's biggest data centers are in San Jose but
network A's biggest customer base is in New York, that means that network H
sends lots of traffic through the San Jose peering link, and then network A
needs to carry tons of traffic on their backbone all the way to New York.
Meanwhile, network A sends acks and similar things to network H, and a
majority of those go through the New York peering link, and are then taken
back to San Jose on network H. The problem, the way network A sees it, is
that they might need to get an OC48 between San Jose and New York, whereas
network H can get away with an OC3/OC12 on the same path.
Thus, network A finds it unjust that they have to pay all this money for
this OC48 when network H, which is the network sending them all this
traffic, can get away with a much cheaper circuit, and thus they use this
excuse to try and bill network H in order to make as much money as possible.
Thus the "free ride" argument..

If traffic is balanced, then I guess each side feels like they're using
equivalent amounts of the other's network, and thus that it cancels out...
(of course, in a real life implementation, this is presumably significantly
more complicated, and I think at that point the logic vanishes in favour of
simple greed)

Vivien

3

So, can anyone explain why C&W, UUNET or Genuity care about traffic
balance, other than to limit competition by providers who are better
at attracting particular types of customers than them?

You have the cart before the horse (effect before cause), there are really
two principles that come before the example policy effect above. They are
truisms.

For the purposes of the rules below the term monopolistic peering refers
to core networks that have policies that would limit their peers to 10 or
so networks IF they were uniformly applied to all their current peers
(which they are invariably not (even though a few might be snubbed for
general purposes of crassness, ergo C&W depeering a few arbitrarily)).

1) The first rule of monopolistic peering is that the policy MUST
overwhelmingly favor the writer of the policy. This is a truism, no
company defines a policy that requires them to pay settlements, only vice
versa.

For example, this means that for all the lip service paid to settlement
based peering compensating the parties equitably you will not find any
that allow both parties to be paid. The few settlement based peering
contracts I've heard of typically are written as transit fading to
settlement free peering when some goal is met (the "correct" ratio and the
"correct" quantity of traffic). That is, only one side ever gets
compensated. If UUnet offers you a settlement based peering contract you
can bet they will not extend you the exact same compensation if you manage
to turn the tables on them and attract customers that suck traffic instead
of push. Otherwise people would rush out and implement that as a business
model.

2) The second rule of monopolistic peering is that the policy MUST be
written in a way that allows you to severely limit who you peer with.
This is a truism, in order to get the number of qualifying networks down
to 10 or so one must write sufficiently restrictive policies.

See, Sean, your error was assuming the rationalization accompanying the
rules you see in the more restrictive peering policies legitimately
represents the effect of the the policy. heh. :wink:

Hrmmm...

By the way, many large networks are not monopolistic. Some have hundreds
of peers and continue to evolve and grow their networks, taking into
account new market entrants that become sufficiently established.

Presumably these networks will provide lower latency and more direct
routes (asuming that is what you want) than once upon a time tier one
networks that have aggressively shrunk the number of strategic business
partners (um, peers, yeah them, who without your network becomes rather
boring).

+------------------- H U R R I C A N E - E L E C T R I C -------------------+

4

It seems as if a good solution to this kind of an imbalance would be for
the peering partners to agree to listen to each others' MEDs. That way
access provider B gets the traffic handed to them a close to the
destination as possible. True, that means they need a beefier backbone,
but the value in the peering arrangements they don't lose as a result
of imbalanced traffic would seem to compensate. Are there any known
iplementations like this among backbone providers?

-C

5

So, can anyone explain why C&W, UUNET or Genuity care about traffic
balance, other than to limit competition by providers who are better
at attracting particular types of customers than them? If you are good
at being a webhoster, your traffic will have one profile. If you are
good at being an access provider, your traffic will have another profile.

There are several reasons to care about traffic ratio. Where I
think the mistake is made is that providers are looking at ratio,
but that the ratios they use are fixed regardless of the type of
network they are evaluating. That said, it's hard to get more
flexable guidelines past the lawyers and bean counters, particularly
in a large orginization.

Here's a few interesting cases, first, the ratio problem.

         A1--------------B1---User
         > >
         > >
         > >
server---A2--------------B2

Consider "A" is west coast, "B" is east coast. User requests flow
B1, B2, A2, while reponses flow A2, A1, B1. Provider 1 ends up
carrying more bits a longer distance, and thus incurs a higher
cost.

There are several responses to this argument, each with their own
problems:

* That's what you get for building and end user network. If you
  don't like it, build a data center network. Most people don't
  like suggesting their business model is broken.

* Use BGP MEDs to make the return route A2, B2, B1. This moves
  the cost to network 2, which may or may not be fair. Many times
  provider 1 does not trust provider 2 to do this properly. Even
  when they do, sometimes it is impossible. BBN and ATT are good
  examples. If someone sends you a single /8, you have no choice
  but to hot potato it out, as meds make no sense. The only solution
  is deaggregation, which has a large number of other problems.

* A settlement should be paid from network 2 to network 1. This
  is possibly acceptable, if it comes in the form of a settlement.
  Often the pricing resembles transit, below.

* Network 2 should buy transit from network 1. Most of the medium
  to large networks are trying to be transit free, and reject this
  outright. Also, it's quite likely they would by transit from,
  well, anyone else just so network 1 doesn't get the money from it.

There is an important factor here many of the depeering crowd are
missing. The overall traffic ratio of your network is more or less
fixed, and is determined by your customer base. Unless you can
convert peers to customers (which I have never seen someone be
successful in doing on any scale), you will simply move the problem
around. That is, if you're 2:1 with Sprint, and depeer 5 10:1
guys, they may well buy transit from Sprint, moving them to 3:1
(due to traffic volume). Now what, depeer Sprint?

Most people from their billing software can add up all customer in
and all customer out. If your ratio is under that number, you will
_NEVER_ reach it, no matter what you do. Since individual peers
will be different, you probably want your limit to be about twice
your customer ratio, at a minimum.

This is why I believe you have to evaulate people based on value.
Consider someone like @home peering with someone like Globix. One
is a pure end user provider, that in fact prevents most of it's
users from running servers and the like. The other is a pure web
hosting company, with lots of content and almost no users.

These two networks _cannot exist without each other_. If they
refused to peer with each other based on ratio, it would be utter
folly. Clearly there is great value to both of them in peering,
even though the ratio may well be 10:1 or higher.

One of the funniest results of the ratio dance is that it may well
create more competitiors for a large network. A tight ratio (eg,
1.5:1) is really a requirement that you have a similar customer
mix, so you have a similar amount of in+out traffic. How many web
hosting networks, who didn't want to compete for end users have
been forced to go after end users big time? How many access only
networks have done things to attract server users? Companies that
could have enjoyed much less competition have forced people to
compete with them by ratio.

Equally interesting to me is the "minimum traffic" numbers that
many large networks want to put forth. Some of them are quite
high, with major networks requiring well over a gig of actual
traffic to qualify for peering. This has the effect of pushing
the restrictive peering policies down to smaller providers. If a
smaller provider has a lot of peers, they send less traffic to any
individual peer. One of the easiest ways to get that traffic level
is to pull peering with a bunch of transit customers of the network
you need to increase traffic with, which of course increases your
reliance on that paritcular peer.

One could wonder if some large providers pushed C&W due to the
lack of traffic between their networks (since we know C&W had some
issues where they couldn't grow their network last year, and had
trouble turning up new customers) and that wasn't one of the
catalysts for this most recent action.

6

I almost got caught by this one a few months ago. I was fixing to sign a contract with Exodus for a 100bT circuit when I noticed some funny-looking language and asked some probing questions, and then realized that I had to double their quoted rates before comparing them to everyone else. This moved them from the front of the pack back to UU-land. UUNet is another story. They not only charge significantly more than everyone else, but they calculate 95th percentile on the higher of incoming and outgoing rather than the average. When I asked my salesperson why she couldn't give me a competitive rate, she said "Because we're UUNet." She seemed pretty taken aback when I explained to her that UUNet actually had a pretty bad reputation in NetAdmin circles and I wasn't interested in paying a premium for their name. She still declined to give me a competitive rate.

I hear that InFlow charges for average traffic rather than 95th percentile. They're not a backbone, but I wouldn't be surprised to see backbone networks start doing that before too long. It would require some excess capacity, but they would probably make more money in the long run.

7

so, if it is primarily the content providers that these "de-peer"
rampage folks have their middle finger directed toward; if hosting
provider brand X does content caching close to peering locations
(thus the return path very likely being symetrical and neither party
is hauling the traffic farther than "necessary"), will the "de-peer"
folks consider different ratios or no ratios at all? ignoring the
how/what part of doing the caching.

Sat, Jun 02, 2001 at 02:24:41PM -0400, Leo Bicknell: