[ On , April 19, 2001 at 21:35:30 (-0700), Sean Donelan wrote: ]
Subject: Re: How to game the system (was Re: What does 95th %tile mean?)
From a provider's perspective it does matter. The most costly part
of any network is the last 1% because most of the fixed cost of the
network is based on the peak capacity, not average capacity. Billing
based on average usage works best if you have a large population and
excess capacity. Generally its the largest of the large customers who
have the most to gain by gaming the system. A few T1 customers gaming
the system is just noise.
Hold on a moment here. We cannot eat our cake and have it too!
If an ISP is billing by 95th percentile peak bandwidth usage then they
cannot, by definition, be billing by 100th percentile peak bandwidth
usage at the same time (or even 99th percentle peak)! That last 1%, or
even 5% must get lost in the noise lest you over-charge your customers!
Indeed when the customer's pipe is much larger than his required
capacity a Nth percentile metering scheme is the only known fair way of
defining what the peak usage is!
Secondly if you don't have any excess upstream capacity then it doesn't
matter how hard any customer tries to pump, or pull, full line rate
through their port -- it's just not going to happen. When you're
upstreams are flat-lined then everyone gets his fair share of what
meager pickings are to be had and everything else either gets dropped on
the floor or is flow-controlled by higher level protocols.
Obviously 95th percentile peak usage billing works best for the ISP when
that's how your upstreams bill you too because then you'll likely have
excess upstream capacity, almost by definition. That way you make sure
you get paid for the peak usage you'll be paying for. However even if
your upstreams are already pegged (and you're not buying more bandwidth
because the numbers just don't make it affordable) then it'll still work
out OK (though you'll undoubtably be wanting to adjust your prices so
that you can afford more capacity). In the "pegged pipe" scenario your
customer's can't draw blood from a stone, not even the largest of the
large. That's what makes Nth percentile billing fair for both parties,
in fact -- they don't pay for what they can't get no matter how fat a
pipe you hand them.
On the other hand if a customer's trying to avoid paying for 100mbit of
usage even though all he could get from you was a 100mbit ethernet port
then he's almost infinitely more likely to use some form of rate limiter
(even if it's just ALTQ in a BSD router) to even things out properly
than he is to try and play games with some form of window juggling
against your counter sampler. A real rate limiter will adjust the
packet rates in such a way that higher level protocols won't complain or
suffer from packet loss, etc.. Heck unless your game-playing customer
is going to write some custom application to transfer bulk data using
UDP or similar they're only going to do themselves more damage than good
by bursting their pipe on and off to try to fool any sampling window.
Indeed you can probably even make your customers happier by selling them
a proper rate limiter that they can control!
In any case, just as I said, if your sample rate is around 5 seconds or
so then there's not bloody much anyone can do to push more data through
any pipe in such a way that they'll fool your rate calculations and
that'll adversely affect any other customers, unless maybe you've made
the mistake of selling a 100mbit pipe to someone who really does want at
least 50mbits at peak times and you've only got two or three other
customers total who only want a megabit or so each. That's about the
only scenario where 50% sample-rate traffic bursts would cause any
noticable effect on other customers. Even in that case those two or
three tiny customers are such a small part of your business that you
just can't be bothered to care about their streaming sessions looking a
little jerky. In this case you can probably even give your one big fish
a real rate limiter just to make everyone happier all around!
If you can't afford to sampe every five seconds then do it every minute
and change your Nth percentile to 97 or 98 instead of 95!
But you might have an OC48 customer who pegs their link requiring you
to add more peak capacity to your network. But times their traffic, so
their average usage never exceeds an OC24, and pays the average price.
This is the type of customer which nailed the electric companies.
That just does not compute. If their average use is OC24 then it does
not matter whether they push/pull OC48 for 2.5 seconds and zero for 2.5
seconds, or OC24 for 5 seconds. It all gets lost in the noise because
either way your other customers still get to use the other half of that
OC48's worth of bandwidth. You do not have to buy more peak capacity
for your upstreams if the peaks are only 50% of the width of your sample
period, at least not if your sample period is less than 5 minutes or so.
This is the Internet -- it's not a circuit-based telephone connection
network, and it's not ATM (though either may be used to transport IP, of
course). It's certainly not equivalent to a power distribution grid
where excess demand starts to look like a short circuit and causes real
damage. The traffic on the Internet is little bunches of packets that
can interleave between each other. Real-world traffic flows on IP
networks are incredibly variable and resilient, partly due to the heroic
efforts of higher level protocols such as TCP.