Customer AS

Sean Doran writes:

> We can't pretend to be able to replace the phone networks until we can
> achieve similar reliability. Phone networks typically are spec'ed to
> two minutes a year downtime.

Uh huh, while I agree with you that the Internet is not
remotely as stable as voice networks are, I think that
this has a great deal to do with the revenue difference.

I strongly disagree. It has far more to do with the primitive
engineering methods we are employing.

The mere fact that I've actually seen networks built in the real world
that simply never go down and that don't cost an order of magnitude
more leads me to believe that it can be done. I must admit that these
networks are much smaller than the national networks a large provider
tends to build -- on the order of twelve cities, fifteen or sixteen
sites, and a hundred or two hundred ethernets. On the other hand, such
a network is more than large enough that a "normal" network of this
sort will experience substantial downtime in some of its links. The
networks I've been involved with, though, go down orders of magnitude
less frequently than you normally expect. Why is that?

1) redundancy in all components and links.
2) No "cowboy"ism.

I'll describe these phenomena in reverse.

In a typical network, changes are made on the fly by networking
"cowboys" to router configurations. Very little care is taken, and the
attitude when customers get screwed is typically "oops, lets fix that
quickly!". This is Bad. Automated tools should be making the updates,
and only in response to carefully considered changes in policy. You
can build things like that -- I've seen it done.

Second, as I've noted before, most national providers simply aren't
putting in fully redundant networks. I can't understand this -- full
redundancy is something I've seen done at so many places -- and yet it
just isn't done.

If the money figures for Internet connectivity were anywhere
near the figures for voice, and was billed by use in small
increments, you would expect greater investment in reliability,
as one sees on voice networks.

But since making the network reliabile doesn't require an investment
so much larger than just building the network in the firstplace (it
requires discipline and redundancy), I don't see how this could be the

Frankly, where there is significant competition and little
or no loss-leading and cross-subsidy, the Internet
is precisely as reliable as people are willing to pay for.

People *would* pay for more reliability. I know -- I've seen people
switch providers on that basis. However, the real deep problem is that
there aren't providers selling reliability at the moment. This reminds
me of the way Detroit claimed, for years, that no one cared about auto
safety. Somehow, though, they "discovered" in the last decade that
people actually care deeply about safety and have started competing on
safety features.

I want to emphasize, though, that reliability isn't really that much
more expensive, and in fact, *unreliablity* is expensive -- it wastes
personel time, pisses off customers, and it usually implies
inefficient management practices that mean more human time is being
spent than is necessary.