Process management

Customer Support and process management seems to be the Achilles' heel of
the Internet industry. This industry has seemed to build up a terrible
reputation for poor customer support. Coming from the Nuclear industry, I
am truly jaded when it comes to process management, but the Internet
industry seems to have taken a track as far as possible on the other side.

Ok, so I'm wierd. I enjoy reading the incident reports nuclear plant
operators have to file. One nuclear power plant operator discovered
the diesel fuel for their standby emergency generators was actually
syrup. How many ISPs actually test what's in that tanker truck before
filling up their generators? Or how about the time all the uninterruptiable
power supplies at a plant turned off in unison because of a bug in the UPS
firmware. How many ISPs would even know that other people had a problem
with a particular brand of UPS, and they should check theirs?

Has anyone, anywhere, ever really solved these problems? I don't know,
but I suspect not. The nuclear industry generates a LOT of paperwork, but
its just a bunch of dead trees if no one reads and learns from it.
The nuclear industry seems to suffer from the 'Its not my fault because
I followed approved procedures.' The internet industry seems to suffer
from the 'Its not my fault because no one controls the Internet.' The
airline industry suffers from 'Its always the captin's fault.' The
telephone industry suffers from 'Its not my fault, we're the telephone
company, we can't be wrong.' And so forth.

No one just 'knows' this stuff. The cliches do ring true. You either
learn it the hard way from your own mistakes. Or you learn it the easy
way from other people's mistakes. I'm not saying mistakes can be
eliminated. Too many useful discoveries have resulted from mistakes.
AS7007 was pretty nasty, but if we had learned our lesson, it might
have been worth it. The problem is we didn't learn our lesson.

We really need to work on communicating the lessons learned. The U.S.
Military has a wonderful web site on 'lessons learned' which they update
almost every day. If you are cynical, you might say that just shows the
military makes mistakes almost every day. On the other hand, if the
lessons are really learned, it means every day there is one less mistake
which will occur in the future.

I suspect one of the academic types would love to write a grant proposal
to operate a web site for Internet operations lessons learned. The question
is would anyone contribute anything to it. Doubtful after the first month.
And who gets the book deal with O'Reilly.

There apparently have been several organizations developed recently that
claim to be attempting to address this problem. However, I have not seen
an industry-wide affect by the operations of these organizations.

Then let's get together and form one. The standard operating procedure
for forming a new organization is everyone puts in $25,000 and the
organization puts out a press release announcing the start of a search
for an executive director. We can name W.C. Fields as an ex-officio
board member so we can say to tiny tykes, "Go away son, you're bothering

As you point out there are a number of organizations which have popped
up. However it seems to be a lot easier to get mega-corporations to
contribute funding in exchange for having their name on a 'worthy cause'
than it is to do something concrete.

With the millions of dollars ISP salespeople have told me they are spending
on their customer service center phone switches, why do I continue to have
such a problem getting an answer the question "What phone number should
I call if I want to report a problem?"