Discussing, or not discussing, major business outages

I am not going to name names, but I have it on good
authority that late this week a large hosting provider
went out of business abruptly. So far, the news that
it did so has not been discussed anywhere, that I can
tell, which is starting to disturb me. There have been
some significant dislocations related to this shutdown,
obviously, and this sort of event seems to me to be
something that involved parties should be at least
sending warnings out on nanog or other appropriate
discussion lists.

I and a third party ISP executive discussed this lack of
discussion some yesterday; their opinion is that businesses
and ISPs formerly hosted at the now defunct site are loath
to admit their connecitvity is down lest their customers
defect en masse causing a snowball of business failures,
when they will be up and stable via new providers shortly
and are not themselves fundamentally unsound. I understand
that logic but cannot entirely agree with it.

I would like to see the issue discussed in general terms
at least; what is appropriate for notifications, what are
fair responsibilities to customers, the public, other ISPs etc.
in terms of this sort of event.

-george william herbert
gherbert@crl.com I am not employed by, and do not speak for, CRL.

[SNIP]

I would like to see the issue discussed in general terms
at least; what is appropriate for notifications, what are
fair responsibilities to customers, the public, other ISPs etc.
in terms of this sort of event.

Having gone through this myself (as all of you here probably remember),
this can be a Most Difficult Time for everyone involved. Creditors
threatening to shut you down, and leaks only exacerbating the problem.
Customers worried about loosing their connectivity. Customers worried
about loosing their customers. Employees worried about their paychecks -
or worse, know there isn't one but trying to keep things going anyway.
Etc., etc., etc. It's really the bad side of this business that a lot of
people never see - thank god.

When a business goes under, the technical contacts of that business should
communicate with the technical contacts of the businesses *directly*
impacted by the possible failure. The business contacts of that business
should communicate with their creditors and customers regarding business
issues. Mailing lists and the like should only be notified after no more
damage can be done.

There really isn't an operational issue here. If a network is off-line,
and you are trying to get to a downstream, you should contact the
downstream. Just as if the network were having a normal outage, not going
out of business. Since the network will have notified the downstream (as
per my previous paragraph :), then the downstream can decide what is in
their best interests to tell you - IN PRIVATE. Not blasted out onto a
mailing list with a few hundred readers.

Don't get me wrong, I would really like to know about outages, etc., but I
can say from personal experience that "leaking" information can *serious*
endanger customers and employees - we're talking about PEOPLE here - that
had nothing to do with the failure. Sometimes a day or two can make the
difference, and posting to a mailing list only makes vendors want to cut
their losses immediately. Ruining some random person's livelihood, someone
who has worked hard and does not deserve to be put out of business, because
you were pissed at their upstream's NOC guy 6 months ago is bordering on
sadistic. (No, I don't know if that's why someone anonymously posted about
Priori. And yes, I'm still trying to find out what piece of sadistic shit
did that to my [ex]customers. You can insult, backstab and try to bring me
down, and all may be forgiven - or at least forgotten. But don't fuck with
my friends, because that I will never forget.)

So, if a network, hosting house, dial-up shop, or anyone else is going down
in flames, I do not think the list should be notified until #1) everyone
involved is notified and #2) everyone involved agrees no more harm can be
done. (We can discuss an addition to #2 like: "or everyone is shut down
and moved" or something like that.) Please note I said "everyone
involved", not "everyone on NANOG". Just because you run a network does
not mean you are automagically "involved" in the death of someone else's
business on the 'Net.

All IMHO, of course. :wink:

-george william herbert

TTFN,
patrick

I Am Not An Isp
www.ianai.net
ISPF, The Forum for ISPs by ISPs, <http://www.ispf.com>
"Think of it as evolution in action." - Niven & Pournelle

In a utopian world, the upstream of whoever had to close doors would
pickup the circuit keeping the affected customers "in business" and
extend them circuit offers wil the install waived or something.

A somewhat similar case like this, someone I know who was providing
a slew of services had to finally close shop. He's keeping his
circuir up for 30 days and giving his customers several options of
places to go who can give them a quick turn-around.

But alas, we live in a REAL world...

-r

In a utopian world, the upstream of whoever had to close doors would
pickup the circuit keeping the affected customers "in business" and
extend them circuit offers wil the install waived or something.

What if it is the upstream who is closing the doors?

Besides, that's not always possible. It is simple if we're talking about a
single-homed hosting house who is only late on their transit bill, but it
gets very complicated when multiple vendors, upstreams, utilities,
landlords, lease payments, contracts, employees, insurance, etc., etc. are
involved.

-r

TTFN,
patrick

I Am Not An Isp
www.ianai.net
ISPF, The Forum for ISPs by ISPs, <http://www.ispf.com>
"Think of it as evolution in action." - Niven & Pournelle