OOB customer communications (Re: Looking for Support Contact at Equifax)

Twitter URL is an rss feed as well.

I could see someone complaining about the idea of letting a third party
carry outage info like that... at least in my environment.

Really, it could be a blessing or a curse for your marketing team, but it
does depend on how you handle it I suppose. Potential clients could use the
information to see your ability to meet SLA targets before they sign on. If
you're not the big dog in the arena, it can very easily work against you by
enhancing the BS that competitors will provide. Or, of course, it can give
them a warm fuzzy feeling knowing that they can readily see the issues and
know how you've been affected.

A lot of places still work with an MO of secrecy. The service I support is
one of them. From a technical perspective, the idea is solid though.

--WJM IV

William McCall wrote:

I could see someone complaining about the idea of letting a third party
carry outage info like that... at least in my environment.

How else do you propose getting outage information to your customers?

If the information provider is under your control (not a 3rd party), then whatever took out your primary services (internet services, phone services) might also take out your access to the outage channel that is under your control (e.g. fat-fingered admin messes up an ACL).

I *like it* when my service provider has an OOB network outage channel. My biggest complaint has been with networks that setup a channel like this but then get "too busy" during an outage to make use of it. If you are going to setup a channel like this, make sure you use it. Also, if you post a partial update, make sure you follow up with more information when you have it. Some of us read the archives to see if this information was posted and followed-up on in a timely fashion, to evaluate the outage reporting service record before signing up.

Suresh Ramasubramanian wrote:

If your email and phone communications are down due to a connectivity
break, and your customers get connectivity from you [assume no backup
links, by default .. you'd be surprised at how many smaller customers
get by with a single link and no backups at all. If their
connectivity is down too - they just cant get to twitter right?

Um... what about text messages to these newfangled cell phone thingies?

Unless you are AT&T or Comcast (or similar) and your customers have U-Verse or Comcast Triple Play (or similar) it has to be a fairly widespread outage to take out your customer's landlines, internet, and cell phones too. If your network has that big of an outage, your customers can follow the outage news updates on the radio.

jc

How else do you propose getting outage information to your customers?

I should have clarified. Third party physical control isn't necessarily the
issue, but third party administration and delivery (in the context of
twitter) is.

Dedicated servers are cheap and you can maintain control of the content. Its
not quite the same as using twitter or other third party SaaS that is
similar (which can, invariably, control the content at its whim and is a
nightmare to manage persons authorized to view such outage info, depending
on the service)

Or even a mailer that is outside of the scope of your service ops and permit
only customers to subscribe. Again, its more about distribution in these
environments. If I'm Company A, I really don't want to readily provide my
competitor, Company B, with information on outages and a full history of it
for them to use in some marketing device (which can't be compared because
Company B does not publish their info, but instead provides some nice
glossy-paper stats).

Physical control certainly can't be the question.. or we'd have the same
argument in circles, "If we have physical control, how can we ensure the
outage doesn't affect this net too? Better question, why can't we fail over
to the net that's working to send these notifications/updates for our down
services if the net isn't affected?" That point is moot.

My biggest complaint has been with networks that setup a channel like this

but then get "too busy" during an outage to make use of it. If you are
going to setup a channel like this, make sure you use it. Also, if you post
a partial update, make sure you follow up with more information when you
have it. Some of us read the archives to see if this information was posted
and followed-up on in a timely fashion, to evaluate the outage reporting
service record before signing up.

The way notifications should be distributed is in a proactive manner and
followed up as a ticket or some other relevant mechanism. Implementing a
process like this is trivial in many environments. Incident response should,
in most cases, include a mechanism like this that has already been deployed
today. Modifications (technically speaking) should not be a big issue.

--WJM IV

That should work then, and a reasonable solution for not having to
sign up for a third party service.

William McCall wrote:

I should have clarified. Third party physical control isn't necessarily the
issue, but third party administration and delivery (in the context of
twitter) is.

Dedicated servers are cheap and you can maintain control of the content.

But useless if the customer's data connection is down and their local cell phones are the only remaining method of communication.

If 25% of our users would check their twitter feed first and let their boss know "They are aware of the problem and this is the ETR", that means the other 75% who are trying to call have less competition getting that same update from a human (or our auto-attendant).

We're never going to tweet every down T1 because those are easy to manage the customer contacts for and also not of interest to 99% of our customers. I'm really only talking about the outages that would affect the majority of customers where proactive notification isn't feasible (by the time you've made it 10% through your list, you've already received calls from 95% of the people who want such notifications anyway because they all called at the same time, right when it stopped working...).

Mike

But useless if the customer's data connection is down and their local cell
phones are the only remaining method of communication.

If 25% of our users would check their twitter feed first and let their boss
know "They are aware of the problem and this is the ETR", that means the
other 75% who are trying to call have less competition getting that same
update from a human (or our auto-attendant).

Cellphones can still receive data and websites. You can use an alternate

service that provides SMS functionality without having to have it in a
public forum. This still allows you to maintain control of who is seeing it
without trying to figure out which customer BigDaddyPimpin is on Twitter.
Depending on your situation, you really might not want all of those outage
updates to become someone else's information. If you don't have much
competition against your business model, it might make sense to provide
this. If you have competitors ranging from sleazy snake oil to overpriced
big name, this data becomes a good source of your operations for your
competitors to use in whatever way they like.

We're never going to tweet every down T1 because those are easy to manage
the customer contacts for and also not of interest to 99% of our customers.
I'm really only talking about the outages that would affect the majority of
customers where proactive notification isn't feasible (by the time you've
made it 10% through your list, you've already received calls from 95% of the
people who want such notifications anyway because they all called at the
same time, right when it stopped working...).

Mike

Proactive notifications can and are tailored to meet individual SP needs.

If I'm providing circuits, my customers want to know when THEIR circuits are
affected. If I'm providing some other service, relevant notification of
failure are useful. Tailoring outage information to be proactive and
relevant isn't something new and the practical implementation has already
been deployed by major players in the SP realm. The implementation of these
systems may be non-trivial in your environment, but customers are starting
to demand a notification of potential trouble BEFORE someone has spend 15 or
20 min to determine that the service is truly down.

And if your argument holds true, those 95% didn't bother to check twitter.
They called you anyway. And remember, you still have that website thats
hosted offsite, offnet. And you still control it. Twitter doesn't magically
change the way information is delivered just because its twitter. It does
change who is ultimatly in control of the content.

--WJM IV