Using twitter as an outage notification (was: Fire, Power loss at Fisher Plaza in Seattle)

Paying a lot more to host the website with higher "burst" capacity
during an emergency, isn't an option.

The only other idea I've had is to sign all the customers up to receive
an SMS via some sort of broadcast service (the news will fit easily in
one SMS).

If the event is suitably calamitous we will do that for you -

http://news.bbc.co.uk/1/hi/help/5194672.stm

brandon

In article <200907041222.NAA23352@sunf10.rd.bbc.co.uk>, Brandon Butterworth <brandon@rd.bbc.co.uk> writes

Paying a lot more to host the website with higher "burst" capacity
during an emergency, isn't an option.

The only other idea I've had is to sign all the customers up to receive
an SMS via some sort of broadcast service (the news will fit easily in
one SMS).

If the event is suitably calamitous we will do that for you -

http://news.bbc.co.uk/1/hi/help/5194672.stm

The "event" (typically closing a High School because of snow, but we have swine flu these days too) is currently reported mainly by local radio stations. However it doesn't scale - there are perhaps two hundred of them trying to phone in to one radio station during the same 15 minutes after they made the decision, half an hour before the school is supposed to open for the day.

Another problem with a literally "broadcast" system is that it takes them too long to read out the names of the schools which are closed, even if trying to cover just one county.

Nor does it matter to anyone except a particular closed group of perhaps 1000 households whether any one school is closed - so telling everyone is a bit of a waste.

So it seemed to me that a Tweet from the school would be an ideal solution.

But a system like yours, if it could be divided up into a few tens of thousands of SIGs (one for each school), is the kind of "more traditional" solution I was thinking about.

However it doesn't scale

Anyone who's seen the "fail whale" might argue the same about Twitter.

Cheers,

Michael Holstein
Cleveland State University

With a past week of highly visible outages in the data center/provider industry, take a look at the section: "Crisis Communications Moves Fast" in this story, for another view of using communications channels like Twitter. http://www.datacenterknowledge.com/archives/2009/07/06/the-day-after-a-brutal-week-for-uptime/

-- Nevin Lyne
-- CTO
-- EngineHosting.com

However it doesn't scale

Anyone who's seen the "fail whale" might argue the same about Twitter.

Cheers,

Michael Holstein
Cleveland State University

With a past week of highly visible outages in the data center/provider industry, take a look at the section: "Crisis Communications Moves Fast" in this story, for another view of using communications channels like Twitter. http://www.datacenterknowledge.com/archives/2009/07/06/the-day-after-a-brutal-week-for-uptime/

-- Nevin Lyne
-- CTO
-- EngineHosting.com

Just to add something to this, twitter has been slow all afternoon and now I am getting the "fail whale"

Twitter is over capacity.
Too many tweets! Please wait a moment and try again

I presume that this has something to do with the Michael Jackson Memorial Service now underway.

I just thought I would point out in real time the obvious danger of using a backup service that itself could fail under load,
especially if your outage and the load could be correlated, say in a disaster or public emergency situation.

Regards
Marshall

Regards
Marshall Eubanks
CEO / AmericaFree.TV

However it doesn't scale

Anyone who's seen the "fail whale" might argue the same about Twitter.

Just to add something to this, twitter has been slow all afternoon and now I am getting the "fail whale"

I just thought I would point out in real time the obvious danger of using a backup service that itself could fail under load,
especially if your outage and the load could be correlated, say in a disaster or public emergency situation.

yep, got that too several time, but everytime i "reload" the page it works , there were plenty of outages before twitter
got the 35 million cash injection , but your absolutly right , its centralisted , so there will be serverfarms over serverfarms to get over
these "event" peaks. Would a decentralised system like http://laconi.ca/ not a better choice ?

just my 50 cents

http://identi.ca/macbroadcast/

However it doesn't scale

Anyone who's seen the "fail whale" might argue the same about Twitter.

Just to add something to this, twitter has been slow all afternoon and now I am getting the "fail whale"

I just thought I would point out in real time the obvious danger of using a backup service that itself could fail under load,
especially if your outage and the load could be correlated, say in a disaster or public emergency situation.

yep, got that too several time, but everytime i "reload" the page it works , there were plenty of outages before twitter
got the 35 million cash injection , but your absolutly right , its centralisted , so there will be serverfarms over serverfarms to get over
these "event" peaks. Would a decentralised system like http://laconi.ca/ not a better choice ?

In a real crisis, redundancy rules.

Regards
Marshall

just my 50 cents

http://identi.ca/macbroadcast/
-- Les enfants teribbles - research / deployment
Marc Manthey
Vogelsangerstrasse 97
D - 50823 Köln - Germany
Vogelsangerstrasse 97
Geo: 50.945554, 6.920293
PGP/GnuPG: 0x1ac02f3296b12b4d
Tel.:0049-221-29891489
Mobil:0049-1577-3329231
web : http://www.let.de

Opinions expressed may not even be mine by the time you read them, and certainly don't reflect those of any other entity (legal or otherwise).

Please note that according to the German law on data retention, information on every electronic information exchange with me is retained for a period of six months.

Regards
Marshall Eubanks
CEO / AmericaFree.TV

... and simplicity.

It's always "fun" when those outages pages rely on sql backends etc, so they're capable of tens or hundreds of users, so they look fine normally. When an outage happens and people really need the information and want it, things stop working.

I've been advocating a distributed system with static HTML pages being generated and pushed out when things change. Huge load capability, you can put it anycasted at multiple IXes so it's geographically and ISP resiliant, larger ISPs can even request to get their own mirror. Keeping it simple.

No takers yet though, people seem to have too much confidence in complicated, centralized, nice looking solutions.

In a real crisis, redundancy rules.

... and simplicity.

It's always "fun" when those outages pages rely on sql backends etc, so
they're capable of tens or hundreds of users, so they look fine normally.
When an outage happens and people really need the information and want it,
things stop working.

I've been advocating a distributed system with static HTML pages being
generated and pushed out when things change. Huge load capability, you can
put it anycasted at multiple IXes so it's geographically and ISP

resiliant,

larger ISPs can even request to get their own mirror. Keeping it simple.

No takers yet though, people seem to have too much confidence in
complicated, centralized, nice looking solutions.

--
Mikael Abrahamsson email: swmike@swm.pp.se

http://www.coralcdn.org/

In a real crisis, redundancy rules.

... and simplicity.

It's always "fun" when those outages pages rely on sql backends etc, so they're capable of tens or hundreds of users, so they look fine normally. When an outage happens and people really need the information and want it, things stop working.

I've been advocating a distributed system with static HTML pages being generated and pushed out when things change. Huge load capability, you can put it anycasted at multiple IXes so it's geographically and ISP resiliant, larger ISPs can even request to get their own mirror. Keeping it simple.

This would seem to be ideal for P2P, which is decentralized and has proven quite resilient under attack.

No takers yet though, people seem to have too much confidence in complicated, centralized, nice looking solutions.

Have you talked to the guys at BitTorrent ? I could make introductions during the Stockholm IETF if you need them.

--
Mikael Abrahamsson email: swmike@swm.pp.se

Regards
Marshall Eubanks
AmericaFree.TV

Nice, looks very much like the thing I was advocating. Hard part is getting authorities et al interested in such an "ad hoc" solution. Preferrably they could do both and then we can see which one works best in an emergency :stuck_out_tongue: