UPS failure modes (was: fire at NAC)

ok, what UPSes do telcos use (besides their monster battery arrays)

-Dan

Date: Thu, 29 May 2003 16:53:43 -0400 (EDT)
From: Sean Donelan

Yep, tieing together "redundant" systems with parelleling
gears turns two independent systems into one "co-dependent"
system. In a failure situation, you want to compartmentalize
the failure. Loosing half your systems may be better than
loosing all your systems.

Too bad a substantial amount of equipment doesn't allow for
redundant plugins. The ability to plug { servers | routers |
whatever } into two totally separate power feeds is nice.

Anyone for building a rackmount transfer switch for two inputs?
Assuming it didn't fail (!) -- would the economies of scale work
for or against it compared to big transfer switches? Between
dealing with _much_ smaller current levels and the opportunity
for mass production, what are the chances of something like this
working?

Eddy

Too bad a substantial amount of equipment doesn't allow for
redundant plugins. The ability to plug { servers | routers |
whatever } into two totally separate power feeds is nice.

Anyone for building a rackmount transfer switch for two inputs?
Assuming it didn't fail (!) -- would the economies of scale work
for or against it compared to big transfer switches? Between
dealing with _much_ smaller current levels and the opportunity
for mass production, what are the chances of something like this
working?

I'm not sure if this is exactly what you're looking for, but:

http://www.apc.com/products/family/index.cfm?id=14&web_displayed=

List price is between $359 and $559, depending on if you want network management and how many outputs.

-- Kevin

Dan Hollis wrote:

ok, what UPSes do telcos use (besides their monster battery arrays)

What's wrong with our monster battery arrays?

-Jack

nicholas harteau wrote:

We run a configuration similar to this, except we do failure per-row
with one APC Symmetra supporting between 3 and 6 cabinets, depending on
the projected load. In the past 2.5(?) years, we've had one controller
failure that did not cause an outage. All the batteries are fine,
though we have not run them to zero. It turned out to be a very clean
install as far as conduit and cabling goes, we're very happy with it.

We do the same for one of our data centers, except we borrow one of the telco generator trucks due to lack of facility for an onsite generator. We have it setup to power both our Symmetra's as well as the telco's battery array one floor up. We've replaced two batteries and one powersupply in about 4 years (all recent) and had the Symmetra panic and warm reboot when electricity was being rerouted once.

The warm reboot was interesting. It reboot every APC MasterSwitch and 90% of our routers. 10% of our routers and all servers stayed live even though their MasterSwitch's reloaded firmware. Go figure. I wouldn't have minded except that the Cat had been up for over 3 years without a reboot and I really wanted someone onsite when that took place. :slight_smile:

-Jack

They dont tend to fit in 19" rackmounts

-Dan

Thus spake "Dan Hollis" <goemon@anime.net>

> Dan Hollis wrote:
> > ok, what UPSes do telcos use (besides their monster battery arrays)
> What's wrong with our monster battery arrays?

They dont tend to fit in 19" rackmounts

You wouldn't mount a "monster array" in 19" racks anyways.

Telco battery strings often take up entire floors, or at least significant
portions of them, and per building codes have to be enclosed (e.g. basement)
to contain fumes, leaks, or explosions. Things get a bit more complicated
safety-wise when you're talking 24-48 hours of capacity.

S

Stephen Sprunk "God does not play dice." --Albert Einstein
CCIE #3723 "God is an inveterate gambler, and He throws the
K5SSS dice at every possible opportunity." --Stephen Hawking

We are a Powerware house. We had a large number of 3kVA and 6kVA units in
our previous data centre (no-one would stump up the cash for a large unit
so we had to buy them as we needed them). After about 5 years (very rough
figure), we've now had 3 or 4 units fail, sometimes in the UPS, sometimes
in the bypass unit. They all seem to be component failures (in the case
of the bypass unit, a leg broke off a small capacitor). I don't think we've
replaced any of the batteries in that time and they're still all holding charge
well, even at full load.

Several data centers at which I have been involved in the maintenance of
have used Powerware UPSes. We've had great luck with them everywhere i've
been, and this includes service and support. The service I have received
from them (at least in the Southeast US) has been nothing short of
phenomenal, including calls from the reps years after they were purchased,
even on small installations, with promises made and kept time and time
again. I have never been disappointed.

I'd like to echo the results of other posters: We replaced Liebert with
Powerware in two or three of those installations and had great luck, where
the Liebert units would come out with spotty failures on a fairly regular
basis (components, etc).

Personally, i'd like to hear from more people who are doing serious work
with DC power in a collocation environment (with the market reality of
customers who want AC). I have seen customers request DC and nothing but.

Tim

Yes, I was specifically asking what telcos use when they dont use the
battery arrays.

Eg what do they use in remote cabinets or customer premise sites...

I thought it was a simple question, guess not.

-Dan

> Thus spake "Dan Hollis" <goemon@anime.net>
> > > Dan Hollis wrote:
> > > > ok, what UPSes do telcos use (besides their monster battery arrays)
> > > What's wrong with our monster battery arrays?
> > They dont tend to fit in 19" rackmounts
> You wouldn't mount a "monster array" in 19" racks anyways.

Yes, I was specifically asking what telcos use when they dont use the
battery arrays.

they use smaller battery arrays... There's a pile of fujitsu tdm gear with
four deep-cycle batteries in the basement of our facilty belong to
quest...

Well I did some checking off-list and this is what qwest uses at customer
premise sites:

http://www.marconi.com/html/about/mz5a50photos.htm

-Dan