Typical last mile battery runtime (protecting against power cuts)

Hi folks,

At $day_job, I have a team of engineers who are oncall for critical services in the United Kingdom. For $reasons, the national power grid is announcing the possibility of rolling power cuts over the coming months. Right now it's "unlikely", but possible. If cuts do happen, it'll be 3+ hours, possibly several times/day.

I'm looking at the cost/benefit of deploying small UPSes at people's homes, to protect their network access when oncall. Just to power the home router (+ONT if FTTP), and keep a charged laptop. I figure anything smallish should be enough for a few hours.

Question is, how much battery runtime can I typically expect from ISPs' last mile infra.

People will have a random mix of DSL, FTTP, DOCSIS. Another alternative is tethering with 4G.

- For FTTP, I *think* (but am not sure) that the UK mostly uses PON, so guess it would be runtime of OLT and onwards
- For DSL: runtime of DSLAM cabinet and onwards
- For CATV: CMTS and onwards, maybe any active equipments in the HFC to the CPE?
- For 4G: BSS and onwards

Could anyone with last mile experience help with some ballpark figures? I.e. 15 min vs 8h or 8 days.

Thank you,
Israel

Hi,

At $day_job, I have a team of engineers who are oncall for critical services in
the United Kingdom. For $reasons, the national power grid is announcing the
possibility of rolling power cuts over the coming months.
Right now it's "unlikely", but possible. If cuts do happen, it'll be 3+ hours,
possibly several times/day.

They have been discussing it here in Italy as well.
The isp/telecommunication industry here is tryng to get Cos/pops/cabinets listed as critical infra and removed from rolling power cuts.

Question is, how much battery runtime can I typically expect from ISPs'
last mile infra.

- For FTTP, I *think* (but am not sure) that the UK mostly uses PON, so guess
it would be runtime of OLT and onwards

Here this is mainly ran from pops that have ups and generator systems so several hours to days of uptime depending on site.
OTOH I have seen providers daisy chain customer sites in a ring that crash miserably when 2 customers loose power isolating all in between sites.
But that is not the norm...

- For DSL: runtime of DSLAM cabinet and onwards

Street cabinets for fttc services here have low times if any.
Same thing for mini dslams mounted on poles in the middle of nowhere.
0 to 2 hours for these.
Most have batteries/capacitors in the cabinet but not all and they are not designed for extended power outages 2 hours max.
Some are remotely powered from the CO, but that does not seem to be a thing anymore. Too costly
DSL ran from COs are protected as for fiber above.

- For CATV: CMTS and onwards, maybe any active equipments in the HFC to
the CPE?
- For 4G: BSS and onwards

Don't operate a 4g network, so take this info accordingly , but here it depends on the tower from what I have seen.
All towers I have seen have battery backup , a lot have generators too.
I would say they have higher times than the fttc times above.

HTH

Brian

This would be highly market specific. In many cases, probably most
cases, there is no regulatory requirement for availability for
internet service whatsoever.

One specific case where it is regulated, Finland, the regulation is
available in Finnish, Swedish and English, the English document is
available at:

It classifies service to five priorities with different availability
requirements. From your ballpark, 8h would be the closest fit, but in
theory the higher priorities have indefinite availability before the
system is exhausted by means of generation.

In practice I would default to expecting 0 min availability during
power outage, regardless of how resilient my CPE is. We can scarcely
make the Internet work at the best of times.

Hi,

I'm looking at the cost/benefit of deploying small UPSes at people's
homes, to protect their network access when oncall. Just to power the
home router (+ONT if FTTP), and keep a charged laptop. I figure anything
smallish should be enough for a few hours.

Living in an area served by PG&E, I've had my share of power cuts. At home
I have a 600va UPS that protects my cable modem, RPI router, and POE switch
which serves 2 APs. That lasts about 30 minutes, which gives me enough time
to fire up my generator.

Tip of the day: I also have a 1000va UPS that protects my garage door opener.
This makes it a lot easier to a. get a car out if needed, and b. get my
generator out of the garage.

Lastly, in the spirit of happy wife, happy life, I have another 600va UPS
that covers my tankless water heater. It heats using natural gas, but the
control panel still needs power. That thing lasts pretty long.

Question is, how much battery runtime can I typically expect from ISPs'
last mile infra.

YMMV, of course, but I went through numerous outages recently. And by
numerous, I mean enough for our City leadership to get pissed off at PG&E
and demand explanations.

So far, my current ISP (Spectrum cable) has had 0 outages as a result of
power loss. Which is pretty impressive, given the instability of the grid
in this area.

Thanks,

Sabri

Question is, how much battery runtime can I typically expect from ISPs' last mile infra.

See comments inline, this is my experience in the US. UK may be substantially different.

People will have a random mix of DSL, FTTP, DOCSIS. Another alternative is tethering with 4G.

- For FTTP, I *think* (but am not sure) that the UK mostly uses PON, so guess it would be runtime of OLT and onwards

This is likely to run indefinitely. OLTs are for the most part going to be in telco central offices and datacenters with reliable generator backup. PON outside plant covers long distances passively, not requiring any local power.

- For DSL: runtime of DSLAM cabinet and onwards

If fed directly from the central office, indefinite. If fed from a remote terminal, maybe a couple of hours assuming that there are batteries installed at the terminal (and that the batteries are maintained).

- For CATV: CMTS and onwards, maybe any active equipments in the HFC to the CPE?

DOCSIS/CATV is likely to go dark pretty much instantaneously on loss of local power unless your customer is very close to the HFC head-end. Outside cable plant is fed with AC power injected locally at power poles or at pedestals into the center conductor of the coaxial trunk line. This powers the trunk amplifiers and fiber-to-coax media converters. This isn't typically backed up at all. Cable companies figure that if the power is out nobody is watching TV or using the Internet. Sometimes there may be a small UPS but I wouldn't expect more than a few minutes runtime.

In the case of prolonged outages I've occasionally seen a small Honda generator chained to a pole feeding one of the power injectors but this was very obviously a manual process done a while after the outage started where the power outage was localized and there were probably some VIP customers unaffected by the power outage.

- For 4G: BSS and onwards

Cell sites typically have generator backup. Figure at least a day. A prolonged outage requiring refueling with weather making it difficult to get fuel to the site(s) could be a problem.

Living in South Africa where load shedding is the order of the day since the end of last year (and to continue for the next 2 years, at least, if not more), I can tell you that until network operators have had to deal with this, they are ill-prepared.

For equipment inside major data centres, you will be fine. But for equipment in commercial buildings, street cabinets, e.t.c., uptime will be directly related to how much faith operators have put into the national grid, which translates to some UPS, and whether the building or cabinet is serviced by a well maintained generator.

As for the home, a UPS is not a terribly good idea to keep the "wi-fi" going... typical UPS's use Lead Acid batteries which are shallow-cycle batteries that are not meant to be discharged below 50%. It's the usual disclaimer - UPS's are meant to give you time to save your work and shutdown, not provide extended backup.

If customers expect the power to be out for 3+ hours at a time, multiple times a day, I'd recommend getting a cheap 1.6kW inverter that can power a 24V 100Ah Li-Ion battery (either 1x 24V or 2x 12V in series). It's not a lot of energy, but if you are powering the CPE + ONU + IP phone, it's more than enough (you can easily squeeze 9hrs of continuous run-time on a single charge, maybe even more). While these inverters are slow to charge Li-Ion batteries (about 10A of charge current), the load is so minimal that the battery is being discharged even slower. So it works out, even with extended, multiple outages in a 24-hour cycle.

Operators running gear outside of major data centres will want to invest in large Li-Ion packs designed for the load and frequency of grid outages. Investing in Lead Acid batteries, while cheaper initially, will become operationally expensive, as they don't do well with high-cycle counts, even the deep discharge ones. Not to mention, the rather poor energy density they possess for the amount of weight they carry.

Depending on the importance of the site, some solar power may be necessary, even though the UK is not the most well-lit country in the world. For such cases, you can augment the solar plant with a generator, mainly to recharge the batteries, rather than to power the load.

Mark.

They have been discussing it here in Italy as well.
The isp/telecommunication industry here is tryng to get Cos/pops/cabinets listed as critical infra and removed from rolling power cuts.

I would say plan for the worst, because there will always be some other department or governmental function that says they are more critical than the next one. And even if they may designate certain functions as critical, electrical distribution grids are not always built to that degree of granularity, and those critical functions are often caught in the crossfire as other non-critical loads are shed from the network.

And even after all that, if the blackouts need to intensify, all bets are off, since the main purpose then becomes preventing grid frequency drop, rather than servicing loads.

Here this is mainly ran from pops that have ups and generator systems so several hours to days of uptime depending on site.

If major data centres need to spend more on fuel than they planned for, I'd suggest making a generous allocation for an increase in co-lo costs for your next budget cycle, as the data centres will, invariably, pass those costs on the longer the city has to shed power from the grid.

OTOH I have seen providers daisy chain customer sites in a ring that crash miserably when 2 customers loose power isolating all in between sites.
But that is not the norm...

This is one of the reasons we refuse to turn customer sites into Metro-E PoP's, or PoP's of any kind.

Street cabinets for fttc services here have low times if any.
Same thing for mini dslams mounted on poles in the middle of nowhere.
0 to 2 hours for these.
Most have batteries/capacitors in the cabinet but not all and they are not designed for extended power outages 2 hours max.
Some are remotely powered from the CO, but that does not seem to be a thing anymore. Too costly
DSL ran from COs are protected as for fiber above.

A lot of this will be driven by what competitors do. If there is no competition that can keep their street cabinets going, the others won't. It's a great opportunity for anyone willing to make lemonade out of the situation.

Don't operate a 4g network, so take this info accordingly , but here it depends on the tower from what I have seen.
All towers I have seen have battery backup , a lot have generators too.
I would say they have higher times than the fttc times above.

In dense metro's, mobile sites will be well invested. It starts to get tricky when you go out into the sticks.

Also, when the power goes out, so does the wi-fi. That means everybody moves their traffic away from the home wi-fi and on to the nearest cell tower. While radio bandwidth and signal coverage does not suffer that much, it hits the backhaul hard, between the tower and the mobile carrier's core. So nice flashing LTE/4G/5G signals on your phone translates to GPRS-esque performance. In some cases, we have seen mobile operators downgrade radio coverage to 3G, in order to manage this. Who knew 3G performed a bad as GPRS or EDGE, in 2023 :-)?

In the most extreme of case, the cell site could run out of power as well, as there isn't enough time for the batteries to recharge between outage cycles, or the field teams can't replenish fuel for the generators in time.

In the absolutely extreme cases (as we see here in South Africa, for example), cell sites can be raided and batteries stolen, especially if there is darkness all around. I would not expect this to be the case in the UK, especially if power outages are not the norm and people live a fairly middle-class life, but if I were Vodafone, for example, I'd have my risk department planning for such an eventuality already.

Mark.

Agreed, this is a good place to start. It's a bit doom & gloom, but most people underestimate just how much work it takes power companies to generate and distribute energy. Consequently, they never have to think about generation and/or load management, if they had to do it on their own, meaning they will usually consider the cheapest solution possible, which for electricity, usually ends in tears.

Mark.

Living in an area served by PG&E, I've had my share of power cuts. At home
I have a 600va UPS that protects my cable modem, RPI router, and POE switch
which serves 2 APs. That lasts about 30 minutes, which gives me enough time
to fire up my generator.

I'd assume it doesn't take you that long to fire up the genie, if you are home when the power goes out :-).

Out of interest, depending on how long you've had the UPS, how many times have you changed the battery?

Tip of the day: I also have a 1000va UPS that protects my garage door opener.
This makes it a lot easier to a. get a car out if needed, and b. get my
generator out of the garage.

In South Africa, garage door motors historically come standard with a 12V 7Ah Lead Acid battery. What most people don't realize is that within 1.5 to 2 years, those batteries are dead, and since there was power most of the time, they never noticed, until the power went out and the battery did not have sufficient energy to drive the motor.

It is not uncommon to see folk here moving to Li-Ion batteries with the same capacity to drive garage and gate motors. Personally, I try to only use Li-Ion packs in well ventilated, not-too-tight spaces, that come with a BMS and a charger that is predictable :-). So I avoid these little ones, and feel safer with Lead Acid batteries for garage and gate motors.

Lastly, in the spirit of happy wife, happy life, I have another 600va UPS
that covers my tankless water heater. It heats using natural gas, but the
control panel still needs power. That thing lasts pretty long.

There are tankless water heaters that can take standard AA batteries to spark the igniter as well, but yes, if yours is electric, makes sense to put a UPS on it. At my place, I use both a PV-based controller to heat traditional water tanks, and also have just as many tankless water heaters. The former is for end-of-day showers and dish washing once the sun has done its thing, the latter is for the morning jobs for the same things, or if it's cloudy/rainy outside.

Even though I have whole-home backup, I did place a UPS on the PV-based controller, because even if it is connected to solar panels, it is still, in essence, an inverter, meaning it requires a grid before it can generate solar power. That is what the UPS does for it. It was my first deployment of renewable energy, so I needed the UPS before I backed up the entire home.

YMMV, of course, but I went through numerous outages recently. And by
numerous, I mean enough for our City leadership to get pissed off at PG&E
and demand explanations.

Judging by our situation, I'd take the city leadership demanding explanations any day :-). We are way past that, down here. It's so bad, folk are deploying self-generation and self-storage by the truck loads. It's the gold rush for solar and battery installers right now. People have all but given up on the power company (Eskom) ever going back to its glory days.

So far, my current ISP (Spectrum cable) has had 0 outages as a result of
power loss. Which is pretty impressive, given the instability of the grid
in this area.

Not bad.

Mark.

If it's just a little gasoline generator, 30 minutes is about right.
It takes 10 minutes to decide the power isn't coming back soon and
another 10 to drag the generator out of the shed, hook up the wires
and get it going even though it's cold, wet, and hasn't been run for
several months. That leaves 10 minutes to spare figuring out how to
convince the UPS that the generator power is good enough to retransfer
and stay.

Regards,
Bill Herrin

Indeed - I was guessing given how reliable PG&E have been for Sabri, a lot is probably pre-wired. I may be wrong.

Mark.

Pre-wired makes it a standby generator, which 9 times out of 10 is
automatic start with an automatic transfer switch. It's running within
seconds whether you're home or not. Electricians cost too much and
20kva natural gas / propane generators with an ATS don't cost enough
more than the portables.

Compare:

and:

understanding that an electrician will cost you $2000-$3000 for the
labor with any genset modification to the house wiring.

Regards,
Bill Herrin

Aware of all this - I operate one or two submarine cable landing stations...

What I mean by "pre-wired" is that, perhaps, the generator is pre-setup and wired into the house, but is not in standby mode to manage costs, and perhaps, to be reliable since ATS's are often dodgy.

Maybe a manual start is required. Maybe a changeover switch has to be flipped. That sort of thing.

Of course, we are speculating, and Sabri can answer best.

Mark.

Not for more than a decade now, at least not in the U.S. When you're
up to whole-house generator prices everyone expects electric start.
Half the portables have electric start. Most lawnmowers have electric
start. Once you have that, the cost to make the switch automatic
instead of manual is trivial. And the mass-produced consumer-grade
switches are quite reliable.

Regards,
Bill Herrin

Fair point, but it is not uncommon (YMMV) for backup solutions to not be whole-home, and this includes generators. I have not lived in the U.S. on any meaningfully extended basis, so I can't speak to the degree to which folk that install backup power choose their cost/value matrix. But in many other places I've lived, especially where the power is frequently out, splitting the house loads into different panels a generator or small solar/battery system can support is commonplace; mainly for convenience but also to reduce the chances of a fire or electrical shock from having to run wires ad hoc.

The same would also apply to renewables, especially where budget is limited and folk can't afford to backup the entire home.

This is often cheaper than investing in large backup solutions, while still providing some degree of convenience to power what one would consider critical items, whatever that means to them.

I've been helping quite a number of folk wire small inverters with limited power and backup battery time into DB boards that feed only lights and plugs to specifically drive wi-fi, TV, an IP phone and a fridge. The inverters and accompanying battery aren't terribly expensive here (US$500 - US$800 for a Lead Acid system, and up to US$1,400 for a Li-Ion one), and labour in Africa is dirt cheap (less than US$100 for an installation), so it's budget-friendly, and keeps basics going. One could even charge a laptop.

In general, the main things folk will not backup are electric stoves, electric ovens, electric water heaters, electric space heaters and air conditioners. Obviously, in places with winter periods, serious plans have to be made to avoid death from cold.

Mark.

Hi Mark,

Living in an area served by PG&E, I've had my share of power cuts. At home
I have a 600va UPS that protects my cable modem, RPI router, and POE switch
which serves 2 APs. That lasts about 30 minutes, which gives me enough time
to fire up my generator.

I'd assume it doesn't take you that long to fire up the genie, if you
are home when the power goes out :-).

Yes, there have been times where I wasn't at home.

Out of interest, depending on how long you've had the UPS, how many
times have you changed the battery?

All the "small" ones, I bought in 2019, they still work fine. I have one larger
UPS for my homelab in my garage that I've had since 2014; I changed the
batteries in that last year.

Tip of the day: I also have a 1000va UPS that protects my garage door opener.
This makes it a lot easier to a. get a car out if needed, and b. get my
generator out of the garage.

In South Africa, garage door motors historically come standard with a
12V 7Ah Lead Acid battery. What most people don't realize is that within
1.5 to 2 years, those batteries are dead, and since there was power most
of the time, they never noticed, until the power went out and the
battery did not have sufficient energy to drive the motor.

Those must be different from ours, because we don't have that...

So far, my current ISP (Spectrum cable) has had 0 outages as a result of
power loss. Which is pretty impressive, given the instability of the grid
in this area.

Not bad.

Pretty impressive. How do they do that in SA?

Thanks,

Sabri

Those must be different from ours, because we don't have that...

Even before we had power issues in South Africa, garage and gate motors had batteries in them. So it appears to be historical, for some reason or other.

Pretty impressive. How do they do that in SA?

Providing backup power is not an issue in major data centres.

GPON providers also do rather well in keeping the lights on during an outage. When outages were still relatively limited 5 or so years ago, you couldn't keep your home link up for more than an hour during an outage. Now with outages being more sustained, neither I nor a reasonable sample of folk I know have had their GPON fibre out during an outage, even when they occur multiple times in the day. I suspect decent use of batteries and generators.

Wireless ISP's seem to be doing just fine too, as they, obviously, have limited scale. Again, batteries and generators.

The mobile carriers suffer the most, as do operators who deploy Metro PoP's in commercial buildings that aren't consistent with backup power promises. The mobile operators are having to deal with battery theft and incomplete battery recharges between multiple outages in a single day.

Mark.

https://www.costco.com/honeywell-18kw-home-standby-generator-with-transfer-switch.product.4000106705.html

and:

Amazon.com

understanding that an electrician will cost you $2000-$3000 for the
labor with any genset modification to the house wiring.

I'd say I have something in between. I have a WEN GN875i: Amazon.com

That's 7kw rated and 8.75kw peak. More than enough to support my home.

I previously had one of those smaller 2200 watt generators. The problem
with those is that you're now limited to 1600 watt running, which barely
powers the fridge, lights, internet, and maybe some tv. Our power usually
goes out when it's very warm, so I like some AC.

What I mean by "pre-wired" is that, perhaps, the generator is pre-setup
and wired into the house, but is not in standby mode to manage costs,
and perhaps, to be reliable since ATS's are often dodgy.

Maybe a manual start is required. Maybe a changeover switch has to be
flipped. That sort of thing.

Mine is electrical (but not automatic) start, I have to flip the main and
a circuit breaker, which is protected by an interlock switch. Similar to

The interlock switch ensures that I'm not backfeeding to the grid, and
was necessary to pass inspection.

Usually I have it up and running within 10 minutes. That's how long it
takes for my UPS script to kick in and start shutting down servers.

Thanks,

Sabri

I'd say I have something in between. I have a WEN GN875i: Amazon.com

That's 7kw rated and 8.75kw peak. More than enough to support my home.

Yeah, plenty of juice.

I previously had one of those smaller 2200 watt generators. The problem
with those is that you're now limited to 1600 watt running, which barely
powers the fridge, lights, internet, and maybe some tv. Our power usually
goes out when it's very warm, so I like some AC.

Makes sense.

Mine is electrical (but not automatic) start, I have to flip the main and
a circuit breaker, which is protected by an interlock switch. Similar to
Amazon.com

The interlock switch ensures that I'm not backfeeding to the grid, and
was necessary to pass inspection.

Usually I have it up and running within 10 minutes. That's how long it
takes for my UPS script to kick in and start shutting down servers.

Awesome!

Thanks for sharing.

Mark.

My all electric house is in a rural area. The generator that came with the place is a 20KW Onan, The bad news is in can't handle the house. I think it is the Aux Heat on the heat pump that is the problem. I have to also power the well pump and the septic pump.

The one thing I made sure of was remote monitoring of the Utility power. I get an email and a text when the power goes out and when it comes back. Unfortunately the generator is not Internet aware.