Telecom Collapse?

Mike Lyon wrote:

That makes two of us...

Anyways, for residential VOIP, where are we these days with E911? Are
providers like Vonage and such providing reliable E911 when people
call 911? That is one of the major problems I see with the residential
realm going with VOIP offerings...

Where we are, the SLC units on the telephone poles have batteries. Until
very recently, DEAD batteries. We'd lose power, and the POTS line would
go out. We've got our own genset and UPSs to bridge the gap, so we kept
power, the cable Internet service stayed running, and the Vonage VOIP.
The only thing NOT working was POTS.

After many calls to Verizon, most of which were met by people telling me
I was an idiot, I filed a complaint with the Commonweath and had our
police chief file a complaint with the E911 folks he interfaces with.
Suddenly, Verizon wanted to help. A manager came out and asked what was
up. I suggested he get the batteries changed, and get a program in place
to regularly test and replace them. It got done. But for over 3 years,
when the power went out, 911 wasn't available. Of course the power
usually goes out during bad ice storms, when people are more likely to
be hurt.

We're going to give up the POTS line because it costs a lot for poor
service. Verizon has only themselves to blame. But I do expect them to
show up in Washington, DC, since all the other big companies are lining
up for their handouts. Politicians say small businesses are the backbone
of the economy, but clearly when it comes to buying members of Congress,
the big companies have the money to spend.

Chris Adams wrote:

Once upon a time, Paul Stewart <pstewart@nexicomgroup.net> said:

What worries me the most is a power outage longer than say 8 hours.
This is the typical battery time at most cell sites, telco remotes and
many telco CO's. Beyond those 8 hours, it's quite probable that the
site will go down and you'll have no cell or landline anyways.

The AT&T (BellSouth) remotes around here installed in the last 10 years
or so typically have natural gas generators installed, and the COs have
a pair of generators for redundancy. Even many of the cell towers have
generators. The telco infrastructure is pretty well backed up (I don't
know how well tested any of it is of course).

The ILECs that use my service have generators at the large sites and a number of generator trucks to make rounds recharging remote battery systems. Quite a few of them have permanent generators installed to power one or more remotes in the field. Some are still using remote power technologies on their remotes.

The storm that blacked out northern Oklahoma a couple of years ago left some towns without services for over a month. The phone system itself was impacted in a few of the ILECs, but they never dropped below 80% and most of that was due to actual line damage, not power. A few of the ILECS effected didn't even blink the entire time.

That being said, most small ILECs can cope better with the costs. It's easier to manage < 10 towns than it is to manage 100+ towns.

On the other hand, it appears that the cable infrastructure doesn't even
all have batteries (I know some people whose cable voice and data
services blink with the power).

None of the cable services I know of around here can make 8 hours, much less 1-6 weeks. Such outages are rare, but they do happen and Oklahoma takes more than it's fair share. Lots of modems support battery backup, but the cable plant itself is prone to power outages.

Jack

And it gets better:

AT&T to reduce workforce by 12,000 - AT&T Inc. will layoff 12,000 of its employees, or 4 percent of its total workforce, in response to recent economic pressures.

Sprint/Nextel has had negative net income of $326mm, $829mm, and $505mm for the last three quarters.

Verizon seems to be doing alright, about a billion in net income each quarter.

GBLX has had negative net income of about $70mm each of the last three quarters.

L3 has been doing slightly worse.

I am sure Q4 numbers aren't going to be great.

I hadn't thought about this, but it's going to be pretty interesting the next year or so.

This is straying far from network operations, but I think 911 generally engenders an unnecessary degree of hysteria. As I suggested before, the marketing of this fear from certain quarters has apparently been quiet effective.

The probability of any single individual needing to call 911 is already minute. The probably that all available cell and VoIP services also won't work precisely at the moment that a 911 call needs to be made is even smaller.

You're probably more likely to accidentally brutally stab yourself in the neck whilst combing your hair.

A sense of perspective in these things can be useful, in my opinion. Cries of "but think of the children!" are merely entertaining, in my opinion.

Joe

The article goes on to quote some other source regarding Hawaiian Telecom's
collapse. The *very first sentence* of the quote:

"Customers initially had complained about poor service."

I quit reading after that, as I could already see where this was heading.

Joe Abley wrote:

This is straying far from network operations, but I think 911 generally engenders an unnecessary degree of hysteria. As I suggested before, the marketing of this fear from certain quarters has apparently been quiet effective.

Many will agree with you; unless 911 saved their life. Of course, we could let those people die, I guess.

The probability of any single individual needing to call 911 is already minute. The probably that all available cell and VoIP services also won't work precisely at the moment that a 911 call needs to be made is even smaller.

911 services are heavily used when a geographical area has an emergency, and that emergency usually includes not having power.

You're probably more likely to accidentally brutally stab yourself in the neck whilst combing your hair.

Unless you live in a natural disaster prone location. Or if your grandmother's alert bracelet requires a phone line for notification.

A sense of perspective in these things can be useful, in my opinion. Cries of "but think of the children!" are merely entertaining, in my opinion.

I agree. Think of the elderly! Think of the "This is where my neighborhood used to be." And not to leave the more stable big cities out, think of the looting and pillaging!

That said, I'd keep POTS and Cell available. I don't believe in single homing. :wink:

Jack

That the old ILECs are having problems due to the fact that few if any
of them know how to run a decent business is not exactly news. IMO, it
might be best if some of them were finaly placed in the position of
figuring out how to come into the 21st century and actually compete
for business.

But I agree with Alex... If we have another poorly run group of
businesses pleading for tax payer money, I think I'm gonna have to go
somewhere and lose my mind for a few days.

-Wayne

For my own $0.02 worth, I would like to point out the kind of engineering that was done during the days of Ma Bell - when it was THE phone company, and had the world in it's pocket - was quite spectacular and resulted in telecommunications systems that largely stood up and continued functioning despite the worst that could be thrown at it. But today, in our competitive (ahem) marketplace, the kinds of resources that made this level of engineering possible on such a wide scale, are no longer economically possible and is only infrequently done. Besides, most people readily accept failure. Except when it hurts them, that is, and then only after the fact is any kind of examination done and possibly steps taken to address the risk. But until then, people want cheap and that's the only selling point that matters. So your cable voip works until it doesn't, and nobody is responsible to you for it not working when you needed it to..... at least it was cheap....

I wasn't going to say anything, but as long as you brought it up ...

http://www.renesys.com/blog/2008/12/fiber-to-the-home-ideal-econom.shtml

Outlandish and bizarre, yes, but perhaps no more so than the other things
you read in the
papers these days? --jim

I think we've figured out the next get together for the next nanog.
Make sure there is a gun range within an hour drive....

Daniel Senie wrote:

Mike Lyon wrote:
  

That makes two of us...

Anyways, for residential VOIP, where are we these days with E911? Are
providers like Vonage and such providing reliable E911 when people
call 911? That is one of the major problems I see with the residential
realm going with VOIP offerings...
    
Where we are, the SLC units on the telephone poles have batteries. Until
very recently, DEAD batteries. We'd lose power, and the POTS line would
go out. We've got our own genset and UPSs to bridge the gap, so we kept
power, the cable Internet service stayed running, and the Vonage VOIP.
The only thing NOT working was POTS.
  
We run a fixed wireless business and with modern embedded hardware, that is designed to be installed on remote sites, like mast sites, we can for very little money add battery backup for one week (7 days !!) The cost of that is less than $200 pr. site and would power up to 4 routers easily.

As the west of Ireland has terrible power in the rural areas (as in daily power cuts), we've implemented the power backup everywhere. A minimum of 2 days.

In the regular winterstorms, when tree's fall into our overland telephone cabling, roads get flooded etc., we've had customers telling us, that the only thing that stays working for them, is the broadband from us. Some even ask us, how they can power the kit in an emergency and as our kit runs on anything from 10-28 volt, they can just hook it up to a car battery.

As for E911 or similar services, as mentioned before, there is always a cellphone. Any GSM provider is enforced to provide 911/112 services as part of the license, even to phones that have no sim-card in it. And all of the phones allow you to call 911 and 112 without a sim-card.

That's for some people, that can't get a phoneline, the only way of having E911/112 services.

Pots will often fail during powercuts, especially if you are sitting on a pair gain/multiplexer.

Kind regards,
Martin List-Petersen

That the old ILECs are having problems due to the fact that few if any
of them know how to run a decent business is not exactly news. IMO, it
might be best if some of them were finaly placed in the position of
figuring out how to come into the 21st century and actually compete
for business.

Having finally broken away from the local ILEC and moved to more fertile grounds, I can concur with the above. Concentration these days seems to be on maximizing profit and bonuses for execs while stripping away every possible expense so the books look good and make the company more desirable. One of their latest schemes was to give away additional lines to customers, pumping up the overall access line count. From what I can tell, a higher access line count increases the "worth" of the company.

New technology, or, rather, a mandate for new technology was there, but without a decent budget, there is no way to even come close to meeting that mandate. And that's where they remain today. Unfortunately, I think in the end, the company will be sold and the execs will get their big bonuses, but that seems to be the way of things these days.

But I agree with Alex... If we have another poorly run group of
businesses pleading for tax payer money, I think I'm gonna have to go
somewhere and lose my mind for a few days.

It wouldn't surprise me in the least if they started begging for a cut. I think everyone these days has bailoutitis.. I'll gladly take a cut.. I'm easy, though. A few million will surely carry me through the next few years.. :slight_smile:

The marginal cost of POTS service isn't subsidized by other services;
at the margin, POTS is profitable. The subsidy covers some of the
fixed costs (but not all of them, some of the fixed costs are covered
by POTS revenues). So ... every time a POTS line is taken out, the
fixed costs that were being covered by the revenue from that line now
have to be covered from somewhere else (= More Subsidy).

     -- Brett

Joe Abley wrote:

This is straying far from network operations, but I think 911 generally engenders an unnecessary degree of hysteria. As I suggested before, the marketing of this fear from certain quarters has apparently been quiet effective.

The probability of any single individual needing to call 911 is already minute. The probably that all available cell and VoIP services also won't work precisely at the moment that a 911 call needs to be made is even smaller.

   We haven't really had a major catastrophe where we've been totally
   dependent on IP yet, AFIAK. Maybe all of the qos, call gapping and
   the rest of the stuff the TDM networks do to deal with disasters
   will be left in the dustbin of Moore's Law, but maybe they won't. One
   thing is certain: we'll definitely find out one day, and it's not
   likely to be from a position of having taken the precautions,
   congratulating ourselves IMO.

    Mike

http://www.nap.edu/catalog.php?record_id=10569 is probably worth
reading.

    --Steve Bellovin, http://www.cs.columbia.edu/~smb

I am not sure. Business lines are significantly higher priced than residential lines and the conventional wisdom was that there is a cross sudsidy. How it shakes out across all phone lines is unclear to me.

A lot depends on the economic realism of depreciation schedule. I'm not familiar with how plant is depreciated.

An interesting issue related is the book value of assets.

Does anyone believe that the book value of telecom assets approximates its market value?

In other words, I suspect at least some of the competitive players are insolvent (negative net worth) since their physical assets would only fetch a pittance in a Chapter 7 auction.

And yes, I decline to identify specific cases. :slight_smile:

Regards,

Roderick S. Beck
Director of European Sales
Hibernia Atlantic
13-15, rue Sedaine, 75011 Paris
http://www.hiberniaatlantic.com
Wireless: 1-212-444-8829.
French Landline: 33+1+4355+8224
French Wireless: 33-6-14-33-48-97.
AOL Messenger: GlobalBandwidth
rod.beck@hiberniaatlantic.com
rodbeck@erols.com
``Unthinking respect for authority is the greatest enemy of truth.'' Albert Einstein.

The ILEC is the carrier of last resort. The wireless carrier doesn't have
to build coverage everywhere. They don't need to serve that hog barn that
requires a 10,000 feet copper loop while playing $17/month.

The problem is that whether the take rate for POTS is 75% or 95%, the ILEC
still needs to maintain the plant, and capital expenses to maintain the
plant are a killer.

Either the FCC needs to release ILECs from their coverage obligations so
that they can do what CLECs have done and build to the most profitable
areas, or subsidize the plant for both POTS and broadband services.

Frank

They gladly hit you up for an FCC mandated universal service fee on your monthly phone bill precisely to fund those subsidies. I remain unsympathetic to their plight.

jms

I find it amusing that:
1. Many assume one is able to get POTS everywhere
2. How some use the term "POTS" when in reality they're referring to VoIP

Pardon the length, but to make the point, here's one of many Canadian examples some of us are intimately familiar with:
-Construction conglomerate starts up a CLEC
-Construction conglomerate doesn't permit ILEC into new subdivisions it's building in the heart of ILEC's territory, instead all POTS infrastructure including a "new CO" is built by its money-printing press...err, newly registered CLEC, which begins providing voice and data there
-ILEC's "mortal enemy", the local cableco, owns minor % of CLEC, and also happens to serve this new subdivision with its cable-based products
-A year passes. VoIP over HFC...pardon me, "Digital Cable Phone" is introduced. Cableco buys out remainder of CLEC.
-Cableco decides to throw out all the new equipment the CLEC has and begins forced migrations of customers to its VoIP...sorry, "IP Telephony" service over the cable network, refusing new POTS orders
-Cableco founder dies...oh wait, that's probably unrelated

Often MDUs (residential condominiums typically) here will create exclusive agreements with cablecos and others to provide "POTS" (POTS look-alike is often the result). But wait, cries the poor CLEC, what about my CRTC-given right of access to buildings so I can do the same thing?

You don't always have a choice. You just can't get "POTS" in such cases. If a change such as the one described happens, you simply have no choice but to move. The question then is, is the sole alternative equally as reliable? That seems to vary greatly on an individual basis.

If I'm just a user plugging in my 1980s Nortel phone into the same RJ-11 jack I had 10 years ago, it still looks like POTS with the same 911 reliability to me, right? Just because my provider runs the largest HFC network in the province, has at most four hours of battery at the nodes and even less at an MTA, isn't a LEC, doesn't have the ability to get anywhere close to interfacing with the PSAP, relies on a third party to do all 911 prov for them, this party happens to be a CLEC of questionable quality and possessing severely broken OSS, doesn't mean that I'm not perfectly safe nor that I can't call this system "POTS", right?

How about CLECs who put up a "CO" in the field (and literally in a field!) and have no clue on how to power it in such a way as to prevent 13 hour voice and data outages? That reminds me I still need to request credit for that Sunday in November. If you guys are on nanog and reading this, just send over the $, eh? :slight_smile:

So it can be argued both ways. Ultimately, it all comes down to marketing and hype. With everything going to IP at both the core and edge (yes, I chose the terms deliberately) and analogue-digital-analogue or TDM-IP-TDM-IP conversation happening so many times, the terms "POTS" and "VOIP" are becoming nothing but marketing speak open for abuse. Often, confused by marketing of the "big boys", the end users have no clue what they're using, especially when it's CPE-less like VoIP-behind-POTS or "hosted PBX" or FTTB or cable or even things powered by field equipment. A certain company here tells DSL folks they're on fibre and another one emphasizes to staff to refer to their cable phone service as "it's not VoIP, it's IP telephony" (I'm not kidding).

Regards,

Erik (Caneris) wrote:

So it can be argued both ways. Ultimately, it all comes down to marketing and hype. With everything going to IP at both the core and edge (yes, I chose the terms deliberately) and analogue-digital-analogue or TDM-IP-TDM-IP conversation happening so many times, the terms "POTS" and "VOIP" are becoming nothing but marketing speak open for abuse. Often, confused by marketing of the "big boys", the end users have no clue what they're using, especially when it's CPE-less like VoIP-behind-POTS or "hosted PBX" or FTTB or cable or even things powered by field equipment. A certain company here tells DSL folks they're on fibre and another one emphasizes to staff to refer to their cable phone service as "it's not VoIP, it's IP telephony" (I'm not kidding).

Regards,
--
Erik
Caneris

None of the above matters if the supposed POTS lines has a greater
availability over the true VOIP phone you use via your residential
internet service. If "they" can trick the customer by providing the
"analogue-digital-analogue" service so well that the customer doesn't
realize it then the originating comment that started this tangent is
moot. They are providing a reliable E911 service over IP.

If they're not providing a more reliable service than we're back to the
same point. E911 over ip (and VOIP) are generally less reliable than
true POTS.

Regards,

  Chris