Fiber cut in SJ

So... is mail not getting in/out from Nanog right now,
or is the fairly major fiber cut in San Jose not newsworthy
on the operational list anymore....?

I'm gonna guess that people were too distracted with "Oh crap, where'd
the internet go?"

I couldn't get to outside email from inside for a while.

So since there's the question, for those not in the know, the word is
that there was a cut through half of a 1000 strand cable owned by
Level3, affecting 502 fibers.

It's my understanding that they've been splicing since 23:45 GMT and
that they're moving through the mass of fibers at an impressive rate.

I have it on good authority (fiber provider, around
5:30 pm pst) that the outage happened in two phases.

First, an electrical contractor backhoed a large fiber
link in downtown San Jose (address deleted due to security
concerns) this morning, causing moderate damage.

Initial repair efforts apparently went nowhere; there wasn't
enough cable and trench space to work with.

Around 2pm PST the repair lead onsite had the remaining
cable sliced at manhole access points a couple of
hundred meters each direction from the original cut,
to give them clean surfaces and workspace for the
splice. Which of course took out the other hundreds
of pairs which had survived the initial backhoe.

We certainly didn't get any warning that was going
to happen, and neither did anyone else we were able
to talk to over the course of the afternoon.

I also understand from a couple of fiber providers
that an unusually large amount of other fiber had
been groomed onto that one cable over the last few
years of consolidations. Eight or nine people I had
talked to thought they had geographically distinct
ring loops that turned out to be on that one cable
when the second cut took it down hard.

-george william herbert

That's just plain silly. As if we (or even your imagined 'terrorist') don't
know where the fiber runs around here.

Mindlessly classifying everything as 'secret' is a tactic I'd expect of DHS,
not NANOG. 'Need to know' does not appear anywhere in the constitution.

well.. theres lots of ducting going down streets but not that many folks know
which of them are the major cable routes, i think keeping specific detail
discrete is reasonable

in a fire near where i am a couple years ago:

it seemed a bit of a coincidence that both the active and protect paths of a
major sdh route got hit in this attack and it took out a lot of long distance



The point I'm trying to make is that over classifying everything as 'secret'
or 'confidential' at this late date is useless. The horse is already out of
the barn.

You can omit the site of a fiber backhoe accident from an email and say it's
due to security concerns, but I can call any telecom vendor who sells SONET
or metro ethernet services and get them to fax me a map of their network. At
the very minimum all I have to do is keep an eye out for USA markings on the
street. Or I could call USA and the next day people with paint cans would be
marking up the street, showing me exactly where to dig.

If someone wants to cause trouble, the information they need is freely
available. The so-called security provisions most telecom companies use are
just enough to deter curious teen-agers.

Maybe a market difference.. most maps I've obtained in the UK have been under
NDA with established relationships already. Altho I suspect they're more
concerned at showing me who's duct and fiber they're actually on..


Eight or nine people I had
talked to thought they had geographically distinct
ring loops that turned out to be on that one cable
when the second cut took it down hard.

Perhaps now people will begin to take physical separacy
seriously and write grooming protocols and SLAs into
their contracts?

Or was this type of service "good enough"?

--Michael Dillon

What was the actual cost of this outage to operators in SLA credits?
Perhaps it's just a function of economics: it's cheaper to plan for
the 3 fiber cuts per 1000 route miles per year (or whatever the silly
rate was I came across somewhere) and hand out credits than actually
engineer physical redundancy.

But seriously, has anyone quantified what this or similar cuts *actually cost*?