TransAtlantic Cable Break

Mailman List Mirror (Read Only)
1

Well, despite all this talk about the Latin fiber break, the real news is
that one of the TransAtlantic cables has had one of their two cables
severed. Repair is not expected until after the US July 4th holiday.

So none of the customers on that well known system have any ring
protection at this point nor will they during the next two weeks.

This is a lesson for those who put all their eggs in one basket.

This cable system is also a major IP backbone player.

So it shouldn't be too hard to guess their identity.

Regards,

Roderick S. Beck.
Global Wholesale Bandwidth, LLC
Manhattan & Paris
email: info@globalwholesalebandwidth.com
New York Landline: 212-942-3345
Paris Landline: 33-1-4346-3209
GSM Wireless: 1-212-444-8829

This e-mail and any attachment thereto may contain information which is
confidential and/or protected by intellectual property rights and are
intended for the sole use of the recipient(s) named above. Any use of the
information contained herein (including, but not limited to, total or
partial reproduction, communication or distribution in any form) by other
persons than the designated recipient(s) is prohibited. If you have
received this e-mail in error, please notify the sender either by
telephone or by e-mail and delete the material from any computer.

2

Isn't that the way a ring works? Sounds like everything is working
as designed.

-Jim P.

3

This cable system is also a major IP backbone player.

So it shouldn't be too hard to guess their identity.

You think?

4

Right. When one side got chomped by a turbo-shark or whatever got it,
the other side took over. And until they fix the first side, any failure
on the other side will mean it will fail over to the *other* other side.

Oh, there *is* no "*other* other side"? That must be what Roderick meant.. :wink:

5

Tell that to the 10 gig wave customers who lost service. Very few cable
systems provide protection at the 10 gig wave level.

the real news is that one of the TransAtlantic cables has had one
of their two cables severed. Repair is not expected until after
the US July 4th holiday.

So none of the customers on that well known system have any ring
protection at this point nor will they during the next two weeks.

Isn't that the way a ring works? Sounds like everything is working
as designed.

-Jim P.

Roderick S. Beck.
Global Wholesale Bandwidth, LLC
Manhattan & Paris
email: info@globalwholesalebandwidth.com
New York Landline: 212-942-3345
Paris Landline: 33-1-4346-3209
GSM Wireless: 1-212-444-8829

This e-mail and any attachment thereto may contain information which is
confidential and/or protected by intellectual property rights and are
intended for the sole use of the recipient(s) named above. Any use of the
information contained herein (including, but not limited to, total or
partial reproduction, communication or distribution in any form) by other
persons than the designated recipient(s) is prohibited. If you have
received this e-mail in error, please notify the sender either by
telephone or by e-mail and delete the material from any computer.

6

Yes, since most cables have multiple IP backbones on them, and this one
has only one ...

This cable system is also a major IP backbone player.

So it shouldn't be too hard to guess their identity.

You think?

Roderick S. Beck.
Global Wholesale Bandwidth, LLC
Manhattan & Paris
email: info@globalwholesalebandwidth.com
New York Landline: 212-942-3345
Paris Landline: 33-1-4346-3209
GSM Wireless: 1-212-444-8829

This e-mail and any attachment thereto may contain information which is
confidential and/or protected by intellectual property rights and are
intended for the sole use of the recipient(s) named above. Any use of the
information contained herein (including, but not limited to, total or
partial reproduction, communication or distribution in any form) by other
persons than the designated recipient(s) is prohibited. If you have
received this e-mail in error, please notify the sender either by
telephone or by e-mail and delete the material from any computer.

7

Thank you Vladis for pointing out that untidy detail ..

:slight_smile:

Best Regards,

Roderick.

> So none of the customers on that well known system have any ring
> protection at this point nor will they during the next two weeks.

Isn't that the way a ring works? Sounds like everything is working
as designed.

Right. When one side got chomped by a turbo-shark or whatever got it,
the other side took over. And until they fix the first side, any failure
on the other side will mean it will fail over to the *other* other side.

Oh, there *is* no "*other* other side"? That must be what Roderick
meant.. :wink:

Roderick S. Beck.
Global Wholesale Bandwidth, LLC
Manhattan & Paris
email: info@globalwholesalebandwidth.com
New York Landline: 212-942-3345
Paris Landline: 33-1-4346-3209
GSM Wireless: 1-212-444-8829

This e-mail and any attachment thereto may contain information which is
confidential and/or protected by intellectual property rights and are
intended for the sole use of the recipient(s) named above. Any use of the
information contained herein (including, but not limited to, total or
partial reproduction, communication or distribution in any form) by other
persons than the designated recipient(s) is prohibited. If you have
received this e-mail in error, please notify the sender either by
telephone or by e-mail and delete the material from any computer.

8

Tell that to the 10 gig wave customers who lost service. Very few cable
systems provide protection at the 10 gig wave level.

But surely you knew that going into the deal and did something about it?

Neil.

9

And he just happens to have an email addr that suggests he's involved
with a company that sells that *other* other side. Just saying.

Don't get me wrong... I believe in layers of redundancy, but at some
point it becomes more of a headache than a help.

-Jim P.

10

there's a transatlantic cable system with only 1 IP backbone using it??
Really? is it just a tiny cable with little capacity? Or is the consortium
that owns it not permitting folks to use it for 'ip' for another reason?

-Chris

11

If you don't pay the extra amount for a protected circuit, why should your circuit get protection for free when others have to pay for it? Now, if there are 10G customers with protected circuits who lost service, then hopefully they have in their contract hefty penalty clauses against the carrier. If not, then they are just plain stupid.

-Hank Nussbacher
http://www.interall.co.il

12

Is paying for "protected circuits" actually worth it. Or are you better off just buying two circuits and using both during normal conditions. Use switching at layer 3 to the remaining circuit during abnormal conditions. Most of the time, you get twice the capacity for only twice
the price instead of a "protected circuit" where you only get the once the capacity for twice the price.

Of course, there is still the problem some facility provider will "groom" both your circuits on to the same cable. If you are buying pre-emptable circuits, hopefully you understand what that means.

13

You would be surprised at how many don't split their traffic among several
cable systems or if they do split, do due diligence on the terrestrial
backhaul that most of these cables share on the US and UK sides.

There is lot of cost pressure in the IP market and it is very tempting to
give all the business to one system in order to maximize cost savings.

And even those players who do split their traffic among the 7 major
TransAtlantic cables, many are not aware that six of those systems share
the conduit coming off the UK beaches. Six cables in one duct ...

Tell that to the 10 gig wave customers who lost service. Very few cable
systems provide protection at the 10 gig wave level.

But surely you knew that going into the deal and did something about it?

Neil.

Roderick S. Beck.
Global Wholesale Bandwidth, LLC
Manhattan & Paris
email: info@globalwholesalebandwidth.com
New York Landline: 212-942-3345
Paris Landline: 33-1-4346-3209
GSM Wireless: 1-212-444-8829

This e-mail and any attachment thereto may contain information which is
confidential and/or protected by intellectual property rights and are
intended for the sole use of the recipient(s) named above. Any use of the
information contained herein (including, but not limited to, total or
partial reproduction, communication or distribution in any form) by other
persons than the designated recipient(s) is prohibited. If you have
received this e-mail in error, please notify the sender either by
telephone or by e-mail and delete the material from any computer.

14

Protected 10 gig waves NYC/London are extremely expensive. Say $60K or more per month.

So it usually makes sense for the Layer 3 guys to lease diversely routed 10 gig waves and do the protection themselves using MPLS or load balancing or some other protocol about which I know little …

Roderick S. Beck
Hibernia Atlantic
1 Passage du Chantier, 75012 Paris
http://www.hiberniaatlantic.com
Wireless: 1-212-444-8829.
Landline: 33-1-4346-3209
rod.beck@hiberniaatlantic.com
rodbeck@erols.com
``Unthinking respect for authority is the greatest enemy of truth.’’ Albert Einstein.

15

This strikes me as much ado about nothing.

Mr. Beck (apparently a cable sales person, probably for the cable he says doesn't share the vulnerabilities) started this out by dismissing a cut that caused actual outages in an apparently less important part of the world, telling us that the "real news" is that one side of a redundant transatlantic ring was broken, leaving customers relying on the other side. He has then continued to post and post and post on the subject.

Redundancy is a statistical game, a bet that some number of pieces won't all break at the same time. With a single point of failure, it's pretty likely that at some point you'll have an outage. As long as those outages are relatively rare and short, adding a second path makes it significantly less likely that both will break at once, but there's still some chance. Adding additional paths reduces the probability of simultaneous failure further, but never to zero. As you add more paths, you reach a point of diminishing returns pretty quickly. You also add complexity, which can cause its own problems.

If it takes two weeks to repair a broken cable, carriers need to do a cost-benefit analysis. With a single cable, what are the chances of that cable breaking, and how much will it cost them to be down for two weeks, as compared the cost (including complexity) of a second path? With two paths, what are the chances that the second one will break during the same period that the first one is broken, how much will the resulting outage cost, and how does that risk compare to the cost of a third path?

Presumably, most of the carriers are doing this analysis already. If their customers are sufficiently concerned, they're presumably doing such an analysis of their own. If any aren't, perhaps they should think about it, but whether doing that analysis will have them running out to buy an additional path across the Atlantic is far from clear.

-Steve

16

From: Rod Beck [mailto:Rod.Beck@hiberniaatlantic.com]
To: Sean Donelan [mailto:sean@donelan.com], Hank Nussbacher [mailto:hank@efes.iucc.ac.il]
Cc: nanog [mailto:nanog@merit.edu]
Sent: Fri, 22 Jun 2007 10:14:20 -0800
Subject: RE: TransAtlantic Cable Break

Protected 10 gig waves NYC/London are extremely expensive. Say $60K or more per month.

Not bad as DS3’s between Alaska & Seattle used to cost that much.

-Dee

17

Roderick,

are you actually fishing and / or hoping for comments and
mails directly to you to then tell your story and sell some?
You throw so many pieces in here and there, it sounds like
advertising, like your daily promotional verses all in one
email thread. Despite your comments being all so innocent,
this is not isp-bandwidth...

So at the minimum make your role clear for everyone, and
play with open cards. And save yourself the .sig and the
disclaimer please...

Thanks,
Alexander

18

In a message written on Fri, Jun 22, 2007 at 11:56:32AM -0400, Sean Donelan wrote:

Is paying for "protected circuits" actually worth it. Or are you better
off just buying two circuits and using both during normal conditions.
Use switching at layer 3 to the remaining circuit during abnormal
conditions. Most of the time, you get twice the capacity for only twice
the price instead of a "protected circuit" where you only get the once
the capacity for twice the price.

Sorry, it doesn't work like that. I do happen to believe rather
than get a single SONET/WDM protected 10G Wave you are better off
getting two unprotected 10G waves and plugging them into your
devices, and let layer 3 routing take over. It generally saves a
good bit of cost, and it helps you keep the cable system honest.
You know when there is an outage, no way to hide it from you.

However, if you put 15G down your "20G" path, you have no redundancy.
In a cut, dropping 5G on the floor, causing 33% packet loss is not
"up", it might as well be down.

If your redundancy solution is at Layer 3, you have to have the
policies in place that you don't run much over 10G across your dual
10G links or you're back to effectively giving up all redundancy.

19

Leo Bicknell writes:

However, if you put 15G down your "20G" path, you have no
redundancy. In a cut, dropping 5G on the floor, causing 33% packet
loss is not "up", it might as well be down.

Sorry, it doesn't work like that either. 33% packet loss is an upper
limit, but not what you'd see in practice. The vast majority of
traffic is responsive to congestion and will back off. It is
difficult to predict that actual drop rate; that depends a lot on your
traffic mix. A million "web mice" are much less elastic than a dozen
bulk transfers.

It is true that on average (averaged over all bytes), *throughput*
will go down by 33%. But this reduction will not be distributed
evenly over all connections.

In an extreme (ly benign) case, 6G of the 20G are 30 NNTP connections
normally running at 200 Mb/s each, with 50 ms RTT. A drop rate of
just 0.01% will cause those connections to back down to 20 Mb/s each
(0.6 Gb/s total). This alone is more than enough to handle the
capacity reduction. All other connections will (absent other QoS
mechanisms) see the same 0.01% loss, but this won't cause serious
issues to most applications.

What users WILL notice is when suddenly there's a 200ms standing queue
because of the overload situation. This is a case for using RED (or
small router buffers).

Another trick would be to preferentially drop "low-value" traffic, so
that other users wouldn't have to experience loss (or even delay,
depending on configuration) at all. And conversely, if you have (a
bounded amount of) "high-value" traffic, you could configure protected
resources for that.

If your redundancy solution is at Layer 3, you have to have the
policies in place that you don't run much over 10G across your dual
10G links or you're back to effectively giving up all redundancy.

The recommendation has a good core, but it's not that black&white.

Let's say that whatever exceeds the 10G should be low-value and
extremely congestion-responsive traffic. NNTP (server/server) and P2P
file sharing traffic are examples for this category. Both application
types (NetNews and things like BitTorrent) even have application-level
congestion responsiveness beyond what TCP itself provides: When a
given connection has bad throughput, the application will prefer
other, hopefully less congested paths.

20

Sean Donelan wrote:

Is paying for "protected circuits" actually worth it. Or are you better
off just buying two circuits and using both during normal conditions.
Use switching at layer 3 to the remaining circuit during abnormal
conditions. Most of the time, you get twice the capacity for only twice
the price instead of a "protected circuit" where you only get the once
the capacity for twice the price.

I think it's pretty much safe to say that on a 10G wavelength, buying it
as protected on the same fiber route, same glass path is a total waste
of money. (at least for a trans-Atlantic circuit). That's some
seriously expensive electronics protection. For that cost, it'd be
cheaper to hire people to sit and at the box to replace cards when/if
something happens electronically. :wink: