At 11:59am today WorldCom lost 6 major ATM trunks from St. Louis to L.A.
So as you can imagine... MANY MANY ISPs are in some state of down. Just
letting you know.
No time estimate to repair has been made.
Ya, we are going to kill worldcom for this one, they say they have 496 DS3
connections down. We have 3 connections into our Palo Alto POP, and
Worldcom said they all are over different fiber paths. As of right now
they all are down.
Nathan Stratton President, NetRail,Inc.
We have 3 connections into our Palo Alto POP, and Worldcom said they all
are over different fiber paths. As of right now they all are down.
Path diversity is not something obtained and then safely assumed forever.
It must be monitored with dilligence, and this is far from trivial, even for
L1 geeks.
randy
You should never trust what your sales people say. When designing your
backbone, its important to ask them for your DLRs (design layout records)
so that you can see physically, with your own eyes, the paths involved.
-Golan
I know this, I have a copy of the DLRs. What I am saying is Worldcom
changed the DLRs without letting us know, and now they all run over this
fiber cut. We request a copy of the DLRs on ALL our links. We started
this long ago when we had a MCI and Worldcom DS1 die because they went
over the same fiber.
Nathan Stratton President, NetRail,Inc.
I don't know if it is related, but the Internet today has gone to a
slow crawl, but this could be just UUNet. I was trying to get a site
today and only getting maybe 33bytes/sec. A little traceroute and ping
showed
traceroute www.bestfares.com
traceroute to www.bestfares.com (208.212.97.2), 30 hops max, 40 byte
packets
1 corerouter (206.43.30.254) 4.891 ms 0.002 ms 4.640 ms
2 508.Hssi4-0.GW1.SLT1.ALTER.NET (137.39.167.21) 1.159 ms 0.869 ms 0.808 ms
3 508.Hssi4-0.GW1.SLT1.ALTER.NET (137.39.167.21) 8.785 ms 3.812 ms 1.476 ms
4 * 126.Hssi4-0.CR2.ATL1.Alter.Net (137.39.59.185) 293.530 ms 332.660 ms
5 * Fddi0-0.GW1.ATL1.Alter.Net (137.39.37.67) 395.971 ms 434.364 ms
6 165.113.99.35 (165.113.99.35) 2373.699 ms 2191.839 ms 2062.897 ms
^^^^^^^^ ^^^^^^^^ ^^^^^^^^
matterhorn:~$ ping -s www.bestfares.com
PING www.bestfares.com: 56 data bytes
64 bytes from www.bestfares.com (208.212.97.2): icmp_seq=0. time=2542. ms
64 bytes from www.bestfares.com (208.212.97.2): icmp_seq=2. time=6017. ms
64 bytes from www.bestfares.com (208.212.97.2): icmp_seq=5. time=3021. ms
64 bytes from www.bestfares.com (208.212.97.2): icmp_seq=7. time=2702. ms
64 bytes from www.bestfares.com (208.212.97.2): icmp_seq=8. time=2596. ms
64 bytes from www.bestfares.com (208.212.97.2): icmp_seq=9. time=2423. ms
64 bytes from www.bestfares.com (208.212.97.2): icmp_seq=12. time=2575. ms
64 bytes from www.bestfares.com (208.212.97.2): icmp_seq=14. time=2558. ms
64 bytes from www.bestfares.com (208.212.97.2): icmp_seq=15. time=2655. ms
64 bytes from www.bestfares.com (208.212.97.2): icmp_seq=17. time=2849. ms
64 bytes from www.bestfares.com (208.212.97.2): icmp_seq=19. time=2700. ms
And you haven't checked since? In my experience, path redundancy doesn't stay
that way. Either the first or second round of grooming in carrier network
eliminates the path redundancy, and I don't know if it's possible to win
against the transmissions people about that.. We've fought with carriers about
that for a very long time, and didn't get anywhere..
-dorian
Yes, the last time we checked was a little over a month ago. They show the
DLRs were changed last Friday.
Nathan Stratton President, NetRail,Inc.
Worldcom changed the DLRs without letting us know
happens all the time. they do that like we move customers between router
ports.
if you want to maintain path diversity, you have to continually monitor it
with your carriers. and then you'll need to go out and physically every
once in a while. yes, this is major major pain.
randy
Well, I dont move customers connections without letting them know. 
Nathan Stratton President, NetRail,Inc.
Shouldn't/Don't carriers flag paths that are part of path redundancy
links to avoid automatically re-routing them?
Cheers,
-- jra
As best as I've been able to tell, no.
-dorian