Level 3 TPA routing today?

Anyone seeing issues with Level 3 between anywhere
and Tampa, particularly Atlanta and Dallas? We've
seen lots of outbound issues and customers calling
about their sites being down or slow. Ultimately
turned our two links to them off and then everyone
says looks better. I opened a ticket with them and
of course they say no one has reported any issues
whatsoever. I sent them a bunch of traces and will
try turning them back on later tonight since these
non-issues tend to resolve themselves if I wait for
a single homed customer to yell loud enough for them
to fix it.

Thanks,

David

David,

  I'm seeing the same thing in the Atlanta market. I can trace to 1 IP in a
prefix but not another. Almost like maybe there's some FIB horkage across a
multi-path link or something. Fired off an e-mail, awaiting status for
now.

--chip

Anyone seeing issues with Level 3 between anywhere
and Tampa, particularly Atlanta and Dallas?

We just have co-lo in Tampa, but our upstream's connectivity is through
Level3 and we've been seeing intermittent packet loss up there all day. We
alerted our upstream but no updates so far. It hasn't been bad enough for
our customers to call and complain though.

Internap just reported problems with L3 out of Miami:

"we are seeing latency, minor packet loss and path problems to a
number of destinations and other PNAPs via our Level3 (AS3356) upstream
connection in the MIA003 PNAP. "

I've been seeing 30-70% packet loss between Cox Business and Level3 from
  DC to NY since 8:17pm EDT. Maybe via Internap?

                                   Loss% Snt Last Avg Best Wrst StDev
   3. mrfddsrj01-ge706.rd.dc.cox.n 0.0% 100 2.4 5.1 2.2 51.9 8.3
   4. xe-9-2-0.edge1.Washington1.L 67.0% 100 2.5 6.8 2.4 41.6 8.6
   5. vlan99.csw4.Washington1.Leve 69.0% 100 2.7 8.3 2.6 23.7 5.0
   6. ae-93-93.ebr3.Washington1.Le 68.0% 100 3.0 9.9 2.7 30.9 6.3
   7. ae-3.ebr3.NewYork1.Level3.ne 70.0% 100 10.5 15.8 8.1 44.2 8.8
   8. ae-83-83.csw3.NewYork1.Level 71.0% 100 18.9 14.2 8.1 42.0 7.1
   9. ae-31-89.car1.NewYork1.Level 66.0% 100 8.6 25.7 8.5 165.4 41.7

Beckman

We've also been seeing some "weird" (hard to track down) issues all day
with Level 3 in both Tampa and Atlanta, especially from our NMS systems
monitoring systems all over the place.

My contact at Level 3 didn't know of anything going on and couldn't really
find anything. Anyone else have a Level 3 response?

  -Scott

Confirmed outage on their side. Should be resolved now.

Drive Slow,
Paul Wall

I too have a cage at Hampton Oaks, 10M up and 10M tail to me over OC-12,
and I had problems 15-1630ish to wired.com and zimbra.com, among other
sites; mtr had no appreciable loss tracing, but upper layer protocols
were hincky. It was clear by the next morning...

Cheers,
-- jra

We have gigE to Level3 in Orlando, and saw something happen around 1pm today. Customers were complaining of latency and packet loss, and our traffic to/from L3 dropped noticably if only for a few minutes.