Weird Issues within L3

Hi Group,

We have a couple of circuits , internet facing with Level 3 and from
our edge in San Diego, seeing some packet loss when trying a ping to
4.2.2.4, sourcing from 63.214.184.3. anyone seeing a similar issue ?

Packet sent with a source address of 63.214.184.3
!.!!!.!!!!!!!!!!!!!...!!!..!!!!!!.!!.!!!!!!.!..!!.!!.!!!!!!!!!!!!!!!!!
!!!!!!!!!.!!.!!!..!!!!!.!!!!.!
Success rate is 80 percent (80/100), round-trip min/avg/max = 12/13/16 ms

Not a weird issue. It's called packet loss. You might want to try some traces to see where that loss is happening. Basic troubleshooting.

Steve Naslund
Chicago IL

The 4.2.2.x machines are known to have some pretty heavy handed ICMP rate
limits.
  
I'd suggest trying to hit anything on level3 but those...
  
Nick Olsen
Network Operations (855) FLSPEED x106

what's interesting is that out of my XO peering, I show no packet loss
but out of L3, I do see packet loss.

XO out of Denver:

Sending 100, 100-byte ICMP Echos to 4.2.2.4, timeout is 2 seconds:
Packet sent with a source address of 69.171.180.67
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Success rate is 100 percent (100/100), round-trip min/avg/max = 12/15/16 ms

L3 out of San Diego:

Packet sent with a source address of 63.214.184.3
!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Success rate is 98 percent (98/100), round-trip min/avg/max = 1/7/16 ms

L3 out of Phoenix:

Packet sent with a source address of 4.53.104.66
!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!
Success rate is 97 percent (97/100), round-trip min/avg/max = 8/16/24 ms

traceroute out of the L3 pop in San Diego

Type escape sequence to abort.
Tracing the route to 4.2.2.4

  1 4.71.136.185 4 msec 4 msec 4 msec
  2 4.2.2.4 12 msec 12 msec 16 msec

Not to mention known bugs with how traffic is routed in large system like
Google...because of any/multicast and all. (I hear it's a known bug
internally.)

I personally love the Level 3 public DNS servers, but they do carry some
quirks with them, as all things in life do.

I'm not with level3, but the pattern looks exactly like what happens
when a destination node has rather severe rate-limiting in place, and
multiple people are sending ICMP. Try spacing your ping packets. On
Linux, the command switch is "ping -i 3" to send packets every three
seconds; for MTR it's "mtr -i 3" -- If you are using something else,
there should be an "interval" parameter that you can set to minimize
rate-limiting effects.

Have you tried sending real traffic to 4.2.2.4, and looking at the
packet traffic?