debugging packet loss

I'm seeing 2-5% packet loss going through a Cisco 2621 with <10mbps of
traffic running at ~50% CPU. (packet loss based on ping results)

Pinging another box on the same catalyst 2900 switch gives no packet loss,
so it seems the 2621 is the source of the packet loss. I need help
figuring out why it is dropping packets, and how to stop it.

The odd thing is that the interface stats don't show any dropped packets:
  Full-duplex, 100Mb/s, 100BaseTX/FX
  ARP type: ARPA, ARP Timeout 00:01:00
  Last input 00:00:00, output 00:00:00, output hang never
  Last clearing of "show interface" counters 00:10:16
  Queueing strategy: fifo
  Output queue 0/40, 0 drops; input queue 1/512, 0 drops
  5 minute input rate 5257000 bits/sec, 1383 packets/sec
  5 minute output rate 5692000 bits/sec, 1448 packets/sec
     845743 packets input, 392733148 bytes
     Received 403 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 0 multicast
     0 input packets with dribble condition detected
     887446 packets output, 430245866 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 babbles, 0 late collision, 22694 deferred
     0 lost carrier, 0 no carrier
     0 output buffer failures, 0 output buffers swapped out

Ralph Doncaster
principal, IStop.com

I'm seeing 2-5% packet loss going through a Cisco 2621 with <10mbps of
traffic running at ~50% CPU. (packet loss based on ping results)

Isn't ping the first thing to be dropped in favor of other traffic? I
remember a similar issue and Cisco saying that was the behavior. Don't
quote me on that.

jas

Not neccessarily. It's my experience that ciscos will sometime drop icmp
instead of replying when under load...but that's only for packets directed
at its interfaces.

So, I might see 5% packetloss from the router itself, but 0% packetloss
for everything behind it.

Andy

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Andy Dills 301-682-9972
Xecunet, LLC www.xecu.net
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Dialup * Webhosting * E-Commerce * High-Speed Access

If its not the switch or source ethernet segment generally, and it
doesn't appear to be the router itself, I would concentrate on the
ethernet segment between the switch and the router. (Assuming that your
packet loss is while pinging the router, and not something on the other
end of the router.)

-Wes Bachman wbachman@leepfrogDOTcom

I'm seeing 2-5% packet loss going through a Cisco 2621 with <10mbps of
traffic running at ~50% CPU. (packet loss based on ping results)

Pinging another box on the same catalyst 2900 switch gives no packet loss,
so it seems the 2621 is the source of the packet loss. I need help
figuring out why it is dropping packets,

Because pings are processed by the CPU which has already lots of
better things to do than answering to pings. Answering ICMP is
quite low-prio for routing engines.

Trying pinging ERXes... you'll get "packet loss" (correct: "sometimes
no answer") even though the RE twiddles thumbs. There is big difference
between vendors in respect of ICMP echo reply priorization. Cisco
is usually quite "good" at it (except for when BGP Scanner is running
you'll see an increase of some dozends to hundreds of milliseconds
in reply latency).

and how to stop it.

Get a faster router and/or just don't care that much about ping
replies from routing engines of routers. :slight_smile:

Packet loss while _forwarding_ packets counts, not when pinging
loopbacks (yes, I know, there are many customers who can't tell the
difference).

Regards
Daniel