TCP Performance (NANOG Digest, Vol 32, Issue 60)

Sat, 18 Sep 2010 09:34:55 +0000 fuream loqour :

From: "Abel Alejandro" <>
Subject: Troubleshooting TCP performance tutorial

This past week I have been trying to find the root cause of tcp
performance problems of a few clients that are using a third party metro
Ethernet for transport. RFC2544 tests (Layer 2) and iperf using UDP give
good symmetric performance almost 100% the speed of the circuit. However
all kind of TCP tests result in some kind of asymmetrical deficiency,

Walking through the layers, if L2 checks out, & UDP checks out, but TCP
does not, you might have a problem "up the stack", @ L4 with TCP. I
don't know your test rig / setup, but a few ideas come to mind:

- Mismatch in the TCP personality between the hosts - mixing Windows &
some UNIX OS flavors, like Sun Solaris 2.8 or others can produce
incredible degradation in streaming TCP performance either due to buggy
TCP behavior, or one TCP stack supports options the other doesn't, like
one side is doing Classic Reno TCP (max 64K) window, doesn't respect or
process RFC1323 window-sizing options, but the other side does. It
happens UNIX-to-UNIX too, and embedded devices usually don't have robust
TCP/IP stacks as well (like hand-held barcode readers, etc.). Any client
patch an OS recently?

- Someone is doing QoS of some type either expressly or indirectly by
effect. Stream / test TCP on multiple ports / sockets to multiple hosts,
different OS combinations, or a known good combination - Knoppix Linux
boot CD's are great for this kind of thing - send them to a client, pop
'em in, do some testing - once built a test CD for this, fun, gets rid
of that long "troubleshoot loop".

- MTU / other issue? Did your UDP stream pack full-size payloads? If
not, walk up the size of the test packets to the maximum, see what
happens. If it breaks somewhere, you have the beginning of an answer, etc.

- Not thinking it's the client app since you mentioned you've run a
whole set of TCP-based tests.