Analysing traces for performance bottlenecks

Date: Tue, 15 Jul 2008 11:05:34 +0100
From: Sam Stickland <sam_mailinglists@spacething.org>

Hi,

Are there any packages (or Wireshark options that I've missed) that can
follow a TCP stream and determine the limiting factor on throughput. E.g
Latency, packet loss, out of sequence packets, window size, or even just
the senders rate onto the wire. I know how to analyse a trace by hand
for performance issues, but it's relatively time consuming.

Googling for variations on "Analyse TCP stream limit throughput" didn't
find anything.

tcptrace is old and pretty basic, but it can provide a LOT if
information. Combined with xplot, the graphs often point to the exact
nature of a TCP problem, but you need a really good understanding of TCP
to figure anything out.

Kevin Oberman <oberman <at> es.net> writes:

tcptrace is old and pretty basic, but it can provide a LOT if
information. Combined with xplot, the graphs often point to the exact
nature of a TCP problem, but you need a really good understanding of TCP
to figure anything out.

Wireshark also provides tcptrace-like graphs ("Statistics -> TCP Stream Graph ->
Time Sequence Graph (tcptrace)"). They're not quite as pretty, but are just as
effective at tracking down all sorts of TCP problems, provided, as Kevin said,
you have a really good understanding of how TCP behaves.

Matt Cable wrote:

Kevin Oberman <oberman <at> es.net> writes

tcptrace is old and pretty basic, but it can provide a LOT if
information. Combined with xplot, the graphs often point to the exact
nature of a TCP problem, but you need a really good understanding of TCP
to figure anything out.
    
Wireshark also provides tcptrace-like graphs ("Statistics -> TCP Stream Graph ->
Time Sequence Graph (tcptrace)"). They're not quite as pretty, but are just as
effective at tracking down all sorts of TCP problems, provided, as Kevin said,
you have a really good understanding of how TCP behaves

Thanks for all the replies so far. While the TCP graphs are useful they are very difficult to read in Wireshark - they really need to be displayed in xplot, but this requires an X11 setup?

I've found NDT:

This uses a java applet hosted on a web100 patched linux server to record network diagnostics from connecting clients. A typical report might look like this:

    Web100 reports the Round trip time = 122.15 msec; the Packet size = 1260 Bytes; and
    No packet loss was observed.
    C2S throughput test: Packet queuing detected: 1.09%
    S2C throughput test: Packet queuing detected: 1.32%
    This connection is receiver limited 84.33% of the time.
      Increasing the the client's receive buffer (63.0 KB) will improve performance
    This connection is sender limited 1.70% of the time.
      Increasing the NDT server's send buffer (127.0 KB) will improve performance
    This connection is network limited 13.96% of the time.

    The theoretical network limit is 7869.69 Mbps
    The NDT server has a 127.0 KByte buffer which limits the throughput to 16.37 Mbps
    Your PC/Workstation has a 63.0 KByte buffer which limits the throughput to 4.09 Mbps
    The network based flow control limits the throughput to 8.73 Mbps

    Client Data reports link is 'OC-48', Client Acks report link is 'OC-12'
    Server Data reports link is 'OC-48', Server Acks report link is 'T3'

Something that could provide a similar, automated analysis of a TCP stream capture is what I'm after, although I doubt a standard packet capture will be able to provided as many metric as web100 stack can.

Sam

There is a Java version of xplot available now called jPlot. It works
in largely the same way.

http://www.tcptrace.org/jPlot/

Regards,
Tim

There are several similar tools designed for ISP customer care centers
to help analyze network problems reported by customers. Customer calls
with complaint, agent clicks on tool and asks customer to try what is
broken, tool analyzes traffic and guesses what's wrong. CC agent then reads from the script for fixing problem X.

They tend to be expensive and are designed for point-click call center agents. The back-end analysis some of these systems attempt is
impressive, even if the output is overly simplistic. The one I'm most familar with was Adlex <http://www.adlex.com/> which has been purchased by CompuWare. But there are several others.

What about gnuplot? Maybe it provides something more than xplot?

http://www.gnuplot.info/

Tim Sanderson wrote:

What about gnuplot? Maybe it provides something more than xplot?
http://www.gnuplot.info/

R

http://cran.r-project.org/