I wonder if someone knows a tool to use a tcpdump output for anomaly
dedection. It is sometimes really time consuming when looking for identical
patterns in the tcpdump output.
It would be helpful to get a diff between SYN and ACK's e.g. Or look for a
pattern in a URL. Or just get some timediffs e.g. when an ACK is send but
client is waiting for data etc.
We would like to decrease time to investigate the cause for an unusual network
behaviour.
However, I believe this isn't as detailed or low-level as what you're
looking for. In any case, it's a great tool for seeing unusual patterns
or strange behavior on your network.
1. [NOTE: I am biased toward this one, for some personal reasons ;)] I would
highly recommend you to read some of the papers of the gold certified SANS
people - start here:
For this sort of thing, you can do it far more scalably with NetFlow. There are several good commercial NetFlow-based anomaly-detection systems (Arbor, Lancope, Narus, Q1, etc.) and even an open-source project (currently fallow) called Panoptis.
Check out Argus, <http://www.qosient.com/argus/>. (I recommend still using version 2, version 3 is not quite production quality yet...)
Argus is a stream analyzer, instead of a packet analyzer. You can search argus data by tcp flags, by regular expression on the data (if you enable stream data logging, which is optional), or several other options. See the argus site for more information.
SiLK is a powerful toolset for analyzing netflow and pcap data generated from TCPDUMP. It's a slight learning curve, but worth it IMHO. Fairly good documentation too.
You might also look at "softflowd" [1] or similar tool to export netflow records from whatever box your using TCPDUMP to capture data. Then you can output netflow records directly to most of the aforementioned netflow packages. Having the actual packet data is useful later once you've found something suspicious, or for snort.. etc.
I've noticed an increased interest for Panoptis so I thought id send this email out:
Panoptis has been updated so that it compiles/runs with newer systems. It works on Debian Sarge for sure, should do the same on any system with GCC 3.3.5 and CommonC++2 1.5.3 at the very least.
It is still rough around the edges and no new features; Just an update to get it working.