> I'm imagining that even with a relatively speedy box, if you were
> trying to do analysis from multiple interfaces you'd at least choke
> the disk I/O. There's always stringent filters, I guess.
Disk I/O on a sniffer box? Sounds like you've been sniffing something
other than packets my friend.
Why do you say that? In the 10/100 range, yes, no problems. But at
the Gigabit range (say with two GbE cards or a single OC-48 card) on
an x86 box with IDE disks (or even SCSI RAID0), doesn't disk I/O
become a severe problem? Under Solaris or Linux, scaling disk seems
relatively easy with Veritas Foundation Suite on Solaris or GFS under
Linux All Red Hat products
However, I don't think Linux or Solaris can handle the packet capture
capabilities like FreeBSD and BPF can. I've heard things about the
new LPF capabilities and turbopacket, but it's just hard to believe
coming from such a joke/toy operating system.
Whether you are passively tapping a gigabit ethernet or SONET fiber,
or even spanning an entire VLAN or mirroring a gigabit ethernet or
SONET port on a router/switch -- you've got a lot of packets/frames
to deal with, especially if you want to keep all of them for analyzing
later. Sounds like a disk I/O problem to me. Are you doing packet
capture at these rates and ran into no disk problems? How did you
deal with that? We are doing so right now, but only with the IP
headers and some "top N" information. Getting full packets and
keeping them for awhile (a day or two even) is going to take a lot
of I/O and disk space. I don't think it's worth it, really.
You can build your own box like that easily enough. If you're going
for FastE sniffing I highly recommend the Adaptec Quartet 4-port
cards. If you're going for GigE sniffing, I STILL highly recommend
anything Alteon Tigon 2 based (NetGear GA620's were the cheapest
if you can still find them, not the 621/622).
You can accomplish almost anything with the Tigon2's and FreeBSD, agreed.
Another vendor I'm sort of looking at now is Endance (DAG cards):
This only does IP headers, but that's the fun stuff anyways ;>
You don't even have to do anything fancy with the card firmware,
there is a native command for receiving only part of the frame.
Check out the programming manuals at
Index of /~wpaul/Alteon/, and I recommend you use
FreeBSD for this of course. Just add in a PARTIAL_RX_CNT command,
and the card will only DMA part of the packet (say 64 bytes for
full headers) across the PCI bus. Combined with interrupt coalescing
(or luigi's device polling and tuning the card to allocate all
memory to RX and remove the TX functionality completely), you can
sniff quite a few "gigabits" of traffic on a single cheap PC server.
You can dump it through the BPF mechanism and still maintain support
for all your favorite sniffer programs. Or if you're comfortable
writing kernel code, I recommend you make a character device for
sniffer device control, and use it to pass page-aligned malloc'd
memory pointers from userland into the nic driver, which you then
pass to the card as the RX ring buffers. This will let you DMA your
packets directly into userland. If not, at least unhook ether_input().
Can you post more details or catch up with me offline about this? I'm
really very interested in your implementations and results.
Or you can buy these things commercially. My favorite was from a
company called Tekelec, who sold a VERY expensive box which turned
out to be a pentium 200ish box running solaris x86 and completely
useless sniffing software, with a bunch of ISA ethernet cards hooked
up by proprietary (and VERY expensive) cables, all in a box made
out of what I swear was some kind of lead/neutron star material
alloy. Of course that was a couple years ago, maybe they've upgraded
to the current market's $50 processor.
We've been looking at NetVCR from Niksun which sounds similar except
that is actually is FreeBSD-based. Somebody needs to put together a
list of all these companies and do some comparisons of the product
offerings. Like you, I'd rather just build my own box and run with it ;>