IDS experience's

IDS's have been around awhile but recently became interested in their
usefulness. I was wondering if I could get some group feedback on the

1. How many folks have actually deployed either a NID, NNID or HID system?

2. Have they been useful or just generated noise and excess cycles? (1 -
waste of time, 10 - water walker)

3. Any 'real-world' comparative/useful data and/or opinion on different pattern matching, anomoly detection and/or data mining

4. Any feedback on Snort, ISS, Cisco or Symantec? Or other newer/different
approaches ie Okena?

5. Other general good information, ie issues, gripes, etc.?

I would appreciate any help, feel free to contact direct or list and will



:2. Have they been useful or just generated noise and excess cycles? (1 -
:waste of time, 10 - water walker)

:3. Any 'real-world' comparative/useful data and/or opinion on different pattern matching, anomoly detection and/or data mining

The only real value from IDS data is based upon your ability to mine
and interpret it. This is something that IDS vendors have utterly
failed to provide a solution to, and something that most customers
haven't totally wrapped their head around.

In fact, a seperate IDS data mining and interpreting industry has
popped up with players like NetForensics, Intellitactics and I'm
sure there are others. In fact, if SilentRunner took snort logs
(I haven't checked in a while) it would be an ideal solution for

It is to the point where it really doesn't matter what brand of
sensor you install, as none of them do data corelation effectively
enough to be used without a third party data mining solution, for
installations of more than a single sensor.

I have found that even having 0-day signatures for the most obscure
and dangerous exploits, doesn't add much value to an IDS. This
is because even a skript kid with 0-day warez is going to probe,
portscan and reach for low hanging fruit before they will risk exposing
their more valuble toys to a potential honeypot. All an IDS is, is
a policy monitoring device, which you use to make operational decisions,
and potentially to augment your policy enforcement.

The value of IDS data is really only uncovered through corelation.
Anomaly based systems try to do this as part of the detection process,
whereas signature based systems assume it will be done in post processing.
Anomalies are ultimately just a different kind of signature anyway. :slight_smile:

With things like ACID and other front ends to Snort, IMHO, the best
view of the data you can get is a listing of source ip addresses with the
number of unique alerts they generated over a long period of time.

The visualization tools from Intellitactics look like they were lifted
from This doesn't undermine how useful and cool they are,
but it suggests that someone with more skills than I, will think of a
way to parse snort logs into something like NetCDF or some other
scientific visualization format for use with real visualization and
data mining tools.

I spend most of my day watching IDS's that generate massive amounts of
data, and this information is based upon that experience.


Thanks to those that responded, content listed below with a few comments of
my own. Also welcome additional discussion.

A lot of new activity in the space, but very little differentiation beyond
scale. Correlation and mining of useful and actionable information minimal
at best. Multiple 'probes' magnify the problem. Signature based products
based on their maturity still rule although some of the new 'pattern'
matching products appear interesting. Their problem is providing enough
pattern classification detail to understand the reasoning.

Would appreciate any comments on 'intelligent' multi-probe data mining
approaches/products examined and/or enterprise cross-vendor correlation
products. I've seen Bayesian and neural network approaches that appear
promising but are currently closer to a research project rather something

Also welcome vendor feedback although prefer off-list mail.



More people should take the time to compile worthwhile summaries.
Recently I've been evaluating various IDSs... primarily to quickly identify
DOSs so they can be rate-limited if they're specific enough (by a small
source pool or a port that wouldn't interfere with primary traffic) or null
them if the customer's firewall/server/LB goes down and floods the block..

We have a Dragon system which is primarily used to identify portscans over a
multiple IPs and blackhole the source. I'm told it has more functionality
but I haven't had the time to explore its potential.
I've just begun using Arbor's Peakflow system--a traffic and DOS
platform--it uses set parameters to identify traffic anomalies using Netflow
stats. I believe that it has some good potential, but already we've had some
scalability issues and the 'tweaking' is very administratively intensive. It
has missed a few serious anomalies we could see on bandwidth graphs that it
didn't pick up.
And last, I'm about to receive Wildpacket's EtherPeek NX which uses a Gig
span to identify traffic flows and do pretty much the same thing as Arbor's
but all in Software and every packet. I'm very interested to try it because
of its full span and price. Unfortunately, it does cap at a Gig and so
multiple boxes will be needed in a large environment and there is no
aggregation software for the statistics.

I would love to hear more about other's experiences with these products and
values, or other interesting views on the subject.


"Be liberal in what you accept, and conservative in what you send."
--Jon Postel

It appears that this recent report was overlooked:


Crying wolf: False alarms hide attacks

Eight IDSs fail to impress during the monthlong test on a production network.

By David Newman, Joel Snyder and Rodney Thayer
Network World, 06/24/02

One thing that can be said with certainty about network-based intrusion-detection systems is that they're guaranteed to detect and consume all your available bandwidth. Whether they also detect network intrusions is less of a sure thing.

Those are the major conclusions of our first-ever IDS product comparison conducted "in the wild." Unlike previous tests run in lab settings, we put seven commercial IDS products and one open-source offering on a live ISP segment to see what they'd catch.

What we found wasn't encouraging:

   Several IDSs crashed repeatedly under the burden of the false alarms they churned out.

   When real attacks came along, some products didn't catch them and others buried the reports so deep in false alarms that they were easy to miss.

   Overly complex interfaces made tuning out false alarms a challenge.

Because no product distinguished itself, we are not naming a winner (See "No cigar"). The eight products we tested - from Cisco, Intrusion, Lancope, Network Flight Recorder (NFR), Nokia (running on OEM version of Internet Security Systems RealSecure 6.5), OneSecure, Recourse Technologies and the open-source Snort package - all ask too much of their users in terms of time and expertise to be described as security must-haves.

(follow the URL above for the whole story)