Flow collection and analysis

Wondering what others in the small to medium sized networks out there are using these days for netflow data collection, and your opinion on the tool?

Thanks!

I’ll be the minority voice here - I have been very happy with Argus (qosient.com) but it does not have a GUI and that seems to be a factor for some

Dave

Kentik.

Happy.

Mark.

If your looking to go low-cost (free) try:

1) Carnegie/Mellon's very robust, flexible and efficient collector analyzer (command line): SiLK - https://tools.netsa.cert.org/silk

2) FlowViewer - A comprehensive web-based user interface for SiLK which provides textual, graphical analysis, long term tracking and dashboard: http:flowviewer.net or https://sourceforge.net/projects/flowviewer

Best!

Joe

Two open source tools you might consider:

  nfdump <https://github.com/phaag/nfdump>
  pmacct <http://www.pmacct.net/>

John

My company uses Auvik. It’s easy to setup but needs some tuning and is furiously expensive. The traffic analysis is pretty good and can be export for you to use in other tools. If netflow is all you are using it for look elsewhere regardless of what a sales person sales.

Kevin

Not a suggestion, but a question ....

Curious to know if anyone (apart from Cloudflare, obvs !) is using Goflow ? (https://github.com/cloudflare/goflow)

We use, depending on the situation, Intermapper, PRTG, and NTop.

Intermapper has the most powerful viewing app, but it’s expensive in that you have to pay for each flow collector. It’s an actual app (Windows, Mac, and Linux), rather than a web-based interface, so they can do more tricks with the GUI, like drill down and sorting.

PRTG includes its web-based flow collector and viewer for free, and there is even a free 100-sensor edition of the product that lets you use just the flow stuff fir free forever.

NTop is an open source web-based flow sensor and viewer, with a combo paid license model specifically for the flow collection. It connects directly to a mirror port and sucks up the flow info, where is the other products required to have some hardware device that exports flows. But you can get dirt cheap used Cisco routers that do this, such as the 1941, which although bulky do the job for a few hundred bucks. Once you get into 10 Gb speeds though you need dedicated hardware sensors in newer gear, which is pretty pricey. But if you have 10 Gb traffic you can afford it :slight_smile:

Ntop also has a commercial arm called Nbox, Which has a range of hardware based ready to go solutions. However it’s all supported out of Italy, and pretty much by one guy, so we’ve had uneven results with customers that purchased it.

-mel

a question not asked, and answer not provided here, is:
“What are you actually trying to do with the netflow?”

Answers of the form:
“Dos detection and mitigation planning”
“Discover peering options/opportunities”
“billing customers”
“traffic analysis for future network planning”
“abuse monitoring/management/investigations”
“pretty noc graphs”

are helpful… I’m sure other answers would as well… but: “how do you collect?” is “with a collector” and isn’t super helpful if the collector can’t feed into the tooling / infrastructure / long-term goal you have.

Hi,

There is also Elastiflow https://docs.elastiflow.com/docs/ https://github.com/robcowart/elastiflow.

Most of these things, yes.

Add:
Troubleshooting/operational support
Customer reporting

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

I agree with you.

The tool doesn’t really matter. Windows, linux, cloud or not.

It’s really important to first understand what are you trying to solve or improve?

If this step is forgotten, then it will just be another tool to support to add in your long list of useless tools.

My personal favorites are a mix of:

  • Ntop with PF_RING enabled.
  • Nfdump
  • Elasticsearch

I’m sure all the other tools are also very good. Csv in excel or grep/awk could also work if you know exactly what you’re looking for. :wink:

Jean

Elastiflow is pretty cool. https://www.elastiflow.com or the old open source version: https://github.com/robcowart/elastiflow

You can pretty much do the same thing with Elastic’s filebeat (https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-netflow.html).

Pmacct is also good for grabbing netflow http://www.pmacct.net and sending it somewhere (file, database, kafka, etc.) You can also grab BMP and streaming telemetry with it.

If you’re looking for open source DDoS detection using netflow, check out https://github.com/pavel-odintsov/fastnetmon

Shameless plug, check out my tool to look for spoofed UDP amplification request traffic coming into your network https://github.com/racompton/tattle-tale

FYI, you can send netflow to multiple collectors with https://github.com/sleinen/samplicator

-Rich

Has Elastic decided to join the rest of the world in the 21st century yet ?

Last time I looked at it (not too many years ago) they had no TLS support. Bit of a show-stopper in today's security environment.

Samplicator is a nifty tool.

–John

elastiflow is extremely easy to run on an httpd listening only on localhost and proxy behind a simple nginx TLS1.2/1.3 only configuration listening on port 443.

as are a number of other tools.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

Why is it even necessary for such a function?

Once upon a time, Laura Smith <n5d9xq3ti233xiyif2vp@protonmail.ch> said:

I don't know about anyone else here, but frankly in 2022 TLS support should be a first class citizen.

If I have to mess around with running something else as a proxy in front of it then that's the end of my software evaluation.

Crypto is no longer "nice to have" option these days.

Having every thing under the sun trying to implement the complexities of
TLS leads to lots of failures and security issues... so lots of web
things are designed to be simple and only implement HTTP, listen on
localhost, and let a well-optimized front-end (e.g. nginx) handle the
crypto side (as well as all the weird things browsers do).

It also makes it easier from an system admin point of view, because
handling cert updates in nginx is easy and well-known, so you don't have
to figure out 27 different ways alternate software handles certs and
updates.