Howdy!
Checking out various Netflow tools and wanted to see what others are using?
Kentik is cool. Are they the only SaaS based flow digester? I don’t seem to see any others.
Also curious about on-prem solutions as well.
Thanks!
Mike
Hey Mike. Kentik does on-prem, too.
Full disclosure: I work for Kentik and I’m glad you think we’re cool 
Dan
I'm very fond of nfsen/nfdump for on-prem. Setup is not complicated at all
and plugins are widely available.
Also inbefore Solarwinds...
-Matt
Kentik is probably top of the foodchain right now.
But they are certainly not alone in the biz. Ontop of my head...
* Flowmon
* Talaia
* Arbor Peakflow
* Deepfield
* Pmacct + supporting toolkit
* NFsen/Nfdump/AS-stats
* Put kibana/ES infront of any collector
* Solarwinds something something
* Different vendor toolkits
Plixer is also interesting.
nfdump works great with NetFlow but support for IPFIX is somehow limited to basics.
FlowViewer is a robust user interface complement to Carnegie Mellon's SiLK netflow capture and analysis tool suite.
FlowViewer provides the user with text/graphical analysis tools, multiple dashboards, long-term tracking of filtered sets, automatic storage management, raw netflow packet analysis, etc..
All open-source. Easy install. Runs on Linux.
FlowViewer: https://sourceforge.net/projects/flowviewer/
SiLK: https://tools.netsa.cert.org/silk/
Joe Loiacono
Kentik is probably top of the foodchain right now.
But they are certainly not alone in the biz. Ontop of my head...
* Flowmon
* Talaia
* Arbor Peakflow
* Deepfield
* Pmacct + supporting toolkit
* NFsen/Nfdump/AS-stats
* Put kibana/ES infront of any collector
Logstash has a netflow plugin as of 5.x or something (Logstash Netflow Module | Logstash Reference [8.12] | Elastic) to act as a collector.
A walkthrough:
http://www.routereflector.com/2017/07/elk-as-a-free-netflow-ipfix-collector-and-visualizer/
Using the logstash module setup thing adds a whole bunch of pretty netflow graphs and visualizations and such into Kibana for you.
Caveat:
Supports netflow v5 and v9, but does not indicate support for IPFIX explicitly. It definitely does not support sFlow, though if you really want you can stick sflowtool in front of it to translate sFlow->netflow, e.g. sFlow: sflowtool.
There is also https://github.com/robcowart/elastiflow which uses the ELK stack.
Luke Guillory
Vice President – Technology and Innovation
Tel: 985.536.1212
Fax: 985.536.0300
Email: lguillory@reservetele.com
Reserve Telecommunications
100 RTC Dr
Reserve, LA 70084
Not necessarily (only) for *flow, but very nice combo: Luca Deri's
ntopng+nprobe (https://www.ntop.org/products/traffic-analysis/ntop/)
***Stefan
Mike,
All of the architecture's listed are pretty good. Nfsen is great if you
have multiple routers exporting various netflow versions with a single
daemon, but its a bit older and not as pretty/quick as something using
elastic.
Team Cymru has a netflow analyzer that matches your netflow data to
known 'bad IPs'. http://www.team-cymru.org/Flow-Sonar.html
Thanks,
Scott
Thanks,
Scott
+1 for ElastiFlow. Couldn't be easier to set up and run. Logstash has
native support for netflow and sflow now via codecs. Kibana is an
easy-to-use dashboard. I trimmed out a bunch of stuff in the ElastiFlow
config that assumed a unidirectional network (like a corporate site).
How scalable is ElastiFlow ? Let say I will dump 90kflow/s, how big
elasticsearch farm do I need to comfortably store and work with at least
couple weeks of data ?
right now in NFSEN it takes about 3T in disk space and minutes for
simple reports if it spans few time default time intervals.
Thank you.
IPFIXcol+fbitdump is what we use for our IPFIX measurements:
https://github.com/CESNET/ipfixcol/
Can do NetFlow v5/v9 and sFlow as well.
luuk
Netflow Auditor
In-house solution. The interface takes some getting used to, but you can pull a-n-y-t-h-i-n-g from it. Easy setup, great support, highly scalable, priced well.
Best regards,
-Alex
+1 ElastiFlow, the templates are great, a great quickstart to using
netflow on elk stack.
-Vinny Stipo
(To the thread in general)
Those of us using RouterOS have to suffer a bit longer to get ASN-usefulness out of these tools. Well, natively. I'm just about done with using pmacct to inject the ASN into into a local Flow Analyzer. Maybe I can figure out at some point how to get pmacct to spit out a new netflow with the ASN information so these other tools can work out of the box.
Disclaimer: Am Plixer engineer.
If you want to take it for a spin, you can download a fully functional
OVA/QCOW2 30 day eval from the plixer website. I can also get you access to
an AWS AMI as well.
I don’t want to turn this into an Ad. So DM if you need any info/access.
Mike Krygeris
+1 for Plixer Scrutinizer
Also +1 for plixer scrutinizer.
Stipo wrote:
+1 ElastiFlow, the templates are great, a great quickstart to using
netflow on elk stack.
out of curiosity, I set up a test ElastiFlow installation on a small
site recently. It's completely gorgeous from an eye candy point of view
and it's pretty easy to see how you could tap into the ELK APIs to do
interesting data mangling.
On the down-side, it used ~40x the amount of disk space that nfsen used
for the same accounting period, and even though it was only handling
less than 1G traffic at a NF sample rate of 1:10, logstash and
elastisearch managed to peg between 4-6 cores on the server which was
handling it. Granted, these were only E5606 (2011-era Westmere Xeon)
cpus, but even still there was an alarming mismatch between the amount
of compute power required compared to the amount of netflow traffic
being handled. It would be interesting to hear the sort of cpu
requirements needed for larger installations. Obviously you can scale
elkstack sideways, so it wouldn't be difficult to build out something
which performed well. The issue is that burning cpu time can become an
expensive proposition.
Nick