Free Program to take netflow

Dennis_Burgess · May 17, 2019, 2:26pm

I am looking for a free program to take netflow and output what the top traffic ASes to and from my AS are. Something that we can look at every once in a while, and/or spin up and get data then shutdown… Just have two ports need netflow from currently.

Thanks in advance.

Dennis Burgess, Mikrotik Certified Trainer

Author of "Learn RouterOS- Second Edition”

Link Technologies, Inc – Mikrotik & WISP Support Services

Office: 314-735-0270 Website: http://www.linktechs.net

Create Wireless Coverage’s with www.towercoverage.com

Denys_Fedoryshchenk1 · May 17, 2019, 2:43pm

Fastnetmon have that: https://fastnetmon.com/fastnetmon-advanced-traffic-persistency/
I used it for such purposes.

Niels_Bakker · May 17, 2019, 4:00pm

* nanog@nanog.org (Dennis Burgess via NANOG) [Fri 17 May 2019, 16:25 CEST]:

I am looking for a free program to take netflow and output what the top traffic ASes to and from my AS are. Something that we can look at every once in a while, and/or spin up and get data then shutdown.. Just have two ports need netflow from currently.

It sounds like https://blog.apnic.net/2017/01/26/traffic-analysis-better-peering/ would be right up your alley.

-- Niels.

Crist_Clark · May 18, 2019, 4:19am

Been loving Elastiflow. Way overkill for what you need, but it's
actually pretty easy to setup.

https://github.com/robcowart/elastiflow

Hugo_Slabbert1 · May 18, 2019, 4:26am

Also was a favourite last time this discussion popped up (in recent memory):

https://mailman.nanog.org/pipermail/nanog/2018-March/094490.html

Joe_Loiacono1 · May 18, 2019, 4:04pm

Dennis,

You might try FlowViewer

Fairly easy Linux install over top of SiLK, netflow capture and analysis software from Carnegie-Mellon. SiLK is very robust and FlowViewer provides a web-based interface with extensive analysis, graphing and tracking tools. Filtering includes by AS. You can create an MRTG-like set of long-term graphs for each AS and as a group of top 10 ASes (Last 24 Hours, 7 Days, 4 Weeks, 3 Years.)

Best,

Joe

Christian_Meutes · May 19, 2019, 1:29pm

ES, Kibana, pmacct and some glue (JSON to ES batching)

… and of course a lot of time and resources (eg. h/w).

Cheers
Chris

Dennis_Burgess · May 20, 2019, 1:36pm

Please let me clarify. Currently the Netflow data that this customer is sending does NOT supply AS information. So I need something to generate that AS data and display. The goal is to figure out where we need to peer next. Where the top traffic is coming in from (what AS) on our paid transit.

nanog9 · May 20, 2019, 1:42pm

Check out AS-Stats¹, with perl-ip2as

[1] https://github.com/manuelkasper/AS-Stats

Dennis_Burgess · May 20, 2019, 3:55pm

It specifically states it uses AS data from the netflow source. I don't have that

FROM website:
collects NetFlow v8/v9 AS aggregation records

Dennis Burgess,

Mike_Hammett · May 20, 2019, 4:03pm

Please let me clarify. Currently the Netflow data that this customer is sending does NOT supply AS information. So I need something to generate that AS data and display. The goal is to figure out where we need to peer next. Where the top traffic is coming in from (what AS) on our paid transit.

Alain_Hebert · May 22, 2019, 12:08pm

+1 for elasticflow

But make sure to clear the indexes, as it wasn’t included with the project, when we installed ours.

Here’s our solution that delete them after 90 days.

----- Crontab

0 12 * * * (cd /usr/local//scripts; ./_elastiflow_prune.sh) > /dev/null 2>&1

----- Content of the *_prune.sh for Linux

#!/bin/csh -f

set d_current=date "+%s"
set d_90=expr ${d_current} - $ 90 \* 24 \* 60 \* 60 $
set idx=date -d @${d_90} "+%Y.%m.%d"

curl -XDELETE

Mike_Hammett · May 22, 2019, 12:39pm

The last time I looked, Esastiflow didn’t accept a BGP session to learn ASes. Has that changed?

Jason_Lixfeld · May 22, 2019, 1:12pm

I loved using ElastiFlow, but we didn’t quite work out in the end. Here’s my $0.02 -

- ElastiFlow setup is easy-ish.
- ELK setup is easy-ish.
- Scaling ELK is not easy unless you know what you’re doing.

If you’ve got enough flows that you need to scale ELK, you’re probably also using multiple flow exporters, at which point this[1] could bite you and if ELK scaling was hard for you, dealing with this might not be trivial until Rob decides how best to bake a fix into EF.

I learned ELK because I wanted to use EF, but I only learned enough about ELK to get me by. Having to also learn about REDIS and having to learn more about ELK to make it work with REDIS and EF was a show stopper; I just didn’t have the time.

[1] https://github.com/robcowart/elastiflow/issues/205

Niels_Bakker · May 22, 2019, 1:34pm

* nanog@ics-il.net (Mike Hammett) [Wed 22 May 2019, 14:40 CEST]:

The last time I looked, Esastiflow didn't accept a BGP session to learn ASes. Has that changed?

You can put pmacct inbetween to alleviate this.

-- Niels.

Mike_Hammett · May 22, 2019, 1:37pm

nProbe as well. I was just checking if the setup was made simpler.