Netflow collector that can forward flows to another collector based on various metrics.

Drew_Weaver · January 21, 2021, 2:31pm

Good morning everyone,

I am looking for a Netflow collector that can forward flows based on src ip/src net dst ip/dst net to another collector in either real or near time.

If it can be configured via an API that is even better than having to edit configuration files.

If anyone has any suggestions I would appreciate it.

Thanks,

-Drew

Mailman · January 21, 2021, 2:39pm

Hi,

I don’t know if pmacct has an API for it, but it can replicate netflow and also filter what it is forwarding.

github.com

pmacct/pmacct/blob/master/QUICKSTART

pmacct [IP traffic accounting : BGP : BMP : RPKI : IGP : Streaming Telemetry]
pmacct is Copyright (C) 2003-2021 by Paolo Lucente


TABLE OF CONTENTS:
I.	Daemons and plugins included with pmacct distribution
II.	Configuring pmacct for compilation and installing
III.	Brief SQL (MySQL, PostgreSQL, SQLite 3.x) setup examples
IV.	Running the libpcap-based daemon (pmacctd)
V.	Running the NetFlow/IPFIX and sFlow daemons (nfacctd/sfacctd)
VI.	Running the NFLOG-based daemon (uacctd)
VII.	Running the pmacct IMT client (pmacct)
VIII.	Running the RabbitMQ/AMQP plugin
IX.	Running the Kafka plugin
X.	Internal buffering and queueing
XI.	Quickstart guide to packet classification
XII.	Quickstart guide to setup a NetFlow/IPFIX agent/probe 
XIII.	Quickstart guide to setup a sFlow agent/probe 
XIV.	Quickstart guide to setup the BGP daemon
XV.	Quickstart guide to setup a NetFlow/IPFIX/sFlow replicator

This file has been truncated. show original

Beginning line 2093

Kind regards

Karsten

Johan_Hedberg · January 21, 2021, 2:47pm

I’ve been using samplicator for a few years for this, it can be configured to forward based on sender ip/net, but it does not have an API. I’m using it because it’s small, simple and does only one thing.

//JH

Michael_Krygeris · January 21, 2021, 2:53pm

Plixer Replicator will do this via REST API is you are looking for a commercial solution.
If you’re looking for a free solution, Samplicator will do this via config file.
Neither is a “collector” as neither stores the flows. They simply forward/copy UDP streams based on a set policy. It sounds like this is what you are after.

(Full disclosure I works for Plixer)

Mike Krygeris

Joe_Loiacono1 · January 21, 2021, 4:15pm

You might try the SiLK offering from Carnegie-Mellon’s CERT team. A netflow/sflow collector with full tool suite.

Very robust, fast and free.

Simon_Leinen · January 21, 2021, 5:08pm

Speaking as the maintainer of samplicator, I'm not sure it's what Drew
is looking for.

Samplicator just sends copies of entire UDP packets. It doesn't
understand NetFlow/IPFIX or whatever else those packets might contain.

If I understand correctly, drew wants to forward some of the
NetFlow/IPFIX flows, based on source/destination addresses *within those
flows*. Samplicator cannot do that (by a long shot).

pmacct sounds like a good suggestion.

(I used to have a Lisp program that could also do this, and adding an
API would have been trivial... but the program has been decommissioned
recently after >20 years of service. Also I never got around to
cleaning that up so that I could distribute the source.