hey there
any recommendations on freeware flow analysis tool which can show the flow
not only per prefix basis but also show asn and/or country/region as well?
Juniper only.
feel free to contact on/off list.
mehmet
Juniper's flow export is just like everyone else's (*), so any tool will
do the same thing. Country/region analysis would depend on third party
geolocation services, which have nothing to do with netflow. 
(*) Well, except M/T/MX only support NetFlow v5/v8 in the free software
based sampling mode, you need an expensive services card and software
license to do v9 for some reason.
Oh and the sFlow on EX is actually pretty cripled when used for routing.
It's missing support for a bunch of important extended message tpes, and
doesn't fully populate all of the fields of the message types it does
send. For example you won't get any data on ASNs, nexthops, dest
ifindexes, or even netmasks of the src/dst route the flow matched,
making it pretty darn useless for a lot of tasks. It's functional if
you're just analyzing L2 networks at any rate.
Agree people spend some money and hence tend to expect something in
return. But it's also true those good souls developing free collectors
(to stay in topic with the OP) sometimes come to the rescue: ASNs, BGP
next-hop, routes, netmasks can be all looked up at the collector at
pretty no major effort. Variety of methods available depending on the
collector, in place or a posteriori, file or BGP lookup - it's matter
of selecting what fits better the specific job.
Plus, sFlow flow samples are rather successful offsetting some partial
vendor implementations by carrying portion of the sampled packet - in
one go MAC addresses, VLANs, 802.1p, MPLS labels, EXP bits, BoS, etc.
are at the collector doorstep.
OTOH it would be nice to see one day those NetFlow v9 MAC address fields
populated on higher-grade boxes, say, to facilitate analysis of public
peering at internet exchanges ...
Cheers,
Paolo
Mehmet Akcin (mehmet) writes:
hey there
any recommendations on freeware flow analysis tool which can show the flow
not only per prefix basis but also show asn and/or country/region as well?
Juniper only.
Hi Mehmet,
As someone else answered, export v9 flows and then run something
like NFsen or Netflow Dashboard.
On that, Geo::IP will provide country/region info (for example):
http://search.cpan.org/~borisz/Geo-IP-1.38/lib/Geo/IP.pm
Cheers,
Phil
This can be done with mac accounting in your platform. It may require ordering the right PIC in this case, but is easy to collect that data.
- Jared
MAC accounting is not a solution, except for basic things, for at
least two reasons: 1) it is coarse-grained and 2) its segregation
from flow accounting prevents having all the eggs in one basket.
Cheers,
Paolo
Yes you can do an offline routing lookup to try and reconstruct some
missing data (or do some even more interesting analysis, as described in
http://www.nanog.org/meetings/nanog35/presentations/steenbergen.pdf),
but it isn't always a practical solution to missing netmask, nexthop,
and dest ifindex data.
Remember that every RIB in your network can and will have a unique best
path selection (thanks to the EBGP > IBGP rule if nothing else), and if
you have a network of any size at all you'll probably have to deal with
multiple exits to the same network. Even if you were only concerned with
analyzing external traffic, you'd still need to collect a RIB per edge
router using an IBGP feed. In my network this would put you well over 10
million paths, and consume several gigs of ram, not to mention the load
of doing the routing lookups themselves. If you wanted to do traffic
analysis inside your network you'd need a feed from every router, and
maybe even active participation in your IGP. It CAN be done, but it's
not pretty, and I don't think any existing free software has been tested
under these kinds of conditions.
So when a vendor says "we support sFlow", make sure they actually
support the message types and fields you need.
>
Remember that every RIB in your network can and will have a unique best
path selection (thanks to the EBGP > IBGP rule if nothing else), and if
you have a network of any size at all you'll probably have to deal with
multiple exits to the same network. Even if you were only concerned with
analyzing external traffic, you'd still need to collect a RIB per edge
router using an IBGP feed.
Agree.
In my network this would put you well over 10 million paths, and consume
several gigs of ram, not to mention the load of doing the routing lookups
themselves.
If the collector is able to keep up with the pace of the export protocol,
it will not get killed by a couple of BGP lookups per flow/sample. About
the memory figure: reducing information overlap among the RIBs results in
storing a full BGP table in few tens of MBs. On 10M paths that translates
in less than a GB of memory.
analysis inside your network you'd need a feed from every router, and
maybe even active participation in your IGP.
This is separate thing. Relying only on telemetry information for such a
task is one of the possible approaches - perhaps the quicker, if enabled
ad-hoc for troubleshooting, but not necessarily the smarter.
It CAN be done, but it's not pretty, and I don't think any existing free
software has been tested under these kinds of conditions.
Define pretty: is it pretty to move control-plane information over and
over via NetFlow or sFlow? But most importantly: why passing the concept
challenging conditions is something free software should be run far away
from?
So when a vendor says "we support sFlow", make sure they actually
support the message types and fields you need.
Indeed, agree.
Cheers,
Paolo
Hi again,
I think many of you got stuck at the "freeware" part however if there are good suggestions you have which would help achieving what I would like to achieve by paying for a software, I am open to listen suggestions on that as well.
just thought would mention
mehmet
Hi,
any recommendations on freeware flow analysis tool which can show the
flow not only per prefix basis but also show asn and/or country/region
as well? Juniper only.feel free to contact on/off list.
Juniper's flow export is just like everyone else's (*), so any tool will
do the same thing. Country/region analysis would depend on third party
geolocation services, which have nothing to do with netflow.(*) Well, except M/T/MX only support NetFlow v5/v8 in the free software
based sampling mode, you need an expensive services card and software
license to do v9 for some reason.
Shouldn't there be a (**)
(**) Also Except for MX'es with trio chipsets. These can do
inline-jflow that export to IPFIX (modified netflow v9)
All of the open source collector solutions I've tried that can handle
v9 cannot handle IPFIX from the trio cards.
Richard; Do you have something that handles IPFIX?
Bas
Yes there's that too. I haven't actually gotten around to testing the
Trio specific Netflow capabilities yet, but supposedly they only support
IPFIX when using the "built in" sampling capabilities. If you want v9
you'll still need a Multiservice DPC, or you can always stick to classic
RE-sampled v5/v8.
IPFIX is effectively "netflow v10", it's largely based off of v9, but
it's just different enough to be incompatible. Of course it's close
enough that it shouldn't be THAT much work if you already have an
existing v9 parser, but I don't know what software actually supports it
today. The only flow collector implementation which I've spent any
amount of time looking at besides the stuff I've written myself is
pmacct, which IMHO shows great promise, but I don't believe it supports
IPFIX yet. For my purposes I'd have been just as happy if everyone had
standardized on sFlow (especially since I already wrote a parser for it
:P), but alas it isn't meant to be.
Some differences between v9 and IPFIX that googling turned up:
http://www.plixer.com/blog/netflow/what-is-ipfix-vs-netflow-v9/
True: IPFIX is work in progress at the moment and expected to converge
to a public release soon (probably it will be in the CVS by end of the
year). This is thanks to somebody on-list which is kindly offering lab
access to traffic generator, Juniper MX chassis and Trio card.
Cheers,
Paolo