Hello,
I'm currently writing a paper for school and I talk about net neutrality
which brings the subject of NetFlow/IPFIX.
Should those protocols be considered as tools to perform DPI ?
Thanks,
Antoine.
That question can be taken a couple of ways. Netflow is useful for
calculating information useful to providers and operators through sampling
of data on high bandwidth links, where performing DPI is not feasible or
desired. It is not a robust solution for DPI - or analysis of higher layer
packet data, which is typically performed by a mirrored interface or an
inline box/firewall that can perform high speed forwarding.
No - they're flow telemetry exported by routers and switches, and they provide layer-4 information.
It's possible with Cisco Flexible NetFlow and with PSAMP exported over IPFIX to get packet contents; however, few if any collection/analysis systems utilize either extended telemetry format, to date. I've never seen either implemented in a production network.
NetFlow and IPFIX are primarily used for security purposes such as DDoS detection/classification/traceback and botnet C&C identification; for traffic engineering analysis; capacity planning analysis; for troubleshooting; and for billing purposes. Flow telemetry is an essential tool that ISPs and larger enterprises utilize in order to get a view into their network traffic, because it scales for large networks - and it does so because it doesn't typically include packet payloads, just the layer-4 information. It's sort of like a near-time mobile phone bill for the network.
'DPI' generally (but not always) refers to devices which are placed inline and perform full multi-packet payload reassembly and inspection. The term has been used (and misused) so broadly as to becoming essentially meaningless.
NetFlow and IPFIX are merely telemetry formats used by network engineers for the purposes noted above.
This presentation talks about how NetFlow is used by network operators:
<https://app.box.com/s/mnshn99c13uekrggy99b>
Network neutrality is largely an issue of policy and of economics, not of technology, per se.
Another role for IPFIX/NetFlow in the context of DPI (on top of
PSAMP that was already mentioned by Roland) is to serve as a
transport mechanism to travel flow data along with their DPI
classification from probes to remote collectors, for persistent
storage, analysis, etc.
This model is supported on the export side by Cisco with their
NetFlow/NBAR integration and on the collection side by some
collector. Maybe this (*) recent university report from the
good Tomasz Bujlov can be of interest to you. What i described
above is specifically captured in chapters 3.4.6 and 3.5.6.
Cheers,
Paolo
As you'll note in reading that report, NBAR didn't seem to work very well for them; I haven't run across its use in any ISP network, on ISP-grade hardware (i.e., not a small software-based router like a 7200), or even in a large-scale enterprise environment.
Again, I haven't yet run across anyone actually using this on a production network. It would be very interesting to hear from someone with first-hand experience with NBAR exported over Flexible NetFlow in a production environment.
Also, it's important to note that this sort of thing doesn't scale across networks of any real size/speed. There's a great deal of potential utility in the security, troubleshooting, and traffic engineering arenas for on-demand triggered packet sampling of this nature, but so far, the control-plane hooks aren't really there to do this in a programmatic manner (one presumes that SDN of one flavor or another might provide these capabilities). That was my biggest gripe about Flexible NetFlow when I was at Cisco, and one which I felt ensured the technology wouldn't make it into production networks - no organic control-plane interface.
So, perhaps now we can de-conflate flow telemetry and 'DPI', since the real-life export, collection, and analysis of anything other than layer-4 information via flow telemetry isn't at all commonplace (if it in fact exists at all) on production networks), at this juncture.
'DPI' generally alludes boxes positioned at points of ingress/egress symmetry (either natural or purposely engineered) within a network. Flow telemetry per se is not really within the rubric of 'DPI'.
Please note NBAR/NetFlow integration wanted to be an example of
using NetFlow/ IPFIX as a transport for DPI classification info
(where classification could be performed with any other in-line
technology than NBAR).
Whether NBAR works or does not as a classification technology is
out of scope for me here - and seems also out of the op request.
Inline:
So, perhaps now we can de-conflate flow telemetry and 'DPI', since the real-life export, collection, and analysis of anything other than layer-4 information via flow telemetry isn't at all commonplace (if it in fact exists at all) on production networks), at this juncture.
I disagree if anybody conflates here. I don't. I see two disjoint
pieces: classification technology and transport of classification
info to a central location. IPFIX, for example, is general (and
standardized) enough to transport/encapsulate other info than just
flow info, this might include DPI classification or other stuff.
You can also read this as: if you have to travel some info, why re
invent the wheel and not leverage a general-enough, standardized
transport protocol (that btw you can contribute at any point to
enhance if not satisfactory enough)?
And please it's nice to have different positions - no need to escalate.
Cheers,
Paolo
Thank you Matt (offlist), Dan, Roland and Paolo for your answers !
Antoine.