sFlow vs netFlow/IPFIX

I view sflow as subset of NetflowV9 V10/IPFIX. You could produce sflow
behaviour in IPFIX, by adding record of packet sample, and exporting
immediately instead of keeping state of flows.
However you couldn't produce IPFIX behaviour in sflow, inherently so.

sflow is older than IPFIX (v10) or v9 netflow, and I'm guessing no one
would invent sflow today, they'd instead specify some restricted IPFIX
with same behaviour.
I completely understand why sflow was needed netflowv5 time, but I
don't really see much point there now. It just means that collectors
need to be more complicated than they must, by having two parsers.

LINX use IPFIX (which is derived from NetFlow) for the Juniper LAN.

The Extreme LAN uses sFlow.

Edward Dore
Freethought Internet

> That's interesting, given that most larger routers don't support 1:1.

I find that strange, because if you're doing in in HW, doing hash
lookup for flow and adding packets and bytes to the counter is cheap.
It's expensive having lot of those flows, but incrementing their
packet and byte counter isn't.

I know that all JNPR Trio kit (MX, T, EX9k...) do 1:1. I guess if
you're doing it in LC CPU things are very different.

A relevant question might be if the Trio hardware can do 1:1 while
handling multiple ports of line rate DDoS traffic consisting of small
packets with different port numbers (i.e. high pps traffic resulting
in basically 1 flow per packet). No, I don't know the answer (but I
suspect it might be negative).

Here we're using Trio hardware with 1:100 sampling, and are reasonably
happy with the results.

Steinar Haug, AS2116

Hello!

Nice information. Very interesting architecture. They are using L3 on
IX? How big Juniper Lan in comparison with Extreme lan?

A relevant question might be if the Trio hardware can do 1:1 while
handling multiple ports of line rate DDoS traffic consisting of small
packets with different port numbers (i.e. high pps traffic resulting
in basically 1 flow per packet). No, I don't know the answer (but I
suspect it might be negative).

I cannot see why not, it's cheap. You're doing 1-2 LPM on the packet,
QoS lookup, ACL lookup, incrementing various counters, etc., adding
one hash lookup and two counters is not going to be relevant cost to
the lookup time.

Having many entries in the hash table is an issue, incrementing their
counters is not.

Hi Pavel,

The Juniper LAN is VPLS and the Extreme LAN is standard layer 2 with EAPS instead of *STP.

The Juniper LAN peaks at ~2.7Tbps (https://stats.linx.net/lans#lon1), whilst the Extreme LAN peaks at ~0.6Tbps (https://stats.linx.net/lans#lon2).

Edward Dore
Freethought Internet

Thanks! Very interesting. Will dig into details :slight_smile:

Saku Ytti wrote:

I cannot see why not, it's cheap. You're doing 1-2 LPM on the packet,
QoS lookup, ACL lookup, incrementing various counters, etc., adding
one hash lookup and two counters is not going to be relevant cost to
the lookup time.

depends on what you define by "cheap". Netflow requires separate packet
forwarding lookup and ACL handling silicon.

Having many entries in the hash table is an issue, incrementing their
counters is not.

it is certainly an issue if you get splatted with lots of discrete junk
flow, yes.

Neither of these are a problem for sflow. It just plucks packets out of
the data plane at a pre-defined rate and forwards their headers to the
collector. So long as your sampler is accurate, it's great.

Nick

Roland Dobbins wrote:

Inconsistent stats, lack of ifindex information.

I've not yet come across an sflow implementation which didn't fill out
the ifindex field. No doubt they exist.

Not sure what you mean by "inconsistent stats".

Nick

Pavel Odintsov wrote:

From hardware point of view almost all brand new switches support
sflow free of charge (no additional licenses or modules). But be
aware, Cisco do not support this protocol at all (that's pretty weird,
really).

sflow is supported on the Nexus 3k range, but it's available as ingress
+ egress only, non-configurable (despite the fact that the option to
configure is available at the chipset level). This means that you can't
define an account perimeter on the switch, which makes it mostly useless
from a production point of view. Pretty much every other switch on the
market will allow you to specify either ingress or egress.

The only consistent explanation I can suggest for Cisco's vehement
antipathy towards sflow is "not invented here". It's supported in
hardware by several of the other chipsets that Cisco uses, but left
non-enabled in software.

TBH it's a short sighted and disappointing approach to telemetry.

Nick

depends on what you define by "cheap". Netflow requires separate packet
forwarding lookup and ACL handling silicon.

That's not inherently so, it depends how specialised your hardware is.
If it's very specialised like implementing just LPM, sure. If it's
NPU, then no, that's not given.

The cost is many entries in the hash table, not updating them. But if
you'd emulate sflow behaviour in IPFIX then you don't need the hash
tables or the counters.

Neither of these are a problem for sflow. It just plucks packets out of
the data plane at a pre-defined rate and forwards their headers to the
collector. So long as your sampler is accurate, it's great.

ACK and as in explained in earlier post, there is nothing stopping
from IPFIX working like this. sflow is subset of what's possible in
IPFIX.

depends on what you define by "cheap". Netflow requires separate packet
forwarding lookup and ACL handling silicon.

That's not inherently so, it depends how specialised your hardware is.
If it's very specialised like implementing just LPM, sure. If it's
NPU, then no, that's not given.

I don’t think anyone uses dedicated Netflow HW these days. The ASICs have functionality for other things like mirroring, etc. which are augmented for Netflow use. Usually it’s a mix of dedicated functions in the ASICs and then the LC CPU and general CPU on some platforms. Really in the end the router is doing something like SFlow internally.

The cost is many entries in the hash table, not updating them. But if
you'd emulate sflow behaviour in IPFIX then you don't need the hash
tables or the counters.

It would be interesting to get some data from vendors on what the actual limitation is. I know with some new platforms like the NCS 55XX from Cisco (BRCM Jericho) it has limited space for counters, but I don’t know if that contributes to its minimum 1:8000 Netflow sampling rate. The new PTX FPC supporting Netflow has a minimum of 1:1000.

Phil

Are they are doing netflow in HW at all? To me it sounds like they
might be doing it in LC CPU and HW is only doing sampled punting.
Which would explain the performance hit.
I would be very surprised if Jericho could do netflow in HW. And I'm
100% sure PE chip can't do netflow in HW.

Yep, Broadcom doing thing right way! :slight_smile:

But unfortunately they (Cisco Nexus) are pretty expensive and fairly
new for DC and ISP market. It's pretty rare to find big company with
switching backbone on Nexus switches.

But I like this direction of switch silicom unification :slight_smile: Focus moved
from "network brands" with Not Invented Here syndrome to enough smart
agnostic hardware vendors.

Not all of them, just the Nexus 9000, IIRC.

Mark.

As opposed to?

We are looking at the Nexus 7700 for 100Gbps core switching.

Mark.

As opposed to older Cisco switches. Btw, 100GE is pretty new and
actually I have experience only with Extreme Black Diamond 8.

I don't agree with that statement (about rare to find big companies using Nexus). If you want 10 gig/40 gig (or 100 gig soon) your options are Cisco Nexus/Arista/Juniper QFX...some periphery devices as well, but the majority use one of those 3.

The merchant silicon based switches are pretty reasonably priced too.

As opposed to older Cisco switches.

Well, every vendor has older switches.

Btw, 100GE is pretty new and
actually I have experience only with Extreme Black Diamond 8.

Does not mean the Nexus is a bad choice for high capacity core
switching. Just means you know Extreme.

Mark.

Yep, actually do not mean. I've never used Nexus and haven't any
experience with it :slight_smile: I mentioned this in original message. I'm pretty
sure it's awesome switch. But as I haven't any experience I do not
known cons and pros about it.