DDOS, IDS, RTBH, and Rate limiting

Mailman List Mirror (Read Only)
1

> Cisco ASRs and MXs with inline jflow can do hundreds of K flows/second
> without affecting packet forwarding.

Yes, i agree,those are good for netflow, but when they already exist in
network.

Does it worth to buy ASR, if L3 switch already doing the job
(BGP/ACL/rate-limit/routing)?

Not suggesting that anyone should change out their gear though per my other
message, I've seen SPAN make things go wonky on almost every vendor that
ISPs use for switching.

Well, if it is available, except hardware limitations, there is second
obstacle, software licensing cost. On latest JunOS, for example on EX2200,
you need to purchase license (EFL), and if am not wrong it is $3000 for
48port units.

So if only sFlow feature is on stake, it worth to think, to purchase license,
or to purchase server. Prices for JFlow license on MX, just for 5/10G is way
above cost of very decent server.

I believe that smaller MXs can run it for free. Larger providers we've
worked with often have magic cookies they can call in to get it enabled,
but I understand you're talking about the smaller-provider (or at least ~
10gig per POP across multiple POPs) case.

We see a lot of Brocade for switching in hosting providers, which makes
sFlow easy, of course.

> And with the right setup you can run FastNetMon or other tools in
> addition to generating flow that can be of use for other purposes
> as well...

Technically there is ipt_NETFLOW, that can generate netflow on same box,
for statistical/telemetry purposes. But i am not sure it is possible to
run them together.

At frac 10gig you can just open pcap on a 10gig interface on a Linux
box getting a tap, of course.

What we did was use myricom cards and the myri_snf drivers and take from
the single-consumer ring buffers into large in-RAM ring buffers, and
make those ring buffers available via LD_PRELOAD or cli tools to allow
flow, snort, p0f, tcpdump, etc to all be run at the same time at 10gig.

The key for that is not going through the kernel IP stack, though.

> But taps can be difficult or at least time consuming for people to
> put in at scale. Even, we've seen, for folks with 10G networks.
> Often because they can get 90% of what they need for 4 different
> business purposes from just flow :slight_smile:

About scaling, i guess it depends on proper deployment strategy and
sysadmins/developers capabilities. For example to deploy new ruleset
for my pcap-based "homemade" analyser to 150 probes across the country -
is just one click.

Sounds cool. You should write up that use case. Hopefully you've secured
the metadata/command push channel well enough :slight_smile:

Best regards,
Denys

Avi Freedman | Your flow has something to show you; can you see it? |
CEO, CloudHelix | (avi at cloudhelix dot com) | my name one word on skype |

2

> Cisco ASRs and MXs with inline jflow can do hundreds of K flows/second
> without affecting packet forwarding.

Yes, i agree,those are good for netflow, but when they already exist in
network.

Does it worth to buy ASR, if L3 switch already doing the job
(BGP/ACL/rate-limit/routing)?

Not suggesting that anyone should change out their gear though per my other
message, I've seen SPAN make things go wonky on almost every vendor that
ISPs use for switching.

Well, i always try to stay on safe side. Additionally, sure, i do
mirror for RX only, RX+TX often can exceed interface rate too :slight_smile:

Well, if it is available, except hardware limitations, there is second
obstacle, software licensing cost. On latest JunOS, for example on EX2200,
you need to purchase license (EFL), and if am not wrong it is $3000 for
48port units.

So if only sFlow feature is on stake, it worth to think, to purchase license,
or to purchase server. Prices for JFlow license on MX, just for 5/10G is way
above cost of very decent server.

I believe that smaller MXs can run it for free. Larger providers we've
worked with often have magic cookies they can call in to get it enabled,
but I understand you're talking about the smaller-provider (or at least ~
10gig per POP across multiple POPs) case.

We see a lot of Brocade for switching in hosting providers, which makes
sFlow easy, of course.

Oh, Brocade, recent experience with ServerIron taught me new lesson, that i can't
do bonding on ports as i want, it has limitations about even/odd port numbers and
etc.
Most amazing part i just forgot, that i have this ServerIron, and it is a place where
i run DDoS protection (but it works perfectly over "tap" way). Thanks for reminding
about this vendor :slight_smile:

> And with the right setup you can run FastNetMon or other tools in
> addition to generating flow that can be of use for other purposes
> as well...

Technically there is ipt_NETFLOW, that can generate netflow on same box,
for statistical/telemetry purposes. But i am not sure it is possible to
run them together.

At frac 10gig you can just open pcap on a 10gig interface on a Linux
box getting a tap, of course.

What we did was use myricom cards and the myri_snf drivers and take from
the single-consumer ring buffers into large in-RAM ring buffers, and
make those ring buffers available via LD_PRELOAD or cli tools to allow
flow, snort, p0f, tcpdump, etc to all be run at the same time at 10gig.

The key for that is not going through the kernel IP stack, though.

Ntop's pf_ring, which is basically same idea, but can run on Intel cards.
Just maybe because never had myricom in hands, and it is difficult to obtain
them here.

> But taps can be difficult or at least time consuming for people to
> put in at scale. Even, we've seen, for folks with 10G networks.
> Often because they can get 90% of what they need for 4 different
> business purposes from just flow :slight_smile:

About scaling, i guess it depends on proper deployment strategy and
sysadmins/developers capabilities. For example to deploy new ruleset
for my pcap-based "homemade" analyser to 150 probes across the country -
is just one click.

Sounds cool. You should write up that use case. Hopefully you've secured
the metadata/command push channel well enough :slight_smile:

For servers it is ssh with key authentication, and push system doesn't contain private key, it is forwarded
over ssh agent from developer pc. Sure, it is better also to sign by assymmetric crypto update also,
keep keys on smartcard, but in this case it is not necessary.

3

I just hope you're not talking FCX's.... if you upgrade those to 8.x firmware, you'll lose sflow on the 10gb ports. Once you upgrade, they send a corrupted sflow packet, and at *far* less then the rate that you configure. Even if you adjust your parser to compensate for the corrupt packet, they're still dropping the large majority of samples, making sflow pretty much useless.

It's been several months since we reported this, and we're still waiting on a fix.

4

Hello, folks!

Thank you for a very useful feedback! I'm so sorry for my negative
vision of netflow :frowning: It's nice protocol but I haven't equpment with
ability to generate netflow on wire speed and I use mirror/SPAN
instead.

I competely redesigned attack-analyzer subsystem and can process
sampled data now. I just added sFLOW v5 suport to FastNetMon and you
can try it now. In near future I will add netflow v5 support.

With sFLOW support my tool can detect attack on 40-100GE links and
more! Thanks for sFLOW architecture! :slight_smile:

Thank you!

5

Hello, folks!

Thank you for a very useful feedback! I'm so sorry for my negative
vision of netflow :frowning: It's nice protocol but I haven't equpment with
ability to generate netflow on wire speed and I use mirror/SPAN
instead.

I competely redesigned attack-analyzer subsystem and can process
sampled data now. I just added sFLOW v5 suport to FastNetMon and you
can try it now. In near future I will add netflow v5 support.

With sFLOW support my tool can detect attack on 40-100GE links and
more! Thanks for sFLOW architecture! :slight_smile:

You can check new version here: https://github.com/FastVPSEestiOu/fastnetmon

Thank you!

6

Good job - you should really go for NetFlow v9 when you can, as it supports IPv6 and MPLS labels.

Next would be IPFIX.

7

Hello, folks!

NetFlow v5 and v9 support have just added to FastNetMon:
https://github.com/FastVPSEestiOu/fastnetmon

Now you can catch DDoS attacks and collect data from sFLOW v5, NetFlow
v5/v9 and even from mirror port with PF_RING in one tool
simultaneously!

Will be very glad for feedback and testing!