How are these for CORE SWITCHES (distribution) compared to BigIron and the
CISCO 6509?
From what I have heard and reports they are very solid switches.
Thanks in advance
-Shazzy
Some things to know about them:
They use CPU to route ICMP just like all Extreme equipment (makes it
harder to diagnose network trouble using ICMP).
They have a 256k entry ipfdb (fastpath hardware L3 hostbased route-cache).
They're very quick and stable when it comes to forwarding traffic that has
a normal pattern, but they do not perform well when it comes to handling
stuff like DoS attacks that generates packets that are not in its ipfdb.
The last months virus attacks have not been fun to us (both the ICMP and
the scanning from infected customers and our aggregates being scanned from
infected internet hosts).
They do everything in hardware when it comes to access lists, QoS etc.
Either it does it in ASIC without performance impact or not at all.
Just like all other equipment you'd better look it thru thoroughly for
your application and check what drawbacks might hit you etc. I don't know
much about the BigIron. but it's hard to compare to a 6509 unless you know
what's in the 6509. Compare it to a Sup1A with older cards and the Black
Diamond is a performance screamer that'll do circles around the 6509,
bring out the OSMs and all the other 7600 stuff and that's a better core
router probably (but much much more expensive).
I like the fact that all Extreme equipment of the same generation (they
have two total) use the same ASICs and the same software and you can do
the same things in all of them. Very consistant.
> How are these for CORE SWITCHES (distribution) compared to BigIron and the
> CISCO 6509?
> >From what I have heard and reports they are very solid switches.Some things to know about them:
They use CPU to route ICMP just like all Extreme equipment (makes it
harder to diagnose network trouble using ICMP).
Actually, as far as I know, all switches and routers use the CPU to
process ICMP. It is a control protocol and the safest option is to ensure
the vendor has implemented some sort of CPU rate-limiting so it can't be
overwhelmed.
They're very quick and stable when it comes to forwarding traffic that has
a normal pattern, but they do not perform well when it comes to handling
stuff like DoS attacks that generates packets that are not in its ipfdb.
The last months virus attacks have not been fun to us (both the ICMP and
the scanning from infected customers and our aggregates being scanned from
infected internet hosts).
This is the kicker and real question: does it require the CPU to forward
regular traffic? I believe the answer is yes, the Extreme is a flow-based
architecture and the first packet of each unique flow (however it is
defined) will need to be processed by the CPU. This is why the problems
described above occur. The alternative is a packet-based architecure and
does not rely on the CPU for forwarding. It doesn't take a lot of packets
to overwhelm any CPU.
They do everything in hardware when it comes to access lists, QoS etc.
Either it does it in ASIC without performance impact or not at all.
Assuming the CPU doesn't have to process the first packet before it
reaches the ACL, QoS policy, etc..
andy
As long as you only use them for switching, they're fine 
For routing, I wouldn't touch em with a 10 foot pole, but I can also say
that for the BigIron, or the 6509.
If you want a router, buy a router...
Actually, as far as I know, all switches and routers use the CPU to
process ICMP. It is a control protocol and the safest option is to ensure
the vendor has implemented some sort of CPU rate-limiting so it can't be
overwhelmed.
I don't know of anyone else who *routes* ICMP. Yes, ICMP packets destined
for the router, but Extreme actually CPU route all ICMP packets passing
thru.
This is the kicker and real question: does it require the CPU to forward
regular traffic? I believe the answer is yes, the Extreme is a flow-based
architecture and the first packet of each unique flow (however it is
defined) will need to be processed by the CPU. This is why the problems
Yes, exactly what I'm saying. Flow here is defined as a destination IP
number.
described above occur. The alternative is a packet-based architecure and
does not rely on the CPU for forwarding. It doesn't take a lot of packets
to overwhelm any CPU.
Quite, 10kpps is enough, if even that.
> They do everything in hardware when it comes to access lists, QoS etc.
> Either it does it in ASIC without performance impact or not at all.Assuming the CPU doesn't have to process the first packet before it
reaches the ACL, QoS policy, etc..
Well, actually I believe ACLs are processed on ingress before being punted
to the CPU even though the flow hasnt been set up yet. This is the
observation I have seen so far anyway, but I am not 100% sure.
I can understand how a virus like Welchia can affect a flow-based
architecture like Extremes. I was under the impression that CEF enabled
Cisco gear wouldnt have this problem, but Cisco has instructions on their
webpage on how deal with it and cites CPU usage as the reason. With CEF I
thought the CPU wasn't involved? CEF is perhaps differently implemented on
different plattforms?
I think CEF in HW is the key, ASIC based and not Flow based.
I'm not all-knowlegable on which platforms do this, but the 7500, 12000,
2948G-L3, 4908 have it.
This is the kicker and real question: does it require the CPU to forward
regular traffic? I believe the answer is yes, the Extreme is a flow-based
architecture and the first packet of each unique flow (however it is
defined) will need to be processed by the CPU. This is why the problemsYes, exactly what I'm saying. Flow here is defined as a destination IP
number.
No... Flow is defined as at least the unique combination of source and
destination addresses, and, often, the unique combination of source and
destination IP addresses and port numbers + the layer 4 protocol used.
I can understand how a virus like Welchia can affect a flow-based
architecture like Extremes. I was under the impression that CEF enabled
Cisco gear wouldnt have this problem, but Cisco has instructions on their
webpage on how deal with it and cites CPU usage as the reason. With CEF I
thought the CPU wasn't involved? CEF is perhaps differently implemented
on different plattforms?
CEF is a flow-based solution much like Extreme's. There are enhancements
to CEF in some of Cisco's newer products (such as dCEF) which take some of
this off of the CPU.
Owen
> I can understand how a virus like Welchia can affect a flow-based
> architecture like Extremes. I was under the impression that CEF enabled
> Cisco gear wouldnt have this problem, but Cisco has instructions on their
> webpage on how deal with it and cites CPU usage as the reason. With CEF I
> thought the CPU wasn't involved? CEF is perhaps differently implemented on
> different plattforms?I think CEF in HW is the key, ASIC based and not Flow based.
I'm not all-knowlegable on which platforms do this, but the 7500, 12000,
2948G-L3, 4908 have it.
Yup. We have 6509s with Sup2/MSFC2/PFC2, and have had no problems with
ICMP in connection with recent virus/worm attacks.
Oh yeah, we also find the 6509s work very well as routers. Full routing
tables, etc. YMMV.
Steinar Haug, Nethelp consulting, sthaug@nethelp.no
I can understand how a virus like Welchia can affect a flow-based
>>> architecture like Extremes. I was under the impression that CEF
>>> enabled Cisco gear wouldnt have this problem, but Cisco has
>>> instructions on their webpage on how deal with it and cites CPU
>>> usage as the reason. With CEF I thought the CPU wasn't involved?
>>> CEF is perhaps differently implemented on different plattforms?
>>
>> I think CEF in HW is the key, ASIC based and not Flow based. I'm
>> not all-knowlegable on which platforms do this, but the 7500,
>> 12000, 2948G-L3, 4908 have it.
Whether CEF is ASIC-based or in software is not an issue as such.
CEF is _not_ flow routing; CEF tables contain only destinations (not
source+destination or port numbers), they contain entire destination
prefixes not single IP addresses, they are pre-built and maintained
from the routing tables rather than added entry-by-entry as traffic
arrives.
CPU is still an issue in some cases because when a destination is on
an attached network and has no ARP entry, there is no CEF adjacency
for it; accordingly, when traffic arrives for that destination it is
punted to process level in order to trigger an ARP. Once the ARP
succeeds the adjacency is set up and further packets are routed via
CEF (whether hardware or software according to platform). However, if
the destination is not adjacent, this does not apply (since the ARP
entry for the next-hop router will already be present) and all packets
will be CEF-switched.
(Enabling CEF is often mentioned in Cisco docs as a workaround for
worm traffic problems.)
> Actually, as far as I know, all switches and routers use the CPU to
> process ICMP. It is a control protocol and the safest option is to ensure
> the vendor has implemented some sort of CPU rate-limiting so it can't be
> overwhelmed.I don't know of anyone else who *routes* ICMP. Yes, ICMP packets destined
for the router, but Extreme actually CPU route all ICMP packets passing
thru.
I'm not 100% sure what your trying to say above, but all I'm refering to
is packets destined towards the device itself.
> This is the kicker and real question: does it require the CPU to forward
> regular traffic? I believe the answer is yes, the Extreme is a flow-based
> architecture and the first packet of each unique flow (however it is
> defined) will need to be processed by the CPU. This is why the problemsYes, exactly what I'm saying. Flow here is defined as a destination IP
number.
Maybe, maybe not. It could be more granular then that, which would allow
for addition functionality based on other fields in the IP header. Every
additional field it uses to define a flow increase the number of packets
that reach the CPU expotentially. Destination could be enough though with
the way some viruses scan address space at a rapid pace all creating new
destination flows.
Also, the original question was about switching. For layer-2 flows with
unique MAC addresses reach the CPU as well? Probably.
> described above occur. The alternative is a packet-based architecure and
> does not rely on the CPU for forwarding. It doesn't take a lot of packets
> to overwhelm any CPU.Quite, 10kpps is enough, if even that.
Have you tested this? I'm always interested in different vendor's flow
setup rates.
> > They do everything in hardware when it comes to access lists, QoS etc.
> > Either it does it in ASIC without performance impact or not at all.
>
> Assuming the CPU doesn't have to process the first packet before it
> reaches the ACL, QoS policy, etc..Well, actually I believe ACLs are processed on ingress before being punted
to the CPU even though the flow hasnt been set up yet. This is the
observation I have seen so far anyway, but I am not 100% sure.
I'm not sure this would make sense. How would the device know to drop or
forward the packet if a flow, even if it is a drop flow, hasn't been
created?
I can understand how a virus like Welchia can affect a flow-based
architecture like Extremes. I was under the impression that CEF enabled
Cisco gear wouldnt have this problem, but Cisco has instructions on their
webpage on how deal with it and cites CPU usage as the reason. With CEF I
thought the CPU wasn't involved? CEF is perhaps differently implemented on
different plattforms?
CEF certainly can limit the amount the CPU is used, and DCEF even more.
I'm not sure that Extreme has an equivilant feature though.
andy
> I don't know of anyone else who *routes* ICMP. Yes, ICMP packets destined
> for the router, but Extreme actually CPU route all ICMP packets passing
> thru.I'm not 100% sure what your trying to say above, but all I'm refering to
is packets destined towards the device itself.
Which I was not.
Maybe, maybe not. It could be more granular then that, which would allow
for addition functionality based on other fields in the IP header. Every
It isn't. The ipfdb is basically a DestIP, port and mac address in its
pursest form. This is the default.
Also, the original question was about switching. For layer-2 flows with
unique MAC addresses reach the CPU as well? Probably.
It would in basically all switches I know of.
Have you tested this? I'm always interested in different vendor's flow
setup rates.
Well, empirical studies say that "clear ipfdb" on a full ipfdb table makes
the switch become unresponsive and fully occupied with ipfdb entry
creation for something like 10-40 seconds. No, I have not measued it more
closely than that.
I'm not sure this would make sense. How would the device know to drop or
forward the packet if a flow, even if it is a drop flow, hasn't been
created?
Because the ACLs aren't applied to flows but are matched separately before
a forwarding decision has been made. Think of it as a PXF grid that does
things before the CPU.
As far as I know they do this:
L3 packet comes in.
It's matched for ACL (ACLs are used to QoS stuff as well)
matched for policy routing
after this, it's checked in the ipfdb and if it's not found then punted to
the CPU. If it's an ICMP packet it's always punted to the CPU.
So dropping packets is all done in ASIC.
Redbacks SmartEdge 800 replies to atlest ICMP ECHO in the line card ASIC.
//tlund