switch 10G standalone TOR, core to DC

Mailman List Mirror (Read Only)
1

Hello,

I looking some 10G switches, it should work as TOR or core in DC. It should have more than 40 port 10G in one unit, wirespeed L2 L3, with virtual routers and some other ip functions like some BGP, OSPF, policy routing, 1-2U, MLAG, g.8032 (ERPS) trill-like ?

Other important features are big port buffers ( something similar to Juniper EX8200 - 512 MB per slot), defined counters accessible via snmp (like in junos), L3 statistics accessible via snmp

Extreme 670 looks good but they have small port buffers. It can be also some small chassis with line cards but the cost per 10G ports is too big..

What vendor, model You prefer or suggest as a solution ?

thanks for help
best,
Peter

2

the extreme x670, juniper ex4550, brocade ICX6550 and arista 7150 will most
of this, and probably many others too.

None of them will do trill. The Extreme X670 and Juniper EX4550 will both
do VPLS, though. The X670 won't do BGP.

You won't find a box of this form with large port buffers. There don't
appear to be any of these boxes on the market at the moment, probably
because none of the switch vendors want to bite the bottom out of their
more lucrative chassis-based switches. This is a good market opening for a
new vendor - there is no technical reason why this couldn't be done.

Nick

3

although everyone here seems to hold Cisco in contempt, the Nexux 5548 is a
rock-solid switch - at least that has been my experience with it.

4

Hi,

    I do suggest you go over EN offering with a fine tooth comb.

    We experienced a whole lot of issues with 6 x650:

        . from hardware licensing (start at shipping from the fab and
not when the customers get them);
        . software licensing (have to license every box even the ones in
the labs);
        . known eeprom defect limiting upgrade from XOS 12 to 15;
        . 1 vlan-translation causing all sort of head-aches with
port-grouping (ether-channel);
        . EAPS packets being silently filtered out of VMAN's when you do
not use the Core license;
        ( Undocumented and that is not acceptable when trying to
transport customers owns EAPS traffic on their VLAN's )
        . no VLAN flapping logging;

    Don't get me wrong, they are good campus switches... just not
designed for "our" L2 Core purposes.

    And the Licensing is just an exercise in frustration. I can
understand the business purpose, just not the way they go about doing it.

    As for L3 support, it is fine:

        . include IP tracking in VRRP with is a plus for us
        . Virtual Routers

    We don't need them for BGP and we do not have a MPLS network yet.

    As for the x670, maybe most of the hardware issue has been
addressed, but I doubt the licensing and undocumented limitations is better.

    PS: We're using them (x650), and are planning to keep
using/recommending EN products, but it did cost us a lot of man hours
and un-planned crashes that could have been prevented with better
documentation and support.

    Good luck with your project =D

5

Peter,

Network visibility wasn't mentioned as a requirement, but it is worth
considering since the ToR switches are the best place monitor server
network I/O, tunneled traffic (VxLAN, GRE etc), storage (iSCSI, FCoE,
HDFS etc).

The Nexus 5548 switch does not include monitoring (i.e. no
NetFlow/sFlow). The Nexus 3048, along with all the other 10G ToR
switches so far mentioned on this thread, supports sFlow and provides
wire speed 10G/40G monitoring.

The following article provides additional background:

http://blog.sflow.com/2012/02/10-gigabit-ethernet.html

Cheers,
Peter

6

Someone use this switches ?

1.
Alacatel lucent omniswitch OS6900-X40
Deep packet buffers for simultaneous
high-burst absorption in all ports
gpl 28k$

2.
Hp 5900 af 48xg
large buffer options - configurable buffers
gpl 30k$

What is, exactly, buffer size ? I can't find in documentation

best,
Piotr

7

W dniu 2013-01-30 12:59, Ingo Flaschberger pisze:

2.
Hp 5900 af 48xg
large buffer options - configurable buffers
gpl 30k$

small: Memory and processor
     512 MB flash, 2 GB SDRAM; packet buffer size: 9 MB

http://h17007.www1.hp.com/us/en/products/switches/HP_5900_Switch_Series/index.aspx#Layer%202%20switching
Tab models

small in 5900 series:

High-performance 10 GbE switching � cut-through and nonblocking architecture delivers industry-leading low latency (~1 microsecond) and very demanding enterprise applications; the switch delivers a 1.28 Tbps switching capacity and 952.32 Mpps packet forwarding rate in addition to incorporating 9 MB of packet buffers

or big in 5920:

High-performance 10GbE switching � enables you to scale your server-edge 10GbE ToR deployments with 24 high-density 10GbE ports delivered in a 1RU design; delivers a 480 Gbps (357.12 Mpps) switching capacity in addition to incorporating 3.6 GB of packet buffers

Ultra-deep packet buffering � provides up to a 3.6 GB packet buffer to reduce network congestion at the I/O that is associated with the heavy use of server virtualization, as well as bursty multimedia, storage applications, and other critical services

http://h17007.www1.hp.com/us/en/products/switches/HP_5920_Switch_Series/index.aspx

8

this is incorrect: the ex4550 will do l2vpn/l3vpn but not vpls. The X480
does vpls, but not the X670.

Nick

9

Well, talking about HP�s A5920/A5900 series. Last time I was looking, their virtual routing instances haven�t supported IPv4 multicast, nor IPv6 multicast/unicast, nor any policy based routing.

Michael

10

Hi Peter,
http://www.aristanetworks.com/media/system/pdf/Datasheets/7050S_Datasheet.pdf
Arista 7050S-64 48 x 10GE + 4 x 40 GE, price around 25k$ in gpl.
Large buffers, supports MLAG, DCB, wire-speed L2/L3 (OSPF,BGP), but doesn't have any kind of TRILL implementation.
Have it in production, but for now using for L2 only with MLAG.
As option also can be considered:
Brocade VDX6720, has own TRILL-like protocol to make STP-free topology, also can do L3, DCB but pay attention licensing is painful with Brocade.
Best Regards,Sergey

11

I normally just lurk but I thought I would post to clear up the confusion.
Full disclosure, I am an Extreme Networks TAC engineer.

The x450 does not support any VPLS/H-VPLS/MPLS and is discontinued. It was
replaced with the x460 which does support VPLS/H-VPLS/VPWS. The x480 and
x670 both support VPLS/H-VPLS/VPWS.

The x460, x480 and x670 all support BGP. However, only the x480 can hold
the BGP full-view in hardware. So while you can run BGP on the x460 or
x670 they are really only suitable for iBGP.

All switches require a Core license to run BGP and an MPLS license to run
VPLS/H-VPLS/VPWS.

Andrew

12

Thanks for the clarification on this. The data sheet on the x670 doesn't
actually mention vpls:

www.extremenetworks.com/libraries/products/DSSumX670_1777.pdf

... just that there is an mpls feature set license, but with no details of
what it contains.

Normally vendors can't tell enough about useful features like this, so in
the absence of mentioning it I had assumed incorrectly that it wasn't
supported on this platform. Maybe you could get the documentation updated
to include this information, because this is an important feature?

Nick

13

Thanks for pointing that out. Documentation folks have been alerted. For
a quick comparison of all switches I like the Comparison Guide the best.
http://www.extremenetworks.com/libraries/products/MSComparisonChart_1636.pdf

--Andrew

14

W dniu 2013-02-07 22:54, Sergey Marunich pisze:

Hi Peter,
http://www.aristanetworks.com/media/system/pdf/Datasheets/7050S_Datasheet.pdf
Arista 7050S-64 48 x 10GE + 4 x 40 GE, price around 25k$ in gpl.
Large buffers, supports MLAG, DCB, wire-speed L2/L3 (OSPF,BGP), but doesn't have any kind of TRILL implementation.

from documentation:

shared 9 MB packet buffer
pool that is allocated dynamically to ports that are congested

9MB is a standard size of port buffers..

regards,
Piotr

15

That's pretty standard for a cut-thru ToR switch of this style. Cut-thru
switches generally need a lot less packet buffer space than store-n-forward
switches. Also, ToR boxes tend not to have complex qos requirements.

Having said that, you need to be careful deploying small-buffer boxes. If
you're not careful, you will end up with bad packet loss.

Nick

16

Anyone have worked with the switching vendor Quanta for their 10ge switching as
TOR? [1] Their spec looked interesting and they are quiet cheap.

[1]
http://www.quantaqct.com/en/01_product/02_detail.php?mid=30&sid=114&id=116&qs=63

-bn
0216331C

17

Based on the specs, the Quanta switches look like they use Broadcom
merchant silicon and should have similar performance to other switches
based on the same chipset:

http://blog.sflow.com/2011/12/merchant-silicon.html

While many vendors use merchant silicon, there is variability in
firmware, exposed features, CLI etc.

18

I have fairly extensive experience with the Quanta LY2 10GE switches, and they work very well for some environments. Here are some basic impressions:

- Broadcom Trident chipset

- Similar performance to other Trident switches (ideally line rate, but small buffers)

- Cisco-like configuration interface (similar, not the same)

- Custom Linux kernel and OS

- Basic look-and-feel, but so far the quality has not been a disappointment

- Decent support for topologies with no Spanning-Tree

- Good compatibility with SFP+ transceivers, direct connections, and optics from various sources.

- Basic feature set (OSPF/RIP, but no BGP)

- Somewhat limited troubleshooting and debug tools

One very pleasant aspect of working with Quanta is that they are very responsive to feature requests, often working closely with customers. On the other hand, their release schedules are somewhat non-specific. I've been waiting for full MLAG support for a while (it's supposedly right around the corner).

They are particularly convenient if you are putting them at the top of racks full of Quanta servers, since they have logistics and full-rack staging/shipping.

I wish they had better MIB support, BGP, scriptability, and policy-based routing, but they don't. They are cheap enough, however, that you may be able to get two LY2 switches for the price of one of some of their competitors.