Suggestion for Layer 3, all SFP+ switches

Hello,

we're looking for some L3 switches to be used as distribution devices. They should have all (at leaast 24) SFP+ ports 10G and at least a couple of upstream ports 40G capable, but what is most important, they should be able to run MPLS, EoMPLS and VPLS. Is there any device you would suggest us? We where thinking about NEXUS but I'm sure there are also others, even if it is not so easy to find them on the Internet.

Thank you in advance for your help .

Giuseppe Span�
Datacast Srl

Brocade/arris is eager for business these days. They have a nice switch (10g ports with 40g stacking) that should meet your needs with very aggressive pricing.

Eric

Does the Brocade/Foundry-lineage stuff that went to Arris actually do MPLS? I didn't think ICX did any MPLS.

The SLX (and MLX) line that went to Extreme does but is perhaps overkill (it will also do Internet-scale FIB). The SLX9540 is a 48 port SFP+ pizza box that also has 6 40/100Gb QSFP+/28 ports on it. You'd need the "advanced feature" license for MPLS, and I don't know how mature the MPLS code is. Pricing I've seen is pretty good for what you get, but again it may be overkill.

Juniper has some nice boxes in the EX series with at least MPLS L2-endpoint functionality that might also be an option for this sort of thing, but I don't know any models off the top of my head.

Ruckus ICX switches do not do MPLS. They meet all the other requirements
listed, but unfortunately MPLS was listed as the most important one.

Juniper ACX 5048 is what we use though you need to license 10g ports (ACX5K-L-1X10GE) and VPN (ACX5K-L-IPVPN)

QFX does MPLS but I'm pretty sure it doesn't do VPLs.

ns

look at these...

* Juniper ACX5048 - I've deployed about ~50 of these over the last couple
years and they are great boxes. I'm using them as mpls p/pe running L3VPN
(v4 and tested 6vpe), L2VPN (manual martini l2circuits and bgp-ad rfc4762,
I'll say that IOS XR asr9k has an occasional problem with vpls pw towards
acx5048 rfc4762, overcome with bounce of ldp neighbor only when needed, not
sure who to blame)
(48) 10 gig sfp+
(6) 40 gig qsfp+ - or these can run as (24) 10 gig ports using a break-out
cable

* Cisco NCS5K and its variants as I think there are a few....you might find
a 10/40 gig option here, however I recall the one I tested with in my lab a
few years ago had 10/100 gig. I will say that when I tested it a few year
back that I wasn't ready for prime time, but, in cisco defense, they've had
a few years to make improvements on it and I should, and you should, look
into it.

* Juniper ACX5400 (ACX5448) - I'm seeing this advertised on juniper.net
now...new box - I want one for my lab
- (48) 10 gig
- (4) 100 gig, check if you can slide a 40 gig optic into that qsfp slot

* Juniper ACX5k+ - unsure if it's advertised yet by Juniper... this is a new
box - I want one for my lab
- lots of 10 gig and I recall some 25 or 40 or 100, I don't recall
https://www.mail-archive.com/nanog@nanog.org/msg93672.html

or google - 1/2u 100g Metro-E Aggregation Switch

* Juniper EX4550 - I've ran these are virtual chassis paired top-of-rack in
my small data centers with rock solid performance - with multiple cdn caches
sitting behind them. Now, I have tested L3VPN some years back, and I've
heard they also do L2VPN martini manual pw's...i'm about to give them
another go in mpls testing... check back with my shortly if you wanna know
how it goes. Yesterday I pushed in my first EX4550-EM-2QSFP 40 gig module
into my lab EX4550 in preparation for new supercore (100 gig mx960's) and
had to upgrade junos from v12 to v13 (wasn't avail so went with
14.1X53-D46.7) and now optic was visible in that 40 gig module in the ex4550
- (32) 10 gig ports
- (2) 40 gig

- Aaron

What is your budget?

I know on the low end many operators are using the
Huawei S6720S-26Q-EI-24S-AC. You can get these new for $2500 to $3500, and
the support all the features and port counts you requested. The also have a
lifetime warranty that includes advanced replacement (10 days), TAC
support, and software support all for free if you buy through official
channels. It support MPLS, and also VXLAN.

Extreme seems to have some good options, but I doubt they are that low cost.

For Juniper you need to look at the ACX series which is expensive. Like the
ACX5048 which list price is $40k not that anyone pays list, and that's
before port licenses. The EX series does not have proper MPLS support.

Cisco has mutliple options, but mainly the NCS based on your port count I
think. Supposely the C3850 and C9500 now support MPLS? There is a new 16
port 10G version of the C9500. I haven't looked into Nexus switches. Does
Nexus support full MPLS?

HPE has some low cost options. In their FlexFabric and FlexNetwork lines
that support MPLS in Comware V7.

Who else are we missing? MPLS support really cuts down this list, but I
agree its a critical feature for most service providers.

I've been testing IPInfusion OcNOS running on Dell Z9100 and S4048. I've
run a couple of test cases using MPLS LDP signaled port based and VLAN
based VPWS (pseudowire / e-line / xconnect / Juniper CCC) and VPLS (e-lan)
over an OSPFv2 IGP. It's working well between Dell/IPI to Dell/IPI boxes.
We have had issues with the VPLS between Dell/IPI to Juniper/JunOS where
the circuit will show up on the Juniper and down on the Dell. If we clear
LDP session on the Dell, it comes back up right away. This seems to be
similar to what Aaron is seeing in his multi-vendor environment. The price
on the Dell hardware is really good. The features included with OcNOS are
much better than FTOS9. If you aren't partial to Dell, you can run OcNOS
on a variety of other whitebox switches, like EdgeCore.

I haven't tested MP-BGP and L3VPN or BFD yet, but that is supposedly
supported in OcNOS as well.

-ben

Aren't there issues/concerns with Huawei ?

I think we pay about $10k with discounts and about (4) 10 gig port license to slow start our deployment of ACX5048's.... 10 gig east , 10 gig west , dual 10's facing FTTH OLT (Calix E7)

-Aaron

Thank you very much to everyone.

The budget is around 3000-5000 $ each, possibly.
There are many devices that could match our needs but as usual the dark side of this market is the platforms compatibility. We deployed many Mikrotik and Ericsson devices, hope they will "match" with a Cisco or Juniper or Huawey device with regards to MPLS, EoMPLS, VPLS etc...

Anyway your kind help is really very appreciated, we'll decide for one and will test it, no way out I think.

Giuseppe

Colton,

Cisco has mutliple options, but mainly the NCS based on your port count I
think. Supposely the C3850 and C9500 now support MPLS? There is a new 16
port 10G version of the C9500. I haven't looked into Nexus switches. Does
Nexus support full MPLS?

UADP based platforms, both older (C3650/3850) and newer (C9xxx) do
support MPLS encap and VXLAN encap and can be extended in future to
support others. There are new 9xxx based off UADP 3.0 with 40G and 100G
ports:

Nexus 7k supports MPLS with LDP while Nexus 9k supports MPLS but
with SR (IGP) or BGP-LU (no LDP support).

Łukasz,

Out of all those Cisco models, which meets the OP requirements of " (at
least 24) SFP+ ports 10G and at least a couple of upstream ports 40G
capable" and a " The budget is around 3000-5000 $ each, possibly. "?

The Nexus 7000's look very large with the smallest being 3U in size, so I
doubt they would meet the budget requirement. The Nexus 9000 series seems
to have 1U versions. Assuming he is fine with using Segmented Routing
instead of LDP, any models that fit the bill price wise? Are there any
Nexus products that are lower cost that the Catalyst (C3650/3850) and
newer (C9xxx)?

The Catalyst UADP based platforms seem nice, but most are requiring DNA
licensing driving up initial cost.

Yes, there are issues/concerns with using Huawei in the USA, but in the
rest of the world they are the number 2 vendor. Also, $3500 for that box
with lifetime support and warranty (their TAC is in Plano, Texas) vs
$10,000 for an ACX5048 onetime plus at least $1500 a year for JTAC seems
like a big difference! ACX has 48 ports vs 24 in the Huawei, but you have
to licenses each one of the ports on the ACX making the total cost even
higher. Sounds like your ACX cost more than you E7 that its feeding!

Ben,

The Dell options intrigue me. First question is who do you talk to at Dell
about their solutions as most sales guys just seem to know their laptop and
server lines?
How does Dell's pricing compare with Edge-Core. Considering most of the
hardware is the same Broadcom chipset, what are the reasons you went with
Dell over someone like Edge-Core or the other OEM's?
I have looked into IPInfusion OcNOS and feature wise it looks nice, but
by the time you pay IPInfusion OcNOS for the software and an OEM for the
hardware the costs adds up to a Cisco/Juniper equalivent model.

Dell seems interesting as I think they include an OS for free, but then you
can load IPInfusion OcNOS or Cumulus or others onto it if that doesn't
meet your need. You mentioned FTOS9. Doesn't Dell now have OS10 version?

The ZTE 5960 with 48x SFP+ and 4x QSFP28 (40G and 100G capable) will do it
within the budget listed. We use it for MPLS and VPLS.

Regards
Baldur

Thanks Colton, Since I live in the US, and work for a boss that’s nervous (concerned) about those things, then I comply. I remember mentioning Huawei as an option recently in a meeting and the boss and a few other fellow engineers were nervous and resistant to it. I tend to feel the same.

I see you started a thread on comparing those 2 (zte and Huawei) … and was immediately met with cautionary/warning statements about these some things... from Suresh and Curtis.

So I wonder if because of all this, are ZTE and Huawei sales being adversely affected in the US? …it would seem so, but thought I’d ask y’all.

Google - China Showdown Huawei vs ZTE

http://seclists.org/nanog/2018/Apr/293

- Aaron

Well, if the US government spies on everyone using exported cisco hardware,
why wouldn't the PRC do the same?