Upgrade Path Options from 6500 SUP720-3BXL for Edge Routing

Mailman List Mirror (Read Only)
1

I’m curious what other providers have gone with when moving away from SUP720-3BXL 6500 platforms. I’m platform agnostic and just as comfortable with Juniper as with Cisco.

It’s a conversation were having since the 3BXL’s are running into limits with the large number of prefixes, long eBGP convergence times, and 10G port density.

Basically were looking to carry multiple full routing tables from several 4+ carriers plus internet exchange traffic so the ability to handle 1-2M IPV4 and 500K+ IPv6 and decent 10G port density and/or 40G options as well. Also should have decent CPU capabilities so it can crunch routes in a reasonable amount of time.

Right now my thinking are MX480 or ASR9k platforms. Opinions on those are equally welcome as alternatives, but I’d love to hear from those with personal experiences today vs sales people trying to tell me it would route the world :slight_smile:

Thanks,

Corey T

2

Or, protect your existing investment in 6500 and replace the SUP720 with the
SUP2T. You can then deploy the WS-X6904-40G-XL blades which give you 4 * 40G
or 16 * 10G on a 80G backplane (i.e. 2:1 oversubscription). I'm in the process
of going through this upgrade at the moment and I'm happy with what I'm seeing.

A lot depends on the total traffic throughput you're looking to switch/route.

You can then look to migrate onto the 6880 chassis which gives you a faster
backplane, whilst retaining compatibility with existing linecards.

Simon

3

We gave up and went to ASR9ks but that that was also a pretty big budget upgrade...

4

The 6880 has a single fixed Sup 2T. I believe you meant the
6807. That's what we are looking at to replace our 6500s. There
currently are no high density 10G cards out for the 6807 but our
account rep tells us a 32 port 1/10G SFP+ line card is coming
out in December. If it weren't for this new line card, we would
probably lean toward a different solution.

5

Yep, MX480/960 and ASR9006/9010 are the way to go if you're
looking at decent (Intel-based) CPU's, good performance and
good 10Gbps/100Gbps port density, incuding combinations
thereof.

40Gbps might be a little tricky on these boxes; for that,
looking at Ethernet switches (Nexus, C6880, Juniper EX) are
better options. We don't mess around with 40Gbps - it's
10Gbps or 100Gbps :-).

IOS XR on the CRS and ASR9000 is based on QNX, which suffers
from being only a 32-bit kernel. So even if the hardware
will ship with >4GB of RAM, the OS will only see 4GB (I have
12GB in my CRS's and 8GB on my ASR9001's).

IOS XR on the NCS runs on Linux, which removes the memory
limitation, but it's not clear whether that philosophy will
make it down to earlier IOS XR platforms (CRS, ASR9000).

Whatever the case, I've been following Blackberry for a
while on this, and it doesn't seem like they have any plans
to release a 64-bit version of QNX. AFAIK, their phones are
all 32-bit, so...

Junos has no issue seeing 32GB of RAM (their currently
highest RAM on their RE's), as it's a properly 64-bit OS.
That said, some of the applications that run within Junos
(notably "rpd") are still playing catch-up in terms of how
much memory it can "see", and how well it can use the
multiple cores present on the RE's. A lot of work is going
on in this area, and generally, the later the Junos code you
run, the more enhancements to the software you will see (and
the accompanying bugs, hehe).

I've been testing Junos 14.1R1 in production on a couple of
MX80's and MX480's for some weeks now. No issues to report
(yet).

Mark.

6

❦ 30 juillet 2014 09:53 +0200, Mark Tinka <mark.tinka@seacom.mu> :

IOS XR on the CRS and ASR9000 is based on QNX, which suffers
from being only a 32-bit kernel. So even if the hardware
will ship with >4GB of RAM, the OS will only see 4GB (I have
12GB in my CRS's and 8GB on my ASR9001's).

What's the point of shipping more memory then? Maybe the OS can only
address 4GB per process but is able to use up to 64GB in total (PAE)?

7

That was one argument from Cisco - that when the software
catches up, they might be able to compartmentalize so that
applications gain access to it individually. I didn't grill
them too much on this, as we use IOS XR in the core mostly
(CRS), and we don't need RAM too much since IPv4 is switched
on MPLS labels, negating the need to hold a full IPv4 table
on the routers.

That said, I can see a use-case where the additional RAM on
the CRS and ASR9000 can make sense if IOS XR is allowed to
run separate VM's on the same control plane. I know that iso
one of the ideas behind the NCS, but not sure whether it
will be added to the CRS and ASR9000.

Mark.

8

Right now my thinking are MX480 or ASR9k platforms. Opinions on those are

Or, protect your existing investment in 6500 and replace the SUP720 with the
SUP2T. You can then deploy the WS-X6904-40G-XL blades which give you 4 * 40G

I would generally suggest you look at it as a long term decision, at
least before jumping to the next incremental (modest increase) on the
upgrade treadmill. It depends on whether the 6500 is still a perfect
match for your network other than the prefix limit. Your vendor
should think of your equipment as an "investment" to be protected,
  by exploiting your feelings of loss aversion, but the upgrade
treadmill is a trap..... next thing you know, you will have to
replace the chassis, then you will need new linecards......

Keep in mind most of the MX series makes the 6500 look like a 5 port
linksys home router, when it comes to carrying around and managing
large BGP tables; both in terms of prefix capacity, speed, the
policy/filtering/configuration management functionality of the OS,
and how they will take the route update "beating" during setup of
new multiple BGP sessions...

The SUP2T is about a 100% increase in TCAM size, but still
pretty limited in terms of system resources.

You can also "protect" your investment if appropriate by taking this
late 1990s gear off your BGP edge, or otherwise recruiting it for a
role which it is more suited for in this day and age, where it is
not handling full tables and thus the feeble amount of FIB size, CPU,
memory are no potential hinderance now or on the next 10 years.

The ability to link up 40G ports did not seem terribly useful when
it would all be unsafely oversubscribed.

9

Next up the road are the 6800's.

Essentially SUP-2T's, so you get software parity Day One,
but still the same supervisor module.

We are running 6880's (which are the fixed SUP-2T's, but
with modular line cards), but only a core switching (Layer 2
Ethernet) role. Great port density since the 10Gbps ports
are now SFP+, but oversubscribed line cards 2:1, since each
slot is 80Gbps, but the line card comes with 16x 10Gbps
ports. You can disable oversubscription and go into
performance mode, which disables half the ports on the line
card - we do that.

IP-/MPLS-wise, whatever you can do on the 6500 you can do on
the 6800, but I can't say for sure as we're running them as
switches.

That said, if your goal is IP, just consider the ASR9000,
MX, and whatever else other vendors can do in this space.

Mark.

10

These seem cute anecdotes but I'm not sure how appropriate they are.

CAT6880 is XEON control-plane, and if we compare MX80 and RSP720, where RSP720
has slightly lower performance CPU, RSP720 out-performs MX80 (and MX104) in
BGP convergence and BGP scale.
Certainly if you compare SUP720 to XEON MX960, your anecdote is accurate.

JunOS is architecturally quite similar to IOS-XE, single fat process (iosd,
rpd) doing all the relevant work, running on modern control-plane (linux,
freebsd). One advantage to iosd is, that it's actually multithreaded unlike
rpd.

Obviously Sup2T/6880 2M FIB is limited, but what is JNPR MX scale? Trio has
256MB RLDRAM for everything, looking at my MX IPv4 FIB memory consumption
divided by entry size, it pegs IPv4 entry to 77B (seems massive), which would
translate to 3.5M IPv4 FIB upper bound, if nothing else is there.
Realistically, I don't think JNPR promises anywhere near this. So the FIB
scale may be pretty similar in both.

So I don't think FIB, control-plane or software are selling-points here. Where
MX shines, is deep services, with CAT you have relatively dumb ASIC, while MX
is capable for very deep services with its NPU.

If you can reuse existing LC and skill investment while living with limited
forwarding-plane functionality offered, it seems entirely sensible solution,
and in no way more '90s technology' than MX.

If you need deep services, of course it's wrong box, then MX or ASR9k is what
you should be looking at.