I have an opportuniy to put two 7609s into the core of my network.
Currently we have 3 upstream providers, taking full BGP routes. (2 in one
router and one in another). We have 17 BGP peers/customers (peering to each
router), and adding about one new BGP peer every 2-3 months. It is a modest
network by most standards. We are running OSPF and BGP between the existing
routers.
Not rocket science, nothing special (no MPLS, no VRF etc), very simple
network.
Does anyone have any recommendations on the 7600's as a core BGP router?
Good or bad? Have they been a stable platform in a core/BGP environment?
We run the 6509-e platform in this role with Sup720-3bxl's.. and they have
been rock solid.
Peter Kranz
Founder/CEO - Unwired Ltd
www.UnwiredLtd.com
Desk: 510-868-1614 x100
Mobile: 510-207-0000
pkranz@unwiredltd.com
We use the 7600 platform as a Customer Border device. It attaches
directly to our core, and directly to our customers. This has been a
solid platform. Before this we used to use the 7600 as a load balancer
for a DNS cluster. Worked fairly well. We use the 6500 series for our
main network infrastructure and to border/core/dist layers and they are
rock solid, as long as you stay away from the SXH images. These are a
bit buggy and we have had routers crash due to that image. We have
deployed a few new devices with the SXI and are very happy with them
currently.
Jim Wininger wrote:
We don't run very much Cisco gear (none of their larger, hardware stuff) but I have a couple questions for the Cisco gurus out there...
According to this page:
http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/product_data_sheet09186a0080159856_ps4835_Products_Data_Sheet.html
The Cisco WS-SUP720-3BXL can hold "1,000,000 (IPv4); 500,000 (IPv6)" route entries.
1) Does that mean a) The card can hold 1m IPv4 routes --OR-- 500K IPv6 routes or b) 1m IPv4 routes --AND-- 500K IPv6 routes?
2) I'm assuming MPLS cuts into the number somewhere but could anyone explain it briefly?
3) Do ACLs use some of these resources or do they get their own slice of memory?
That page also reports "up to 40 Gbps per slot of switching capacity; 720 Gbps aggregate bandwidth".
Is the 40Gbps per slot an aggregate or full-duplex value?
Thanks for helping out a Cisco n00b!
We don't run very much Cisco gear (none of their larger, hardware stuff) but I have a couple questions for the Cisco gurus out there...
According to this page:
Cisco Catalyst 6500/Cisco 7600 Series Supervisor Engine 720 Data Sheet - Cisco
The Cisco WS-SUP720-3BXL can hold "1,000,000 (IPv4); 500,000 (IPv6)" route entries.
1) Does that mean a) The card can hold 1m IPv4 routes --OR-- 500K IPv6 routes or b) 1m IPv4 routes --AND-- 500K IPv6 routes?
OR. Or you can have 524288 v4 and 262144 v6 routes...or you can move the split around. I chose:
L3 Forwarding Resources
FIB TCAM usage: Total Used %Used
72 bits (IPv4, MPLS, EoM) 622592 289791 47%
144 bits (IP mcast, IPv6) 212992 8 1%
Adjusting the split requires a reboot.
2) I'm assuming MPLS cuts into the number somewhere but could anyone explain it briefly?
I think the above actually does.
3) Do ACLs use some of these resources or do they get their own slice of memory?
Don't think so.
I did a blog entry about this a while back.
http://jonsblog.lewis.org/2008/02/09#sup720-20080209
That page also reports "up to 40 Gbps per slot of switching capacity; 720 Gbps aggregate bandwidth".
Is the 40Gbps per slot an aggregate or full-duplex value?
AFAIK, cisco always reports these things as input+output = bandwidth. It makes the numbers more impressive.
We've been using 6509s as BGP routers for years and they've generaly been rock stable.
The total size of the TCAM for forwardinging lookups is 9MB. IPv4 and
MPLS routes consume 72 bits per entry, IPv6 and multicast routes consume
144 bits per entry. If you use all IPv4/MPLS, you get 1048576 entries.
If you use all IPv6/multicast routes, you get 524288 entries. Your
actual usage will probably be somewhere in between. ACLs and Netflow are
different TCAM resources.
Also, you have to pre-"partition" the TCAM capacity and reboot in order
for the changes to take effect, so you'll need to decide your profile
beforehand. The command to do that is "mls cef maximum-routes". You can
also check your current utilization of forwarding tcam and acl/qos tcam
with "show platform hardware capacity" on anything SXF or newer.
Woops, I missed this question. On CEF720 (aka the cards numbered 67xx
that claim to be 40G/slot) what you actually have are two 20G fabric
channels to each slot. The fabric channels are full duplex, so you could
have 40G per slot coming in and out of the same card, and Cisco
double-counts the packets (in and out) to get 9 fabric slots * 40G * 2 =
720.
On 6704 cards, ports 1 and 2 are in one fabric channel, ports 3 and 4
are in another. On 6748 cards, even ports are in one channel and odd
ports are in another. On 6724 cards, there is only one 20G fabric
channel. Of course there are literally a list of caveats a mile long to
explain why you won't actually get anywhere near 720G out of the box,
but the big one you want to be careful of is sending traffic within the
same fabric channel (i.e. from port 1 to port 2 on a 6704). This will
cause a bottleneck at around 7-8G, with lots of input or output drops.
You can also view the fabric utilization with "show platform hardware
capacity fabric", though you should keep in mind that these counters are
about as accurate as the others on this platform (wild variations at
best).
The 7600 is actually quite a poor choice as an edge device (any edge) due to its caveats regarding NetFlow, ACLs, and uRPF. It's far better suited to a core role, where it can handle mpps running without the need for these critical edge features.
Funny, I'd argue that they're a terrible choice for a core router, due
to their inability to do line rate on a "any port to any port" traffic
profile, poor MPLS-TE functionality, and all of the caveats regarding
port-channel hashing. I do agree that they're also a poor choice for a
transit/peering edge due to their netflow issues (aka "completely
worthless, don't even bother trying"), ACLs, and route-map suckage in
general, but IMHO the only place they are even remotely usable is a
customer aggregation device.
With a customer agg router you have a lot of control about how you map
the ports <-> fabric channels to avoid intra-channel traffic, on a core
device you have no such luxury and you really don't want your network
taking a crap when your longhaul or even metro traffic shifts around (as
is going to happen on any well connected network). Once you throw in the
need to do MPLS and inter-device traffic rates greater than 10G, they're
an epic disaster in this role. On the other hand, you may not need
netflow on the customer edge if you're doing it on your peering edge, if
you structure your network right you can almost completely avoid having
to do ACLs on them, and the uRPF functionality is probably the least
broken thing about them. You also don't need complex routing policies,
you can hang them off more competent routers as route-reflectors, and
heck a datacenter agg box is probably the only place you want to be
using xenpaks (or even worse, x2) anyways.
But as always, your network requirements may vary. The only real
argument I can come up with against using them as customer aggregation
boxes is that when their interface counters break (which only happens on
days that end in y) you're actually misbilling people, and maybe not in
the direction you'd prefer. 
Personally I'd avoid this platform given 6+ years of trying to make it work
reliably. GSR is far better platform.
I previously ran a single 7609 with dual Sup720's as a Core Internet BGP
Router, running OSPF & iBGP
Never had any problems and was a very stable platform
Stephen Bailey - Senior Lead Systems Engineer
FUJITSU
Fujitsu Services Limited, Registered in England no 96056, Registered
Office 22 Baker Street, London, W1U 3BW
This e-mail is only for the use of its intended recipient. Its contents
are subject to a duty of confidence and may be privileged. Fujitsu
Services does not guarantee that this e-mail has not been intercepted
and amended or that it is virus-free.
Agreed... we migrated away from GSR to 7600 and now looking at migrating
back...
GSR was 100% rock solid for us with PRP-2 processors....
sup720-3bxl has been good but no comparison...
It's hard to classify a single router as a "core", don't you think?
Because of nowadays network scalability demands, Cisco is preparing ASR
14000 series to replace this one, I think. ^^
Basically ASR 14000 is downgrade version of CRS-1, but I consider it is
still developing or beta product.
Alex
Paul Stewart wrote:
Core typically references functionality, not the number of network devices at that layer.
tv
It's always a good idea to check the status/availability of a given platform prior to making any plans which depend upon it.
Because of nowadays network scalability demands, Cisco is preparing ASR
14000 series to replace this one, I think. ^^
Basically ASR 14000 is downgrade version of CRS-1, but I consider it is
still developing or beta product.
As far as I know Cisco cancelled ASR14000 platform, but the developed supervisor will be available to CRS-1 platform....
Janos Mohacsi
Network Engineer, Research Associate, Head of Network Planning and Projects
NIIF/HUNGARNET, HUNGARY
Key 70EF9882: DEC2 C685 1ED4 C95A 145F 4300 6F64 7B00 70EF 9882
About 3 months ago, Cisco Account Team was recommending AS14000 for our
company, and we rejected it.
Poor product development management!
Alex
Mohacsi Janos wrote:
