Getting pretty close to default IPv4 route maximum for 6500/7600 routers.

Hi all,

I am wondering if maybe we should make some kind of concerted effort to remind folks about the IPv4 routing table inching closer and closer to the 512K route mark.

We are at about 94/95% right now of 512K.

For most of us, the 512K route mark is arbitrary but for a lot of folks who may still be running 6500/7600 or other routers which are by default configured to crash and burn after 512K routes; it may be a valuable public service.

Even if you don't have this scenario in your network today; chances are you connect to someone who connects to someone who connects to someone (etc...) that does.

In case anyone wants to check on a 6500, you can run: show platform hardware capacity pfc and then look under L3 Forwarding Resources.

Just something to think about before it becomes a story the community talks about for the next decade.

-Drew

In case anyone wants to check on a 6500, you can run: show platform
hardware capacity pfc and then look under L3 Forwarding Resources.

to fix the problem on sup720/rsp720:

Router(config)#mls cef maximum-routes ip 768

This requires a reload to take effect. If you don't recarve the TCAM and
you accidentally hit the maximum number of prefixes, the entire chassis
will go into software forwarding mode and will require a reboot to recover.
IOW, there is no way of avoiding a reload, so best plan as soon as
possible. More info here:

http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-switches/116132-problem-catalyst6500-00.html

This problem also affects ASR9000 boxes running typhoon line cards which by
default will only set aside 500k slots for ipv4 prefixes. The fix for
these boxes is:

0/RSP0/CPU0:router# admin hw-module profile scale l3

...followed by appropriate line card reloads.

more information at:

http://www.cisco.com/c/en/us/support/docs/routers/asr-9000-series-aggregation-services-routers/116999-problem-line-card-00.html

Nick

I've been configuring 6500/Sup7203bxl's with
mls cef maximum-routes ip 768

The only gotcha is, you have to reload for that to be effective.

Speaking of which, I've had WS-X6708-10GE cards "go bad on reload" in a couple of 6500s.

I see cisco finally released some more info on their "bad memory" announcement from several months back:

http://www.cisco.com/c/en/us/support/docs/field-notices/637/fn63743.html

It seems our "gone bad" 6708s may be included in this issue. If you don't have enough spare ports or spare cards, this puts you in a somewhat precarious situation. You need to reload to affect the v4/v6 route storage change, but you might lose some blades in the process.

Yes, a Sup720/PFC3CXL defaults to 512K IPv4 routes, and reconfiguring
the FIB requires a reload. So I've been quietly expecting a somewhat
serious meltdown when we hit 512K :slight_smile:

Jeff

It would probably be a good time to upgrade the memory on my 7206 NPE-G1 as well (512MB). I was going to replace the router but am going to keep it around for the Fall Semester. Anyone know of any good 3rd party memory modules that are equivalent to the MEM-NPE-G1-1GB? I got a quote for the official Cisco ones last summer and it was around $5,000 lol

This problem also affects ASR9000 boxes running typhoon line cards which by default will only set aside 500k slots for ipv4 prefixes. The fix for these boxes is:

I believe you mean "This problem also affects ASR9000 boxes running ...trident... line cards". Please confirm?

-Drew

er, yes, trident cards, not typhoon cards.

typhoon cards are not affected by this.

Nick

I would like to see Cisco send something out...

There is currently a doc for the ASR9k. We're working on getting on for
6500 as well.

http://www.cisco.com/c/en/us/support/docs/routers/asr-9000-series-aggregation-services-routers/116999-problem-line-card-00.html

I just recently got four sets off eBay. Purportedly genuine Cisco. A
shade over $100. Raid the departmental beer fund. :slight_smile:

-r

Vlade Ristevski <vristevs@ramapo.edu> writes:

Thanks for this e-mail with clear subject :wink:
Did anyone yet calculated roughly when the ipv4 routing table will hit 512K ?

Closer to? Internap announces 507K prefixes to me today. Coupled with the
prefixes I carry in iBGP internally, I've been sitting at 511K for quite
some time, and at occasions, exceeded 512K in the last 2 weeks.

L3 Forwarding Resources
             FIB TCAM usage: Total Used %Used
                  72 bits (IPv4, MPLS, EoM) 802816 511848 64%
IP routing table maximum-paths is 32
Route Source Networks Subnets Overhead Memory (bytes)
connected 0 31 3012 4464
static 1 78 17456 11376
ospf 1 1 310 22392 44784
  Intra-area: 110 Inter-area: 160 External-1: 0 External-2: 41
  NSSA External-1: 0 NSSA External-2: 0
bgp 23456 167707 343507 36808128 75466680
  External: 507479 Internal: 3735 Local: 0
internal 6054 13246152
Total 173763 343926 36850988 88773456

The doc on how to adjust the 6500/7600 TCAM space was just published.

http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-switches/117712-problemsolution-cat6500-00.html

Just had to do this on my router last week. Came in a few mornings ago and
we were software switching, yay!

Why, in your example, do you bias the split so heavily toward IPv4 that the router won't be able to handle a current full v6 table? I've been using

mls cef maximum-routes ip 768

which is probably still a little too liberal for IPv6

FIB TCAM maximum routes :

The IPv6 table will not be as big as the v4 table even after full
acceptance. Given that most providers will be advertising a single /32 and
then rest will be some /48 routes for multi-homed scenarios.

My router looks like this

FIB TCAM maximum routes :

It is generally much better to do the following:

mls cef maximum-routes ipv6 90
mls cef maximum-routes ip-multicast 1

This will leave v4 and mpls in one big pool, puts v6 to something useful for quite a while and steals all of the multicast space which is not really used on most deployments.

This gives us the following (which is pretty great for IP backbone purposes in dual stack):

#show mls cef maximum-routes
FIB TCAM maximum routes :

John, great point!

Regardless, shouldn't need more than 626K to make it to v6 and we wont need
as many for v6. That was one of the problems that v6 was designed to
address.

Yep, exactly… the problem is the carving suggested by most kills the fact that MPLS and v4 are pooled, which on a larger network is very nice, especially if using 6PE where each v6 route may need an MPLS route too.

I botched those numbers.

Let me fix.

According to this countdown:
http://inetcore.com/project/ipv4ec/index_en.html we have 6.41 /8's left. So
that is 107,541,955 IPs.

CIDR - Prefixes