Gang,
I have submitted an internet-draft to the IDR group on virtual aggregation
(VA) (http://www.ietf.org/internet-drafts/draft-francis-idr-intra-va-00.txt).
This draft suggests a few changes to routers that allow operators to control
the size of their FIBs, shrinking them by 5x or 10x quite easily. This would
extend the lifetime of routers that are constrained by FIB size.
There has been a lively discussion of this on the IDR mailing list, including
a suggestion that FIB reduction is more important for lower tier ISPs (tier
2, tier 3...) than for tier 1 ISPs. Unfortunately I don't think that people
from smaller ISPs pay much attention to the IDR mailing list, so they are not
being represented in this discussion.
So I'm looking for input from folks who think that FIB reduction helps them,
so as to better understand their requirements.
Any help is much appreciated.
Thanks,
PF
Paul,
Can this proposal be applied to IS-IS (or other IGP) as well as BGP?
- Alain.
Certainly in principle it can, though that is not in the current proposal.
The basic idea is to suppress installing routes into the FIB when there is a
"virtual aggregate" that you can tunnel to instead.
I remember we discussed this in San Jose NANOG, but I forget the details.
Can you remind me?
PF
Paul Francis wrote:
Gang,
I have submitted an internet-draft to the IDR group on virtual aggregation
(VA) (http://www.ietf.org/internet-drafts/draft-francis-idr-intra-va-00.txt).
This draft suggests a few changes to routers that allow operators to control
the size of their FIBs, shrinking them by 5x or 10x quite easily. This would
extend the lifetime of routers that are constrained by FIB size.
There has been a lively discussion of this on the IDR mailing list, including
a suggestion that FIB reduction is more important for lower tier ISPs (tier
2, tier 3...) than for tier 1 ISPs. Unfortunately I don't think that people
from smaller ISPs pay much attention to the IDR mailing list, so they are not
being represented in this discussion.
Software switched routers have little pressure on fib limitions. For a certain class of application the software switched edge router is in a much better position to accommodate fib growth than a device with a fixed sized cam.
If you're in the business of shopping for sup-2s on ebay you'll probably see significant benefits in fib reduction.
I dunno about that; there's some papers floating about which look at
trie type FIB representations which note significant savings in compressing
FIB to unique entry set. Less memory, less comparsions, less nodes, etc.
Rather interesting stuff.
Try http://www.academypublisher.com/jnw/vol02/no03/jnw02031827.pdf for
fun.
Adrian
Adrian Chadd wrote:
Software switched routers have little pressure on fib limitions. For a certain class of application the software switched edge router is in a much better position to accommodate fib growth than a device with a fixed sized cam.
I dunno about that; there's some papers floating about which look at
trie type FIB representations which note significant savings in compressing
FIB to unique entry set. Less memory, less comparsions, less nodes, etc.
Rather interesting stuff.
Not saying that they couldn't benefit from it, however on one hand we have a device with a 36Mbit cam on the other, one with 2GB of ram, which one fills up first?
Well, the actual data point you should look at is "160k odd FIB from a couple
years ago can fit in under 2 megabytes of memory."
The random fetch time for dynamic RAM is pretty shocking compared to L2
cache access time, and you probably want to arrange your FIB to play well with
your cache.
Its nice that the higher end CPUs have megabytes and megabytes of L2 cache
but placing a high-end Xeon on each of your interface processors is probably
asking a lot. So there's still room for optimising for sensibly-specced
hardware.
Of course, -my- applied CPU-cache clue comes from the act of parsing HTTP requests/
replies, not from building FIBs. I'm just going off the papers I've read on the
subject. 
Adrian
Adrian Chadd wrote:
Not saying that they couldn't benefit from it, however on one hand we have a device with a 36Mbit cam on the other, one with 2GB of ram, which one fills up first?
Well, the actual data point you should look at is "160k odd FIB from a couple
years ago can fit in under 2 megabytes of memory."
The random fetch time for dynamic RAM is pretty shocking compared to L2
cache access time, and you probably want to arrange your FIB to play well with
your cache.
Its nice that the higher end CPUs have megabytes and megabytes of L2 cache
but placing a high-end Xeon on each of your interface processors is probably
asking a lot. So there's still room for optimising for sensibly-specced
hardware.
If you're putting it on a line card it's probably more like a RAZA XLR, more memory bandwith and less cpu relative to the say the intel arch approach.
That said I think you're headed to high end again. It has been routinetly posited that fib growth hurts the people on the edge more than in the center. I don't buy that for the reason outlined in my original response. If my pps requirements are moderate my software router can carry a fib of effectively arbitrary size at a lower cost than carrying the same fib in cam.
So, if I get you right, you are saying that edge routers have fewer CPU
requirements, and so ISPs can get away with software routers and don't care
about FIB. At the same time, folks in the middle are saying that in any
event they need to buy high-end routers, and so can afford to buy enough
hardware FIB so they also don't care (much) about FIB growth.
Are there any folks for whom this statement isn't working?
PF
Paul Francis wrote:
So, if I get you right, you are saying that edge routers have fewer CPU
requirements, and so ISPs can get away with software routers and don't care
about FIB.
"ISP's that can get away with software routers" Also multihomed edge networks, the costs associated with multihoming aren't evenly distributed. The entities most likely to get squeezed are in the middle of the echosystem.
At the same time, folks in the middle are saying that in any
event they need to buy high-end routers, and so can afford to buy enough
hardware FIB so they also don't care (much) about FIB growth.
They care, but you weren't buying 2 million entry cam routers a year ago to deal with the growth of the DFZ. They bought them because they needed or would need a million or more internal routes fairly shortly, which says a lot about the complexity of a large isp. Assuming the dfz growth continues to fit the curve it's pretty easy to figure out when you need enough fib to support 500k dfz entries just as it was when we did the fib bof at nanog 39...
http://www.cidr-report.org/cgi-bin/plota?file=%2Fvar%2Fdata%2Fbgp%2Fas2.0%2Fbgp-active.txt&descr=Active+BGP+entries+(FIB)&ylabel=Active+BGP+entries+(FIB)&range=Year&StartDate=&EndDate=&yrange=Auto&ymin=&ymax=&Width=1&Height=1&with=Step&color=auto&logscale=linear
That's not to say that fib compression is undesirable, that approach has real benefits extending the useful life of a given platform, but there's very little about the current situation that is unexpected, or intractable.
Joel Jaeggli <joelja@bogus.com> writes:
That said I think you're headed to high end again. It has been
routinetly posited that fib growth hurts the people on the edge more
than in the center. I don't buy that for the reason outlined in my
original response. If my pps requirements are moderate my software
router can carry a fib of effectively arbitrary size at a lower cost
than carrying the same fib in cam.
While that's true, planned obsolescence in terms of ram
non-expandability from certain <cough> vendors leaves people on the
edge (who often have to scrape by on thin budgets) hurting badly
enough as to offset any theoretical software-vs-hardware router
advantage they may have originally had.
-r
How's this for a summary of what you are saying:
Some ISPs can get away with software routers, most can't.
Of those that can't, planning ahead by vendors and ISPs has allowed most ISPs
to manage reasonably with well FIB growth.
Nevertheless, some protocol tools to help manage FIB growth would be welcome
by many.
PF