RE: What is the limit? (was RE: multi-homing fixes)

From: Sam Thomas []
Sent: Thursday, August 30, 2001 2:57 AM

> >> From: Randy Bush []
> >> Sent: Wednesday, August 29, 2001 9:18 PM
> >>
> >> > there would appear to be people who assert its not
> >> > unafordable to compute the routes for the current,
> >> > and the forseeable network.
> >>
> >> and there are those of us who think we can probably do
> >> so iff we stop the explosion of /24s.
> Okay, let's get it back out of the realm of opinion and
> into requirements.
> I'll even go down one step more, to /25. Take the entire
> IP addr space,
> break it ALL into /25's, times the BGP table entry size,
> how big is that?

> Also, I just got a price quote for 512MB ECC PC133 DIMMS
> at $49US, retail.

using Jonthan Disher's calculation of 155B/route (a check
of my local neighborhood gsr gave ~185B/route, but this is
all academic anyway so we'll use his number since it was
first and so folk playing along at home can have some
continuity), i get ~5GB for the whole address space as /25s.
(using 185B/route, it came to just over 8GB).

Thank you, it was the route-entry size that I didn't have defensible numbers
on. I knew roughly what they were, but was already out on the limb.<g>

i don't know of any off the shelf consumer hardware that
supports >2GB (i looked, really!), so the price to stick
that $800 ($16 is extremely reasonable shipping cost for
16 memory sticks, see if you can find it :wink: of ram on
something just went up an order of magnitude.

Okay, well I wasn't talking about COTS boards anyway. I was talking about
commodity tech, at the chip level. What us old microP silicon mechanics
called the "component" level. Circuit boards all cost the same to build and
design, regardless of the content of the design, with minor variations. It's
ALL automated these days. Feed a schematic in one end and, about a week
later, get a circuit board design out the other, with none-to-little human
intervention. Manufacturing is about the same. It is the components that
attach to them that contains the variable cost. It is those components whose
cost is dropping like lemmings off a cliff. You see, chip design and
manufacture is almost getting to the same place as circuit boards. The only
human-introduced variable is in the original design and architecture. That
cost is amortised over the volume of units sold. This is where commodity
tech has a huge cost/performance advantage.

> Considering what router vendors charge, with very fat
> margins, for their low-tech routers, affordable tech
> arguments don't hold water. Routers are expensive
> because someone is stupid enough to pay the prices.

i guess we're in the wrong business. we should go develop
discount routers using near-current off-the-shelf
technology. thinking this through, i'm already amazed at
the lack of glut for discount high-capacity routers.

You haven't been paying attention to the stock market then. Why do you think
router vendor stock prices are dropping like flies? Maybe because there *is*
a glut of cheap used equipment? This is forcing more sane prices on them
and, as a direct result, their earnings reports are down.

oops. some folk (like me) actually want the core router
to be able to handle a chassis chock-full-o-oc48s.
that'll take a bit of specialized hardware. oh, and
it would be really nice-ah if the router could forward
ip packets amongst those oc48s at something approaching
line-rate. that'll take an industrial strength backplane.
not finding too many multi-Gb/s backplanes on,
so guess that will have to be custom built, and we'll
probably have to develop some custom circuitry to handle the
custom backplane as well.

I'll bet that COGm, for a Cisco Catalyst 6509, with all GigE blades, is
under $5KUS. Further, that the major cost is the sheet-metal and paint work,
not the electronix. In the last three years, I emplaced 6 of them at ~$200K
each. Please tell me that isn't a very fat margin and yes, I do know about
keystone pricing and how the distro channel works. Even R&D isn't a major
cost, given enough volume. Manufacturing has been commoditized for over 20
years. Are you telling me that Cisco can't give us better tech, for less
money, than they've been doing thus far?

shoot, forgot software. we've run up the cost so much, we'll
just drop in a peecee with a linux kernel and zebra. we
won't be able to route packets through it, though, so i
guess we'll have to custom manufacture some other
device to handle the actual packing shuffling. or, we
could custom develop a single unit and some software that
runs on it. i'm sure there are plenty of
bgp/mpls/multicast/etc clued programmers willing to work
in our software sweatshop for minimum wage to keep costs low.

see, Richard Stallman and the FSF. I'll save my thoughts on open-source
"free" software for another day. Please know that MHSC is a Caldera VAR
(still dealing with Caldera assilimating the "enemy", SCO), and we can sell
you a copy of SCO OpenUNIX these days (but not cheap <sigh>). Yes, I've had
GateD running for some time now. Software cost is a part of R&D costs.

so, after all this, how big of a bargain is using that $800
in ram so far?

All the additional conditions should make ZERO difference to the COGs of the
product. Are you telling me that 8GB worth of table entries is a small

ok, it's still probably at least a few bucks cheaper than
vendor X. let's go sell it. hah! silly customers want a
number to call when it breaks. well, it uses cheap
off-the-shelf memory and processor, how much can it really
cost to support?

You know ... in the past 10 years, with many hundreds of servers, I have had
ZERO CPU failures and ZERO RAM failures, past the infant mortality period.
Even in workstations, all such failures were caused by brilliant actions,
like restricting the airflow (wrapping the darned thing in a blanket, once),
even then, the HDDs were the first to die. With zero moving parts, if you
can keep them cool, they run forever. They obsolece long before they die. I
have one old server, an amdK6-233, whose RAM (128MB of SIMMs) has been in
24x7 operation for over 6 years. It's also running a AHA2940 that has seen
24x7 operation since 1996.

since we used commodity hardware (at least ram and
processor), we won't have to stock spares. i'm sure
we can trust our customers to buy the right parts
off the shelf at "Bubba's Fine House of PeeCee Hardware".

You can kid around if you want to. But, you're only showing irrational bias.
An Intel part is an Intel part, regardless of whether Bubba or Einstein
sells it to you. Likewise with RAM. If the board don't take standard PC133
RAM then the design is defective. Bubba doesn't make the parts. He only
sells them. Unless, Bubba has a silicon foundry in his basement. In that
case, his name probably isn't Bubba.

now, would everyone kindly get off the $EXPENSIVE ram from
vendor X whine?

Those who discount, discounts, wind up broke, before those that don't.

> royally ripped off there. Cisco takes special RAM because
> Cisco designed incompatible RAM circuitry so they could
> charge you more for the Cisco label. It also bolsters
> the FUD-storm and the after-market support vendors.

or, maybe it's because they had the foresight to know that
if they used commodity ram, a bunch of cheapskate morons
would be plugging in the cheapest ram they could find at
the saturday morning swapmeet, and making dozens of
"warranty" support calls because the router crashes all the
time or half the packets come through looking like
swiss cheese.

Maybe the alternative was to hire competent circuit design engineers that
could design circuits that don't have those vulnerabilities? In my book, the
moron was the fool that didn't design with those cheapskates in mind.