Linux Router KIT

So does linux.

Linux can be:

A packet forwarder (router)
A packet filter (firewall)
An IP masquerading packet filter (NAT firewall)
Can run RIP, BGP, EGP, OSPF (via gated)

Maybe I'm being naiive here, but what does Cisco offer beyond this (besides
the availability of higher performance)?

--Adam

Lots of types interfaces? EIGRP? Things that Linux/PC does have is Hard
drives that fail, etc; I guess you could do a flash thing, though. Other
protocols, like IPX, Dec, ATalk, etc are not going to happen.

Also, for a long time, Linux had a hard time with lots or routes.

If you are going to use a PC as a router, use an Unix that has real net
code, like xBSD.

So does linux.

Linux can be:

A packet forwarder (router)
A packet filter (firewall)
An IP masquerading packet filter (NAT firewall)
Can run RIP, BGP, EGP, OSPF (via gated)

Maybe I'm being naiive here, but what does Cisco offer beyond this (besides
the availability of higher performance)?

> Linux doesn't just kill Microsoft's NT and Solaris. It also eats
> Cisco for lunch.

This isn't true. IOS does a lot more than just get packets from
interface A to interface B. (In terms of managability as well as
functionality)

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   ISPF, The Forum for ISPs by ISPs. October 26-28, 1998, Atlanta, GA.
    Three days of clues, news, and views from the industry's best and
    brightest. http://www.ispf.com/ for information and registration.

     Atheism is a non-prophet organization. I route, therefore I am.
       Alex Rubenstein, alex@nac.net, KC2BUO, ISP/C Charter Member
               Father of the Network and Head Bottle-Washer
     Net Access Corporation, 9 Mt. Pleasant Tpk., Denville, NJ 07834
Don't choose a spineless ISP; we have more backbone! http://www.nac.net
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --

So does linux.

Linux can be:

A packet forwarder (router)
A packet filter (firewall)
An IP masquerading packet filter (NAT firewall)
Can run RIP, BGP, EGP, OSPF (via gated)

Maybe I'm being naiive here, but what does Cisco offer beyond this (besides
the availability of higher performance)?

Appliance level reliability, like a toaster. Plug it in, turn it on,
configure it once, forget it exists. That's why we moved our printers to a
dedicated print-server (OSIram), from the Linux hosts.

Actually, a cisco may actually double as a toaster, a foot-warmer at the
least <grin>.

BTW, I thank the list for the kind help offered with our Linux NIC routing
problem. It's still not solved, but we've been otherwise $distracted$. I
plan on implementing some of the suggested solutions this week. Again, the
issue is fall-back if our switch goes out again. For this sort of usage,
Linux routing is far cheaper than keeping a cisco laying around.

If the switch goes off, the lights go out, and the party's over. Linux
routing would keep minimal lights on so the party could continue.

Lots of types interfaces? EIGRP? Things that Linux/PC does have is Hard
drives that fail, etc; I guess you could do a flash thing, though. Other
protocols, like IPX, Dec, ATalk, etc are not going to happen.

Eh?

IPX : http://sunsite.unc.edu/mdw/HOWTO/IPX-HOWTO.html
Decnet : http://linux.dreamtime.org/decnet/
Atalk : anders.com

Also, for a long time, Linux had a hard time with lots or routes.

No longer applies. In fact Linux is now faster than BSD up to about
60-70,000 routes. BSD is faster at about 200,000. In between its about
even.

If you are going to use a PC as a router, use an Unix that has real net
code, like xBSD.

Please, leave religion out of the discussion...

-Dan

Lots of types interfaces? EIGRP? Things that Linux/PC does have is Hard

> drives that fail, etc; I guess you could do a flash thing, though. Other
> protocols, like IPX, Dec, ATalk, etc are not going to happen.

Will you please point out other router than cisco which has EIGRP ?
What kind of interface do you want ? You have async (multiport
async), sync, ethernet, fddi and now atm is coming. BTW, do you get
arcnet with cisco ? :slight_smile:

IPX and appletalk have been there for a _long_ time. There is also a
Linux DECNET project.

> Also, for a long time, Linux had a hard time with lots or routes.

Define lots. You want full BGP table in a PC router ? Why :slight_smile: ?

Without any claims that it's a lot, our machine has 450+ routes, does
BGP and OSPF with 3 NICs, constantly handles over 1Mbps data and it is
a 486/66 with 16M RAM. A 2500 died when it was flooded with small
packets, linux survived. I'm sure the box would still be ok when the
load will double (probably won't see the day, though :slight_smile:

> If you are going to use a PC as a router, use an Unix that has real net
> code, like xBSD.

*shit. You have no clues. Linux was better at networking than BSD
even in 1.2.x days ...

All the limitations of the Linux/PC router are due to the PC hardware
architecture. As seen on the list, people put 8 cards in the same
PC. This exceeds the bus speed of a PC. Even a single 100Mbps NIC
kills the PCI bus in most PCs should it run full speed. Also, you have
to be very carefull with the NIC you choose.

PCs simply were not built for forwarding packets and fast I/O.

Of course a Linux/PC will never beat a cisco :slight_smile: but the cost is
sometimes an order of magnitude lower for roughly the same
performance.

> Lots of types interfaces? EIGRP? Things that Linux/PC does have is Hard
> drives that fail, etc; I guess you could do a flash thing, though. Other
> protocols, like IPX, Dec, ATalk, etc are not going to happen.

Will you please point out other router than cisco which has EIGRP ?

My point exactly.

What kind of interface do you want ? You have async (multiport
async), sync, ethernet, fddi and now atm is coming. BTW, do you get
arcnet with cisco ? :slight_smile:

HSSI?

IPX and appletalk have been there for a _long_ time. There is also a
Linux DECNET project.

Great, another *project*.

> Also, for a long time, Linux had a hard time with lots or routes.

Define lots. You want full BGP table in a PC router ? Why :slight_smile: ?

Isn't that the crux of the conversation here?

*shit. You have no clues. Linux was better at networking than BSD
even in 1.2.x days ...

Heh. Ok. (as in, *yeah right*).

All the limitations of the Linux/PC router are due to the PC hardware
architecture. As seen on the list, people put 8 cards in the same
PC. This exceeds the bus speed of a PC. Even a single 100Mbps NIC
kills the PCI bus in most PCs should it run full speed. Also, you have
to be very carefull with the NIC you choose.

Thus, omre reason to not use a PC for routing..

This was hashed and rehashed not to long ago on some mailing list
(inet-access?)... Whay again?

PCs simply were not built for forwarding packets and fast I/O.

Again, thanks for agreeing.

Of course a Linux/PC will never beat a cisco :slight_smile: but the cost is
sometimes an order of magnitude lower for roughly the same
performance.

Not since the 2600 and 3600 have been released.

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   ISPF, The Forum for ISPs by ISPs. October 26-28, 1998, Atlanta, GA.
    Three days of clues, news, and views from the industry's best and
    brightest. http://www.ispf.com/ for information and registration.

     Atheism is a non-prophet organization. I route, therefore I am.
       Alex Rubenstein, alex@nac.net, KC2BUO, ISP/C Charter Member
               Father of the Network and Head Bottle-Washer
     Net Access Corporation, 9 Mt. Pleasant Tpk., Denville, NJ 07834
Don't choose a spineless ISP; we have more backbone! http://www.nac.net
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --

> What kind of interface do you want ? You have async (multiport
> async), sync, ethernet, fddi and now atm is coming. BTW, do you get
> arcnet with cisco ? :slight_smile:

HSSI?

ATM is doable today. There are multiple HSSI vendors out there:
http://www.lanmedia.com/products.htm
http://www.sdlcomm.com/

> Define lots. You want full BGP table in a PC router ? Why :slight_smile: ?

The bandwidth through a router is not necessarily directly related to the
amount of routing information that one needs to support. For example, one
can easily imagine a multi-homed site that buys transit from multiple
backbones. While no single backbone is deserving of very high bandwidth,
the need to maintain a full routing table for optimal return path routing
could be quite high.

Tony

> Will you please point out other router than cisco which has EIGRP ?

> My point exactly.

Ah, so you are using _only_ cisco in your network ? Tough!

> > What kind of interface do you want ? You have async (multiport
> > async), sync, ethernet, fddi and now atm is coming. BTW, do you get
> > arcnet with cisco ? :slight_smile:
>
> HSSI?

I'm sure that the moment someone will have a reasonably priced card
for a PC and make specs available, linux will support it (probably BSD
as well).

> > IPX and appletalk have been there for a _long_ time. There is also a
> > Linux DECNET project.
> Great, another *project*.

... for a protocol that everyone *uses* :slight_smile:

> > > Also, for a long time, Linux had a hard time with lots or routes.
> >
> > Define lots. You want full BGP table in a PC router ? Why :slight_smile: ?
>
> Isn't that the crux of the conversation here?

No ?! Point being that a PC router (eventually running linux) can be
more than successfully used as a low-to-middle end router, at very
good value for money. Anything more than this looks a bit absurd to me
... If I have the money to pay for multiple transit providers, I most
certainly have the money for a 4500 or more (as in real router) ...

And BTW, I didn't imply that a linux box has troubles with many
routes, but I never tested it myself. But once I read the source code
for route.c and I don't think the number of installed routes is an
issue.

> > ...
> Thus, omre reason to not use a PC for routing..

> > PCs simply were not built for forwarding packets and fast I/O.
> Again, thanks for agreeing.

But I don't :slight_smile: I just think they should not be used for
high-performance stuff, but perform just fine as long as you know what
to expect.

> > Of course a Linux/PC will never beat a cisco :slight_smile: but the cost is
> > sometimes an order of magnitude lower for roughly the same
> > performance.
> Not since the 2600 and 3600 have been released.

Fine, replace "an order of magnitude" with "a lot cheaper" and you're
still close enough :slight_smile:

However, we're beating a dead horse. I think we both realize what can
and can't be done with a PC router.

It's just that you were overly criticizing Linux as a router without
being at least informed and I felt an urge to react because I'm a
happy linux user :-))

Cheers,

Yep, available for pc routers running *bsd or linux.

It is just this point which keeps me running a PC router. I cannot
afford to buy a cisco with 4 100Mbit ethernet cards in it. But I can
afford 100Mbit PC cards all day long. Even one quad card. I would
love to use IOS for the access lists and other goodies.

rob

Anybody else have any problems with exodus, or is it just me?

traceroute to beta.sierra.com (209.67.71.106), 30 hops max, 40 byte
packets
1 gw.netmdc.com (206.29.138.1) 3.689 ms 1.867 ms 1.714 ms
2 netmdc-gw.abq-rtr-01.ihighway.net (206.29.131.21) 14.854 ms 20.363
ms 14.58 ms
3 bordercore2-hssi5-0-6.Bloomington.cw.net (166.48.176.25) 35.153 ms
77.288 ms 35.456 ms
4 core5.SanFrancisco.cw.net (204.70.4.85) 110.107 ms 65.928 ms 78.949
ms
5 * pb-nap-OC3-1.exodus.net (198.32.128.25) 51.899 ms *
6 * * *
7 * scca-05-p1-0.core.exodus.net (209.1.169.41) 86.462 ms 140.852 ms
8 scca-25-p0-0.core.exodus.net (209.185.9.30) 82.383 ms * *
9 * sewa-01-p0-0-0.core.exodus.net (209.185.9.186) 145.048 ms *
10 * * 209.67.64.21 (209.67.64.21) 131.238 ms
11 * 209.67.68.102 (209.67.68.102) 122.367 ms 153.178 ms
12 * * 209.67.73.2 (209.67.73.2) 139.211 ms
13 beta.sierra.com (209.67.71.106) 88.127 ms * 130.032 ms
ZD(ekool@ns1)D(506/ttyp0)D(04:07pm:10/29/98)D-
@D($:/etc)D- ping -c 5 beta.sierra.com
PING beta.sierra.com (209.67.71.106): 56 data bytes
64 bytes from 209.67.71.106: icmp_seq=0 ttl=114 time=130.6 ms
64 bytes from 209.67.71.106: icmp_seq=2 ttl=114 time=191.4 ms
64 bytes from 209.67.71.106: icmp_seq=3 ttl=114 time=246.8 ms
64 bytes from 209.67.71.106: icmp_seq=4 ttl=114 time=132.1 ms

--- beta.sierra.com ping statistics ---
5 packets transmitted, 4 packets received, 20% packet loss
round-trip min/avg/max = 130.6/175.2/246.8 ms

  _ __ _____ __ _________
______________ /_______ ___ ____ /______ John Gonzalez/Net.Engineer
__ __ \ __ \ __/_ __ `__ \/ __ /_ ___/ MDC Computers/netMDC!
_ / / / `__/ /_ / / / / / / /_/ / / /__ (505)437-7600/fax-437-3052
/_/ /_/\___/\__/ /_/ /_/ /_/\__,_/ \___/ http://www.netmdc.com
[---------------------------------------------[system info]-----------]
  4:05pm up 18 days, 19:34, 5 users, load average: 0.00, 0.03, 0.00

Where/how are you doing simulations with that many routes? Feeding full
routes to a linux 2.0.3x box running gated, I get:

# cat /proc/net/route | wc -l
  54677
and that command takes 13s to complete. When dealing with that many
routes on a *nix box, better tools would be nice. On my 3640, I can do a
show ip route blah, and get immediate response. On the linux box above, I
have no such tool. I should fire up a copy of gated 4.x and see how gii
performs, but its kind of a drag not being allowed to use the newer code
on anything but personal testing/development systems.

Anybody else have any problems with exodus, or is it just me?

traceroute to beta.sierra.com (209.67.71.106), 30 hops max, 40 byte
packets

[...]

4 core5.SanFrancisco.cw.net (204.70.4.85) 110.107 ms 65.928 ms 78.949
ms
5 * pb-nap-OC3-1.exodus.net (198.32.128.25) 51.899 ms *
6 * * *
7 * scca-05-p1-0.core.exodus.net (209.1.169.41) 86.462 ms 140.852 ms

We're seeing the same thing. We've been route-map-ing around it since
10/02/98. On that date Exodus claimed that "MCI [C&W] is oversubscribed to
the PB-NAP."

Jim Browne jbrowne@jbrowne.com
    "I wish journalism would return to its glory days, when principled
  incorruptible men like William Randolph Hearst and Joseph Pulitzer were in
       charge."- Vanessa Jackson, Piano Teacher, www.theonion.com

We have been having problems reaching hotmail and ebay for weeks now..
Our customers called them, and they said it was within our network because
they couldn't see us (traceroute clearly showed congestion on their side
though). We finally did get some info out of them:

Exodus net admitted that they have a bandwidth problem to MCI, and their
border routers are severely congested. They are currently in the process
of increasing their bandwidth with C&W. Their estimate (in their own
words) is approximately 40 days until the circuits are upgraded.

So, it will be quite a while before this will be remedied apparently.

Erica L Johansson
Network Administrator
ServiceCo/Road Runner

Seeing the same thing. Got the same story from Exodus that it is a C&W
problem, but I also see problems on traces not from C&W.

-Chris

> > Also, for a long time, Linux had a hard time with lots or routes.
> No longer applies. In fact Linux is now faster than BSD up to about
> 60-70,000 routes. BSD is faster at about 200,000. In between its about
> even.
Where/how are you doing simulations with that many routes?

These are just numbers reported by Alan Cox and Alexey Kuznetsov.

Feeding full routes to a linux 2.0.3x box running gated, I get:
# cat /proc/net/route | wc -l
  54677
and that command takes 13s to complete.

Well thats great for benchmarking procfs B)

I wonder how long 'netstat -rn' takes on *BSD with 54k routes.

-Dan

I'm not sure I follow. What circuits is Exodus going to upgrade?

AFAIK, Exodus has no circuits directly to MCI^H^H^HCW (for private
peering)... Rather, Exodus exchanges traffic with CW at the MAE/NAP's,
and via their Savvis transit ATM VC's (one to the Savvis NYC PNAP at 67
Broad St., and another so some Savvis PNAP in LA).

Some clarification would be greatly appreciated.

Thanks,
-asr

Well, when I called them, the guy that I spoke with seemed a little
clueless and wanted to keep blaming the problem on MCI. I had done some
traceroutes and had seen other areas using various backbones having
problems also. When I was on the phone with the guy, when he still
maintained it was a problem on our side, I simply asked him what their
connectivity was like, or what their network looked like. He had to put
me on hold for several minutes, came back, and then started saying
something about "We are working on a contractual agreement with C&W, but
it will be approximately 40 days before we work that out." I had another
coworker call later on..and that is what they had told us that time.
I honestly don't have any idea what their network looks like, nor who they
have connectivity and/or peearing with. This is just the information they
had given us. They didn't give us really anymore..and the person that
gave us this bit, seemed a little more clueful about what was going on.
I'm just glad that they are finally acknowledging that there is _some_
problem on _their_ side of things.

Now...hopefully they will have it fixed in the 40 days they are
maintaining.

Erica L Johansson
Network Administrator
ServiceCo/Road Runner

I agree, this is a big gaping hole in functionality. This has to go into
gated because only gated has useful information like what the AS path for
the route is, etc.