Software-based Border Router

Mailman List Mirror (Read Only)
1

Hi All!

Just want to ask if anyone here had experience deploying software-based routers to serve as perimeter / border router? How does it gauge with hardware-based routers? Any past experiences will be very much appreciated.

I wanted to know because we've been asked if we want to assume full control of the internet link (up to the router). By assuming control up to the router, we still want to configure iBGP with our parent network so that we can take advantage of some routes available to the parent network's gateway. The saddest part is presently we do not have the router to serve as our gateway this is why we are considering the use of software-based routers.

Thank you.

2

Just want to ask if anyone here had experience deploying software-based routers to serve as perimeter / border router? How does it gauge with hardware-based routers? Any past experiences will be very much appreciated.

Software based routers (e.g. Cisco 7200 series) have been used as border
routers for many years - this is hardly anything new. The question you
should ask is probably: Can such a router handle a full link's worth of
DDoS using minimum sized packets? The answer, of course, depends on your
link capacity, the router itself, features enabled (ACLs, QoS, ...) etc.

There are quite a few people using Quagga based boxes running Linux or
FreeBSD as border routers - this is a possible solution too, giving
you more bang for the buck than a traditional software based router from
the big vendors. Make sure you have enough expertise for the relevant OS
and routing software available.

Steinar Haug, Nethelp consulting, sthaug@nethelp.no

3

Thank you for the prompt response. Just to clarify my previous post, I was actually referring to Linux/Unix-based routers. We've been considering this solution because presently we don't have any budget for equipment acquisition this year.

To be honest, I came across Vyatta Core while searching for viable Linux/Unix-based solution that we can adopt and I'm currently reading its reference guides. Has anyone here used this software before?

Thanks a lot.

4

Dear Nathanael,

Just want to ask if anyone here had experience deploying software-based routers to serve as perimeter / border router? How does it gauge with hardware-based routers? Any past experiences will be very much appreciated.

I wanted to know because we've been asked if we want to assume full control of the internet link (up to the router). By assuming control up to the router, we still want to configure iBGP with our parent network so that we can take advantage of some routes available to the parent network's gateway. The saddest part is presently we do not have the router to serve as our gateway this is why we are considering the use of software-based routers.

I operate freebsd / quagga core routers since 4 years.

pro: cheap, tcpdump at router
con: no support, no wirespeed

expected performance: 100kpps (1,2ghz pentium m) - 700kpps (quad intel
       core 2, 3ghz) - and much more with 10gige cards

issues: 4byte asn produced a crash at quagga (downtime 2h in 4 years)

to develop a good core-router, this means not only to setup a pc with unix and for example quagga, but setup an embedded unix to an appliance, for example with cf-cards (readonly).

Kind regards,
   Ingo Flaschberger

5

While Vyatta is a good piece of software for the Free version, the costs quickly increases as you have to purchase support and the version updates are few and far between with the Free version. The production (paid) version though is quite nice.

Another option though would be RouterOS. If it is a small site, doing BGP could be as little as $399 including the hardware! However, most people that do BGP will need a bit more horsepower. RouterOS will do your iBGP, OSPF, bandwidth controls, firewalling etc. The software license there is $45 beans! Super cheap. Hardware runs as low as $49 bucks to 10k depending on what you are needing. If you would like, please feel free to contact me off-list and I will be glad to recommend the proper hardware.

6

If one has a cisco 7200, then you have a software based border router.

Considerations, for a given router platform are capacity, susceptability to dos, features required etc. Depending on the capacity required a software device could do fine. If it's in front of hosting environment you want to know that it doesn't take dirt nap from a couple hundred mb/s of small packet.

Joel's widget number 2

7

What's your time worth?

Quagga on Linux is a fine software, but messing with the
idiosyncrasies is far more time consuming than buying a Cisco 2811,
adding enough RAM to handle BGP, configuring it once and forgetting
about it.

Also bear in mind that while your ISP's engineers can help you
configure your Cisco router, Quagga is a mystery to them. You can
still get help... but not from someone who also knows how the ISP's
network is configured.

This is not a problem if you have lots of experience with BGP routing. Do you?

Regards,
Bill Herrin

8

Another big problem for Linux/Unix-based routers of this size/cost is
upgrade-ability. If you need to add cards, you are going to have to bring
the router down for extended periods. Likewise, a software upgrade can be
a bigger deal than on a purpose designed router. If a router is mission
critical, Linux/Unixed-based has issues over extended periods.

regards,
Fletcher

9

Another big problem for Linux/Unix-based routers of this size/cost is
upgrade-ability. If you need to add cards, you are going to have to bring
the router down for extended periods. Likewise, a software upgrade can be
a bigger deal than on a purpose designed router. If a router is mission
critical, Linux/Unixed-based has issues over extended periods.

depends on knowledge, as mentioned in previous post.

I have 2 software based border routers - no problem bringing one down.
700kpps for 1200eur that can handle a full view.

and changing line-cards - could be really funny at c6500.

kind regards,
   Ingo Flaschberger

10

I do agree here. If you are not moving a lot of data then something like BSD or Vyatta may be a good alternative. You do still have possible reboots required and things you would not see as often with a hardware-appliance model. However, for cheaper than the cost of 1 appliance you could build in redundancy. I guess the question is how many PPS you plan to push, whether you have regularly scheduled maintenance windows that you could bring it down for a reboot, and whether the additional maintenance involved still keeps you in the black?

I am a big proponent of open source every thing. Although, I am a bigger proponent of stability and less maintenance. If you could prove out a software-based solution against the cost of a hardware solution then I don't see any reason not to go that route.

11

Once upon a time, William Herrin <bill@herrin.us> said:

Quagga on Linux is a fine software, but messing with the
idiosyncrasies is far more time consuming than buying a Cisco 2811,
adding enough RAM to handle BGP, configuring it once and forgetting
about it.

Yeah, because IOS and JUNOS don't have idiosyncrasies. :slight_smile:

12

Not gonna argue with you on that one. However, the world has changed
since the days where the chances of clueful unix systems engineering
knowledge and clueful BGP routing knowledge was highly guaranteed to be
found cohabitating in a single lifeform. You are far more likely to
find that relatively speaking most network engineers have very little
knowledge in unix systems engineering. This list may be an exception
but I would gather that the bulk of the network engineering workforce
are little more than power users (if that) when it comes to operating
systems.

13

We have looked at using open source routers for our border, but in the end we cannot make the numbers add up. Once Cisco released the x9xx ISR2 routers, the x8xx have tanked in price on the used market. So, for about the same as a vyatta router running on newer hardware that you can trust you can get a 28xx or 38xx. If you also want support, Cisco will support these at less than $100/month and that gets you access to the IOS upgrades and a 4 hr. replacement window. I know I sleep better knowing Cisco will drop off a router in less than 4 hours if one of mine fails.

Dylan

14

We use a mix of software and hardware based routers, have found little difference between the two platforms in terms of performance and stability. Our software base routers are serving a couple 100Mbps upstream links running on some HP Proliants with dual PS and dual HD's that we picked up on ebay for a $150 then loaded Quagga on them.

I actually have to give a little bit of a edge to the Linux based systems only because of all the all the other wealth of diagnostics/troubleshooting tools one gets with Linux in general...Its nice to be able to run Wireshark right on the systems if we need too.

As for troubleshooting, I've found the Quagga mailing list to be just as responsive (if not more responsive at times) as Cisco, but clearly your mileage will vary there.

Bret

15

I haven't found that to be the case. The larger memory space available to
the kernel allows for larger BGP tables and filtering tables. I've seen
BSD based systems running thousands of concurrent tunnels and the
processors available in the linux/BSD space bury anything that the router
manufacturers are overcharging you for. A properly planned upgrade or
addition of a card should take a maximum of 15 minutes as everything
should be plug and play. Some of the software based systems also come
from the manufacturer with the hardware.

If the network is configured properly with failover capabilities and only
one unit down at a time, down time is minimal or non existent. Software
upgrades happen in a matter of minutes.

Cheers,
--Curtis

16

Do jitter sensitive applications have problems at all running?
What would you say is the point at which people should be looking for
a hardware forwarding solution?

Differences:
- Hardware forwarding
- Interface options
- Port density
- Redundancy
- Power consumption
- Service Provider stuff - MPLS TE? VPLS? VRF??

Any others?

17

Oh, support contract!!?

18

I have seen software based routers (FreeBSD+Quagga) in production at pennies on the dollar compared to Cisco for quite some years.

Up front, as other people have noted, you need to know what you are doing. There is no 'crying for help 24x7'. By the same token, if you know what you are doing then they can be a very cost effective solutions.

I have yet to see (or try out) MPLS and such, so if requirements need features like that, then probably open source may not be the solution.

The above said, other comments inline below...

Do jitter sensitive applications have problems at all running?
What would you say is the point at which people should be looking for
a hardware forwarding solution?

Differences:
- Hardware forwarding

Yes, absolutely, no hardware forwarding. This must be compensated for by utilizing as advanced/expensive 'commodity PC hardware' as possible. You want lots of CPU horsepower, fast busses (PCI-E x16 if possible) and good NICs so the OS can offload as much as possible to the hardware and not be bandwidth constrained. Even then, no way are you going to get anything close to what you can from a 'real' router. A classic trade off between technical needs & desires vs. financial constraints.

- Interface options

Make sure there are least two NIC platforms. i.e., a pair of onboard dual gigabit plus another dual gigabit card. Bond the interfaces between the separate NIC platforms so one each gigabit link is off say the onboard and one off the NIC card. Utilize LACP.

- Port density

Use VLANs - again, a quality NIC will help with this by offloading a good portion of the overhead to hardware.

- Redundancy

Use a /29 to your eBGP provider and turn up two routers side-by-side. Again, if you are looking for hard core 'carrier grade' stuff, you should not be asking about open source. Pair the two routers, for eBGP sessions, and use a separate interface for them to talk to each other.

- Power consumption

Always an issue, no way are you going to get pps from this kind of stuff like you would from Cisco.

- Service Provider stuff - MPLS TE? VPLS? VRF??

Yup.

Any others?

If somebody is on an extremely tight budget, is technically capable of doing utilizing open source to do what they need, and their requirements are limited enough that an open source platform would work for them, I would suggest they check into it. Ultimately, as always, it is buyer beware. Often with dedicated routers a support contract can cost as much as the router itself after a year or two, but sometimes companies need that support contract because they don't have the in-house skills already, etc.

I would never recommend either open source or dedicated hardware routers to anybody as a 'this is the only way to go' solution.

19

Vyatta has support contracts. If you want hardware, they've got that, too.

20

Vyatta has hardware forwarding? Real hardware forwarding? Where?

Best Regards,
Nathan Eisenberg