Comparison of freeware open source switch software?

Mailman List Mirror (Read Only)
1

I have seen numerous comparisons and RIPE presentations on performance
issues of BIRD vs Quagga vs FRR.

I am looking for the same thing for freeware switch software.
Has anyone done a feature comparison between:
http://openvswitch.org/
https://www.openswitch.net/
https://cumulusnetworks.com/products/cumulus-linux/
...any other I am missing...

I am familiar with:
http://packetpushers.net/open-networking-cheat-sheet/
https://www.networkworld.com/article/2919599/cisco-subnet/clearing-the-fog-around-open-switching-terminology.html
so to clarify I am interested only in bare-metal or whitebox swicthes
and freeware, open source software.

And even better - has anyone done a benchmark to see which performs best?

Thanks,
Hank

2

Here's one you missed:
http://www.projectfloodlight.org/indigo/

If you're only interested in stuff that goes on iron, openvswitch is out - it's pure software meant to run on hypervisors

-Ed

3

Just a note that Cumulus is disaggregated and built on FOSS, but it is not free (costs dollars).

Some of the below lifted from http://packetpushers.net/virtual-toolbox/list-network-operating-systems/

Other somewhat free options, some user assembly required at times:
https://snaproute.com/
https://azure.github.io/SONiC/
http://opennetlinux.org/
https://github.com/facebook/fboss

Others that play in this space while not necessarily free:

http://www.pica8.com/products/picos
https://www.ipinfusion.com/products/ocnos/

4

If you're only interested in stuff that goes on iron, openvswitch is out - it's
pure software meant to run on hypervisors

Not necessarily true anymore. Look for SwitchDev, which is incorporated into the Linux kernel , is undergoing continuous improvement, and allows the kernel to offload forwarding rules to the hardware.

This allows open source software like Open vSwitch and Free Range Routing to work natively/directly with hardware.

http://www.mellanox.com/page/press_release_item?id=1983

https://www.kernel.org/doc/Documentation/networking/switchdev.txt

https://blog.raymond.burkholder.net/
Raymond Burkholder.

5

> If you're only interested in stuff that goes on iron, openvswitch is
> out - it's pure software meant to run on hypervisors

Not necessarily true anymore. Look for SwitchDev, which is incorporated into
the Linux kernel , is undergoing continuous improvement, and allows the kernel
to offload forwarding rules to the hardware.

The overall architecture of openswitch, however, seems (to me) to be focused on software implementations, rather than hardware.

You probably want to look at SONiC as well, as this is what LinkedIn is using; there are a lot of improvements currently going into code. For FR Routing, the performance is changing rapidly, as there are a lot of commits going in just about every week, and the community is quite active.

:slight_smile: /r

http://rule11.tech

6

> Not necessarily true anymore. Look for SwitchDev, which is
> incorporated into the Linux kernel , is undergoing continuous
> improvement, and allows the kernel to offload forwarding rules to the
hardware.

The overall architecture of openswitch, however, seems (to me) to be
focused on software implementations, rather than hardware.

I suppose that could be a fair statement, to a certain degree. But when you strip away all the abstracted things it does, and get into the core of the product, you find an OpenFlow engine, which at the heart, is a mechanism for defining traffic rules. Many of those rules translate easily into the kernel's forwarding information base. And the kernel's FIB can easily be two-way sync'd with hardware.

So, to take this to the logical conclusion, other than for the interesting rules which can be created in OVS or FRR, it is possible to do away with OVS/FRR, and use only Linux itself with iproute2 and bridge commands to drive the hardware directly with no further abstraction or vendor black boxes.

And, rather than using a J or a C syntax, you use an /etc/network/interfaces syntax for permanence.

https://blog.raymond.burkholder.net/
Raymond Burkholder

7

SwitchDev, which is incorporated into the Linux kernel

Neat! I'll have to keep my eyes on this in the future, it'd be cool if we could have VyOS handling routing on the hardware and the vm hosts, would save me a bit of brainpower

-Ed

8

It's my understanding that there simply is no such thing. Because none of the HARDWARE has open source code. Sure, anyone can write software to spirit packets between NICs (linux and *BSD has had that capability for decades.) But doing that "at scale" with the various manufacturers SoCs requires vendor specific code to setup and control the chip. The broadcom "NDK" is just a shim on top of a pre-compiled proprietary SDK blob.

--Ricky

9

https://www.opennetworking.org/

Hardware works quite well. I have a number of whitebox units deployed based
off their designs and will be ordering more.

10

My understanding is the same as Ricky's. At least in the Broadcom word, you
have to license the SDK from Broadcom in order to develop against it and,
more importantly, have documentation of which register does what. I don't
know if you need to license it to program the ASIC (assuming you can do it
without SDK in a sensible fashion).

My understanding was that when you buy software such as Cumulus Linux, what
you are actually paying for is the Broadcom license. You can actually go
and download Cumulus Linux and it's all open source except, you guessed it,
switchd, which is what takes the info from the linux kernel and programs it
into the hardware.

My understanding was that the rest the "open source" OSs operate the same
way.

Please, correct me if it is at all possible to buy a whitebox switch and
then load a "no cost OS" on it and it starts switching packets through
hardware.

-Andrey

--Andrey

11

My understanding was that when you buy software such as Cumulus Linux, what
you are actually paying for is the Broadcom license. You can actually go
and download Cumulus Linux and it's all open source except, you guessed it,
switchd, which is what takes the info from the linux kernel and programs it
into the hardware.

I believe that is Cumulus' business model where they ride the requirement for the Broadcom license. So part of the license fee is for Broadcom and part is for Cumulus.

Cumulus has used their licensing fees to develop and maintain tooling for their ecosystem. In that process, they have released many of their tools to the opensource world. Things like ifupdown2 and Free Range Routing are a result of that model.

My understanding was that the rest the "open source" OSs operate the same
way.

Pica8 does something similar.

Please, correct me if it is at all possible to buy a whitebox switch and
then load a "no cost OS" on it and it starts switching packets through
hardware.

The only case in which I know that you can purchase a bare metal switch and put a 'no cost OS' such as Linux on is Mellanox. They are the ones who have developed the switchdev module I mentioned in an earlier post. There is _no_ requirement for a black box module to interface between the software and the hardware. That whole path down to but not including the hardware is opensource.

There is a large carrier who is making use of switchdev:

https://lwn.net/Articles/675826/

so to clarify I am interested only in bare-metal or whitebox switches
and freeware, open source software.

There is something else called SAI, which may do something similar to switchdev:

http://packetpushers.net/sai-and-switchdev-need-to-succeed/

It's my understanding that there simply is no such thing. Because none of
the HARDWARE has open source code. Sure, anyone can write software to
spirit packets between NICs (linux and *BSD has had that capability for
decades.) But doing that "at scale" with the various manufacturers SoCs
requires vendor specific code to setup and control the chip. The broadcom
"NDK" is just a shim on top of a pre-compiled proprietary SDK blob.

As mentioned above, and in another message, Mellanox is the only one, that I know of, for providing a mechanism to drive the hardware (both switch and network cards) from the OS, without a pre-compiled proprietary SDK blob (black box).

http://www.mellanox.com/related-docs/prod_switch_software/PB_Spectrum_Linux_Switch.pdf

Disclaimer: I not affiliated with Mellanox, only a customer.

12

My understanding is the same as Ricky's. At least in the Broadcom word, you
have to license the SDK from Broadcom in order to develop against it and, more
importantly, have documentation of which register does what. I don't know if
you need to license it to program the ASIC (assuming you can do it without SDK
in a sensible fashion).

If you can live within SAI's capabilities, SAI is freely available for Broadcom. This is what SONiC uses to communicate to the Broadcom chipset, and it works well.

If you really want a rundown, please ping me off list; I have a good bit of information I could share from prior presentations... You can also look up my presentations at nanog/lacnog on the topic, but I'm glad to fill in what I know beyond what's there if it's helpful.

:blush:

Russ