We have been struggling with firmware bugs from a specific router vendor. I am looking to set up a test lab of our core network and a few remote site routers. Protocols would include SR-MPLS, ISIS, EVPN MPLS and L3VPN with a little OSPF sprinkled in. I’d be grateful for any tips or resources anyone has that might cover testing strategies and/or best practices.
In any lab,I find concurrent access to serial ports is still an essential diagnostic tool. In a pinch you can get a used Cisco 2811 for $100, but there are multiport devices from lots of vendors. These let you SSH into the server and then connect to any serial port, giving you separate serial port windows all on the same screen. I’ve become fond of the WiFi-capable multiport modules from get-console.com. The ability to record logs from these serial ports in real-time helps a lot for documenting regression tests.
If you’re able to virtualize some of these appliances, almost any virtualization platform can be setup to connect nodes together. However I really enjoy using Eve-ng. It’s a network virtualization web app that can be deployed on bare metal or in a VM. It lets you diagram, connect and configure setups and do testing with different network nodes. What is pretty nice with this is you can then just click on any of the connected links and wireshark pops up and you’re viewing packet captures instantly. It kind of reminds me of Cisco’s VIRL, but with maybe less overhead? It has enabled me to run and connect several vendor appliances and build up configuration prior to acquiring the physical hardware for deployment.
++ all that was said thus far. Physical equipment with console access is the best way to test software/firmware issues. As for virtualization, it's great for expanding your topology quickly.
Use a virtual bridge in GNS3 or EVE-NG and you can make your smaller footprint physical lab into a larger topology with ease -- especially around cabling. It also allows you to do packet generation & link simulation (packet loss, jitter) much easier. You can even couple it with T-Rex.
if you are just testing control plane features the virtualized stuff works
well. i personally use vrnetlab.
if you are trying to test hw features and use actual hardware:
remote controlled power
optical cross connect boxes(calient/telescent)
if you are testing optical stuff with distance limitation something like a
I love the built-in Wireshark capability in EVE-NG. BTW, EVE-NG Community is free. You just have to get images for anything you want to emulate. Virtual images for various vendor products are sometimes freely available, with trail licenses. For instance Junipers vMX was freely available for a while with a 60 day license. …also vSRX, vQFX, and the new vJunos-switch (I think vEX).