Hey there
Any ISP using Juniper Fusion Provider Edge?
I am trying to chat with an engineer besides Juniper engineers to understand how buggy (or not) this is to go on production for a medium size ISP.
Any feedback good/bad appreciated.
Fusion has made a lot more sense since Juniper changed the licensing model from every switch AND the MX to just the MX.
We’ve deployed it in some of our sites. It is very cool from a forwarding plane perspective, but from a control plane standpoint it’s very…meh. For example, you can’t get SNMP oids for light levels or even read them right from the CLI. You have to log into the satellite switch like you would log into an FPC just to get light levels. That’s probably the dumbest thing we’ve dealt with though. I’ve also heard you can have them do local L2 forwarding, which can be nice for latency and conserving uplink bandwidth, but we don’t do any L2 that way so I wouldn’t know the implications. From what we can tell though, it does give you Trio L3 performance and features with a MUCH cheaper port cost which is exactly what we were looking for, the extended reach of the chassis was just a fantastic bonus.
We also REALLY like that we can have one pair of MX dists for a whole data center with hundreds of thousands of square feet of raised floor and deploy QFX5100 or EX4300 switches in every pod and haul back over just a few pairs of fiber. Saves a lot of time because all that’s required to turn up a new connection is a cross connect in the pod. It also allows us to offer copper ports very far away from the MX device, which would normally require media converters.
We’ve wanted to experiment with doing this over dark fiber in the metro as well, but we want to feel out any kinks locally before we add additional failure modes.
Very interested in hearing about other’s experiences with Fusion, good, bad, and ugly.
-Matt
Hi there,
About Juniper Fusion PE and our experience with it.
For example, you can’t get SNMP oids for light levels or even read them right from the CLI.
Sure it’s possible but also with a big „meh“. Here is how:
"show interfaces diagnostics optics satellite “ (<- on the MX)
BUT at least with MX Junos 16.1R7 and aligned SAT Image aka SNOS these values are wrong
by a pretty big offset. Juniper promised they already fixed it but we can’t confirm (at least not in MX Junos 16.1).
Soon we will take a look at MX Junos 17.3 with aligned sat image.
I’ve also heard you can have them do local L2 forwarding, which can be nice for latency and conserving uplink bandwidth, but we don’t do any L2 that way so I wouldn’t know the implications
Same thing here … we don’t really need it. At least it’s on the roadmap and/or already implemented with higher Junos/SNOS versions.
From what we can tell though, it does give you Trio L3 performance and features with a MUCH cheaper port cost which is exactly what we were looking for, the extended reach of the chassis was just a fantastic bonus.
Yep, that is really amazing and the reason we use it on many MXes. You can get rid of almost all ports you want (restrictions apply tho).
We also REALLY like that we can have one pair of MX dists for a whole data center with hundreds of thousands of square feet of raised floor and deploy QFX5100 or EX4300 switches in every pod and haul back over just a few pairs of fiber. Saves a lot of time because all that’s required to turn up a new connection is a cross connect in the pod. It also allows us to offer copper ports very far away from the MX device, which would normally require media converters.
We use Junos PE NOT as a replacement for any switch and/or ip fabrics within a datacenter. We use it to get rid of many customer/client ports (mainly 1G and 10G ports)
which were directly connected to our MXes before. Atm I would not recommend using any closed fabrics beyond that scope. If it works for you … ok.
We’ve wanted to experiment with doing this over dark fiber in the metro as well, but we want to feel out any kinks locally before we add additional failure modes.
At the moment? Don’t do it. If you run mpls on so called „core router/dwdm/wan facing ports“ you have to know that this is totally not supported on extended satellite ports.
It’s not even on the roadmap. I already started to „collect“ some other ISPs to push juniper towards this feature because technically there no
real reason why fusion should NOT be capable of pushing some mpls labels on already tagged 802.1br packets.
Best regards,
Vincentz
I’ve, generally, always been wary about these “satellite” systems since Cisco introduced them with the ASR9000v back in 2012.
My whole thing is it being a closed system that requires the same vendor hardware from start to finish. Back then, wiring things up redundantly where all ports are in action was not possible, not sure where things stand now but I guess there are solutions to that.
Ultimately, I didn’t want to be in a position where I am no longer satisfied with the solution, but my only recourse is to either lose the investment or continue with the same vendor just to save face.
Recently, we had to vote with our feet as Juniper behaved badly on their EX4550’s and EX4600’s. So we are now switching to Arista, including getting rid of the old gear as it can’t scale anymore. If we had gone this satellite route, we’d have fewer options to maneuver.
The vendors love to push these systems because they say it will simplify your life - and in essence, it probably will - but this is a proper lock-in of note, and no vendor hates that.
Mark.
Hi there,
About Juniper Fusion PE and our experience with it.
For example, you can’t get SNMP oids for light levels or even read them right from the CLI.
Sure it’s possible but also with a big „meh“. Here is how:
"show interfaces diagnostics optics satellite “ (<- on the MX)
BUT at least with MX Junos 16.1R7 and aligned SAT Image aka SNOS these values are wrong
by a pretty big offset. Juniper promised they already fixed it but we can’t confirm (at least not in MX Junos 16.1).
Soon we will take a look at MX Junos 17.3 with aligned sat image.
I’ve also heard you can have them do local L2 forwarding, which can be nice for latency and conserving uplink bandwidth, but we don’t do any L2 that way so I wouldn’t know the implications
Same thing here … we don’t really need it. At least it’s on the roadmap and/or already implemented with higher Junos/SNOS versions.
From what we can tell though, it does give you Trio L3 performance and features with a MUCH cheaper port cost which is exactly what we were looking for, the extended reach of the chassis was just a fantastic bonus.
Yep, that is really amazing and the reason we use it on many MXes. You can get rid of almost all ports you want (restrictions apply tho).
We also REALLY like that we can have one pair of MX dists for a whole data center with hundreds of thousands of square feet of raised floor and deploy QFX5100 or EX4300 switches in every pod and haul back over just a few pairs of fiber. Saves a lot of time because all that’s required to turn up a new connection is a cross connect in the pod. It also allows us to offer copper ports very far away from the MX device, which would normally require media converters.
We use Junos PE NOT as a replacement for any switch and/or ip fabrics within a datacenter. We use it to get rid of many customer/client ports (mainly 1G and 10G ports)
which were directly connected to our MXes before. Atm I would not recommend using any closed fabrics beyond that scope. If it works for you … ok.
We’ve wanted to experiment with doing this over dark fiber in the metro as well, but we want to feel out any kinks locally before we add additional failure modes.
At the moment? Don’t do it. If you run mpls on so called „core router/dwdm/wan facing ports“ you have to know that this is totally not supported on extended satellite ports.
It’s not even on the roadmap. I already started to „collect“ some other ISPs to push juniper towards this feature because technically there no
real reason why fusion should NOT be capable of pushing some mpls labels on already tagged 802.1br packets.
Best regards,
Vincentz
Hi Mehmet,
This has been discussed on the Juniper-NSP list several times, here's
a couple of examples:
https://puck.nether.net/pipermail/juniper-nsp/2018-November/036673.html
https://puck.nether.net/pipermail/juniper-nsp/2018-April/035397.html
https://puck.nether.net/pipermail/juniper-nsp/2016-June/032964.html
Cheers,
James.
thank you this is very useful to know
Mehmet
There is a fundamental product limitation. CoS on Cascade port for MX is not officially supported as well QFX acting as AD.
I agree with those who perceive all these approaches as proprietary lock-in (disguised as cheap).
QFX10k is the AD in Fusion Datacenter. In a Fusion Edge setup it is MX.
Can I test fusion using vMX and vQFX ? Will it work?
-Aaron
Hi Aaron,
Can I test fusion using vMX and vQFX ? Will it work?
I have tried and haven't managed to get it working. It's one of the improvements that I would like to see in vMX and vQFX.
#featurerequest
Cheers,
Sander