L3 over OTN vs pure IP/MPLS

Dear Nanog community

We are currently experimenting TCP degradation issues in some metro markets
where there are multiple POPs and the IP packets have to pass multiple L3
access devices (routers) before reaching the core router. The more L3 hops
that it goes through the more degradation we see in the Internet service.
L3 routers have the capacity in terms of pps and BW and we are using just a
small percentage of that capacity but anyway the service is degraded (No
CRC/Input errors in the links). A couple of vendors are recommending using
OTN or MPLS-TP to backhaul the connections from the access to the core but
obviously deploying OTN means a considerable amount of capex. The
flexibility and simplicity of the L3 based access network is great but
given the performance issues we are experimenting I would appreciate if you
can share your experience/recommendation with either OTN or MPLS-TP. Does
it really make sense to use a OTN/Photonic layer to backhaul the L3
connections in a metro network?

Your input is greatly appreciated

Thanks and have a great day

Are you running your Metro-E networks on dark fibre all the way into
your core, or are you leasing a lit service from another carrier for this?

If the former, you should not have any issues with performance
considering a well-designed network. If the latter, you are in the hands
of your carrier, and you would need to qualify that they know what they
are doing before you overlay your network on top of theirs.

I cannot find any benefit in running your Metro-E network on dark fibre
vs. OTN into the core apart from when you hit the port bandwidth limits
in your Metro-E network and need to go to DWDM to upgrade that so as to
keep the fibre-count down while increasing bandwidth.

We run a large dark-fibre-based Metro-E network without any DWDM
equipment between most rings and our core, and there is no tangible
difference in performance between services delivered on those rings or
services delivered upstream in the core data centres. There is also no
difference in performance between the dark fibre rings vs. the DWDM
rings (although, depending on your equipment vendor, some platforms
could introduce quite a bit of serialization delay, but not enough to
affect actual performance).

Sounds like your vendors are trying to sell you boxes...