Hi Nanog-ers,
Hoping someone may have come across a similar issue. Has anyone ever seen a situation where maybe like a Level3 transport system could be possibly dropping LACP frames..?
End point A - tx and rx counts incrementing for LACP
LACP info: Role System System Port Port Port priority identifier priority number key et-0/0/0.0 Actor 127 5c:45:27:6d:2a:c0 127 56 16 et-0/0/0.0 Partner 1 00:00:00:00:00:00 127 56 16 LACP Statistics: LACP Rx LACP Tx Unknown Rx Illegal Rx et-0/0/0.0 6925 6922 0 0
End Point B - no RX, partner macs are 0s..
LACP info: Role System System Port Port Port priority identifier priority number key et-9/1/0.0 Actor 127 5c:45:27:77:d6:c4 127 68 16 et-9/1/0.0 Partner 1 00:00:00:00:00:00 1 68 16 LACP Statistics: LACP Rx LACP Tx Unknown Rx Illegal Rx et-9/1/0.0 0 6752 0 0
Link works fine otherwise outside the aggregate and w/o LACP. Any inputs will be greatly appreciated.
thanks,
-nevin
What is performing the LACP? The Level3 transport system for the most part
is purley optical, so I don't think it touches LACP. Did you check the hash
values?
Yes. Many vendors are using l2vpn/pseudo-wire services of one sort or
another to provide circuits and most do not transport LACP by default.
LACP uses slow-protocols address:
https://wiki.wireshark.org/LinkAggregationControlProtocol
If they are using ALU gear, they can enable this using the port command:
configure port <port> ethernet lacp-tunnel
I’ve seen optical transport gear be non-transparent in a few situations when
using OTU2 vs OTU2e, but they turned out to be a bug. If a L2 platform is
being used to front-end a router, you could be seeing slow protocols like
CDP/LLDP/LACP being consumed by that switch. If you configure LLDP does it
pass end-to-end?
- Jared
I’ve seen optical transport gear be non-transparent in a few situations when
using OTU2 vs OTU2e, but they turned out to be a bug.
I've seen this as well, including in an SDH transport, where OSPF
packets were being eaten (something Multicast-related).
If a L2 platform is
being used to front-end a router, you could be seeing slow protocols like
CDP/LLDP/LACP being consumed by that switch. If you configure LLDP does it
pass end-to-end?
That's what I was thinking at first when I read the OP's post. If the
circuit is being transported over MPLS or being fronted by an Ethernet
switch, Level(3) would have to tunnel Layer 2 protocols in order to
support your LACP frames across the pw.
Mark.
Nevin, good day.
Thanks all..!
I just had to sit and trace all the cables to make sure the tx/rx lined up for the right circuits as well as hitting the right patch panel ports. Once all that got aligned nicely things started working magically. thanks,-nevin
Nevin, good day.
To the OP's case, commercially, I'd find it interesting to transport a
100Gbps circuit as EoMPLS rather than EoDWDM, considering the amount of
bandwidth one would need to throw at an IP/MPLS network to transport
100Gbps effectively...
Mark.
Or a very reckless oversubscription ratio and misjudgment of the customer,
example, if a provider had 2 x 100GbE capacity between two locations and
sold a customer a 100GbE EoMPLS transport circuit from A to Z, based on the
mistaken idea of "Well these guys probably aren't going to peak more than
35Gbps of traffic at any time in the near future....". Frightening.
Yeah, I wouldn't do that. Easier and cheaper to deliver the circuit over
EoDWDM if you can't reserve enough capacity in the backbone.
You could get away with it by doing an N x 100Gbps LAG, but EoMPLS
traffic may or may not load balance well, depending on platform and payload.
Mark.
Tue, May 24, 2016 at 12:39:03PM +0000, Nevin Gonsalves wrote:
I just had to sit and trace all the cables to make sure the tx/rx
lined up for the right circuits as well as hitting the right patch
panel ports. Once all that got aligned nicely things started working
magically.
Yep, ports in an "up" state, but LACP not working is the sign of bad
cabling: had been hit by this overnight once when I was preparing to
leave the facilities next day for conference, but ought to make 10G
for the new servers working. Took around 1/2 hour to sense what
happened at that time (tx was going to, say, port A and rx -- to port
B, but overall all ports were receiving tx and rx) and 3 hours for
rewiring and swearing: probably I am more skilled in the former than
in the latter 
Thought that you had checked this in the first place; my bad.
Thanks for sharing!