Internet Exchanges supporting jumbo frames?

What's driving the desire for larger packets?

A single bit error will drop a whole packet.
Larger packets will cause more loss. Cables will need to be
shorter or bitrates lower to compensate.

Byte overhead of packet headers?

Are we seeing degradation of packets per second in forwarding
due to the increase in IPv6? Are we seeing IPv6 packets with
hop-by hop extension headers (which forward on the slow path)?

Increasing the packet size will reduce the number of TCP
packets as well as the number of TCP ack packets.
TCP ACK packets are significantly larger in IPv6 than IPv4.

TCP slow start is faster with large MTU. Is this an issue?

Are IPv6 packets with extension headers causing performance
degradation in firewalls?

Thanks,
Jakob.

Thus spake Jakob Heitz (jheitz) (jheitz@cisco.com) on Fri, Mar 18, 2016 at 09:29:44PM +0000:

What's driving the desire for larger packets?

In our little corner of the internet, it is to increase the performance
of a low number of high-bdp flows which are typically dataset transfers.
All of our non-commercial peers support 9k.

Dale

Then it's mainly TCP slowstart that you're trying to improve?

Thanks,
Jakob.

I would hazard a guess that reducing the packet header overhead *and* the Ethernet interframe gap time by a factor of 6 could make enough of an improvement to be quite noticeable when dealing with huge dataset transfers.

Tim McKee

You would hardly notice it.
Helium is 4 times as heavy as hydrogen, but only marginally less buoyant.

Header overhead:
Ethernet=38
IPv4=20
TCP=20
Total=78
Protocol efficiency:
1500: 1500/1578 = 95%
9000: 9000/9078 = 99%

That's 4% better for a TCP packet, not 600%.

Thanks,
Jakob.

The factor of 6 was just in reduction of overhead. Granted in the greater scheme of things the overall 4% is relatively insignificant, but there have been many times when doing multiple 10-100+GB transfers that I would have welcomed a 4% reduction of time spent twiddling thumbs!

If that's an actual concern in your production network, you probably have
bigger issues you need to be dealing with....

scheme of things the overall 4% is relatively insignificant, but there have
been many times when doing >multiple 10-100+GB transfers that I would have
welcomed a 4% reduction of time spent twiddling thumbs!

The 4% increase in available bandwidth is only part of the equation though.
I would think that having to encap/decap 1/6 the number of packets(frames)
from host to host and all routers/switches along the way would be
beneficial, especially since some of these could be processing these in
software. Certainly if there are FW or IPS involved. I'm not sure about
the host side of things, but I'm guessing there would be efficiency
increases there as well.

Chuck