MTU problems with GRE tunnels (fwd)

To whom it may concern,

here is some email I received in the last months followed by
some of my observations which might be related to the problems
discussed. I have posted my obervations to comp.sys.dcom.cisco
and opened a trouble ticket with cisco's technical assistance center.

# Forwarded message:
# > From merit.edu!errors-nohumans Fri Jun 5 23:44:49 1998
# > Message-Id: <3.0.3.32.19980605095358.006ebd4c@mailhost.ip-plus.net>
# > X-Sender: bridge@mailhost.ip-plus.net
# > X-Mailer: QUALCOMM Windows Eudora Pro Version 3.0.3 (32)
# > Date: Fri, 05 Jun 1998 09:53:58 +0100
# > To: nanog@merit.edu
# > From: philip bridge <bridge@ip-plus.net>
# > Subject: MTU problems with GRE tunnels
# > Mime-Version: 1.0
# > Content-Type: text/plain; charset="us-ascii"
# > Sender: owner-nanog@merit.edu
# > Content-Length: 1881
# >
# > I'm experiencing problems with fragmentation due to Cisco GRE tunnel
# > overhead: the way I understand it, the MTU if a GRE tunnel will always be
# > less than the MTU of the underlying IP cloud (in our case 1500 bytes) due
# > to the IP encapsulation overhead. So 1500 byte packets attempting to
# > traverse the tunnel will be fragmented, or dropped if the DF bit is set, in
# > which case an ICMP message is send back to the originating host
# >
# > We're trying to use GRE tunnels extensivly in some fancy added-value
# > Internet services, and it seems that there is a small but significant
# > amount of application traffic out there that has problems when traversing a
# > GRE tunnel with MTU < 1500. We've seen two problems:
# >
# > - 1500 byte packets with DF set. This is either application traffic, or MTU
# > path discovery is broken, because the same packets get sent repeatedly
# > - 1500 byte packets get fragmented, but the destination host cannot cope
# > with the fragmentation (firewall issues?)
# >
# > We see this on a variety of platforms (from 2500, 7507) and a variety of
# > IOS releases (11.1(18)CC, 11.1(2), 11.2(5). Talking to another provider
# > indicates that the same problem exists with other vendors, and is having
# > the same severe impact.
# >
# > Thinking about it, this is a problem is to be expected with IP tunnels of
# > all types, but I am surprised at the extent it's influence on our
# > customer's applications (such as large emails). I do not want to overstate
# > the proportion of traffic we see with this problem - but it does seem to be
# > enough to render GRE tunnels very problematic - to say the least. But I
# > know lots of people are using GRE for this or similar applications...so
# > what am I missing here.
# >
# > thanks in advance for help/tips
# >
# > Phil
# >
# >
# >
# > ______________________________________________________________
# > Philip Bridge
# > ++41 31 688 8262 bridge@ip-plus.net www.ip-plus.ch
# > PGP: DE78 06B7 ACDB CB56 CE88 6165 A73F B703
# >

It's well known problem... not for Cisco (any connectionless-based
tunnelling cause MTU to be decreased) but for those MS-based application
which do not know how to deal with the fragmentation AND use big (1500
bytes) packet sizes. The only object have to be treated here is
applications, not routers... Through it's possible to imagine some ways
to over-fix this by router's software...

The application MUST:
- do not use DF bit;
OR
- do not use long (> 1024) packets at any cases, AND understand ICMP
packets about MTU size and MTU discovery protocol.

Any application do not corresponding to this is niot garanteed to
work in the Internet.

# > I'm experiencing problems with fragmentation due to Cisco GRE tunnel
# > overhead: the way I understand it, the MTU if a GRE tunnel will always be
# > less than the MTU of the underlying IP cloud (in our case 1500 bytes) due

We can confirm that IP in IP tunneling is broken too...

To quote a specific case, Sparc/Solaris networking over a tunnel with IP in IP
will break, e.g. the largest ping you can do is "ping -s xyz 1452", so you have
to reduce the MTU, we lowered ours to 1400 (arbitrary reduction) and it makes
the problem "go away".

The router in question was running 11.1, but from what's been said, it looks
like it's generic across the range.

I think the Perl program posted was a great idea, now if only Cisco would add
a Perl interpreter to IOS :slight_smile:

Paul