Blah...forgot to send this to nanog as well...it seems there is a lack
of understanding of why this works in some situations and not in others,
so attempted to spread some knowledge here.
Also Sprach Leo Bicknell
but I find it slightly (emphasis on the slightly) that someone would
turn on PMTU discovery, and then filter it out right in front of the
boxes where they turned it on.
As someone else mentioned...it happens all the time...disconnect between
server and network admins probably, or something along those lines (or
just general cluelessness)
Also, it seems to me most DSL users are behind PPPoE links with lower
MTU, and should get hit by the same problem.
No. The trick, here, is that the PPPoE (typically) terminates on the
same system that's terminating the TCP connection, so the PPPoE end
system can see that the PMTU is going to be, at most, 1492, so it can
use a lower Maximum Segment Size in TCP to start the whole scenario off
at the 1492 MTU size and try to go down from there. You're seeing the
problem because the tunnel is not terminated on the system that's also
terminating the TCP connection, so the TCP processing can't know about
the 14xx MTU somewhere out there except through PMTU (which is broken in
this case), so it can't set the corresponding MSS to compensate for it
initially.
The temporary hack is to have tunnelbox1 clear the DF bit on all
incoming packets, which just causes the packets to get fragmented going
down the tunnel. A minor performance hit, but it works.
An only slightly better hack would be to have the tunnel and/or firewall
twiddle the MSS on outgoing TCP connections to compensate for the lower
tunnel MTU. Still pretty gross, but won't have as much of an effect on
the TCP performance.
Are the servers really that broken (PMTU enabled, ICMP Can't Fragement
filtered)?
Yes. The last time I ran into this, my test site was www.harvard.edu
(!)...though that's been a year or more ago, so they may have resolved
their issues since then. We ran into plenty more sites that had the
problem, but that's the one that sticks out in my mind because, like I
said, it was the one that I used as a site to try to connect to as a
test.
Does the head end box of DSL services generally do something to work
around this (ie, clear the DF bit)? Am I just being an idiot and
missing something obvious?
I wouldn't say idiot, or missing anything obvious...but you were missing
the whole MSS issue. I've never thought the behavior was intuitive or
obvious...but once you think about it becomes a "Why didn't I think of
that?" sorta thing.
