I35 bridge collapse in Minneapolis

Telephone call gapping by the major long distance carriers into the region seemed to be in effect for a while. I don't believe this is one of the
five critical Mississippi River fiber crossing points, so Internet traffic
appears mostly unaffected.

For four months dozens of our users who are Comcast subscribers have had difficulty reaching St. Olaf College's and Carleton College's network services.

We have worked through everything we can think of with our Onvoy (regional ISP) network engineers. We have isolated the problem a couple of Comcast's IP subnets, but need a contact within Comcast to further troubleshoot.

The behavior in a nutshell:

I've forwarded your message to the appropriate team within Comcast.

  - Alain.


This rings a bell. Do they block the mandatory ICMP
fragmentation-needed messages?

Try this: on your web server, reduce the MTU from 1500 bytes to 1400
bytes and see if the affected comcast users can now access your web

Bill Herrin


Either your firewall/router or the customer's firewall/router is blocking PMTUD packets. Fragment needed, but don't fragment bit set. Look at your ICMP access list and make sure you are allowing: permit icmp any any unreachable from any Internet address. I suspect an overzealous firewall admin is blocking all icmp. Read the acronym to him/her and explain that some icmp is necesary for the Internet to work.


Tellurian Networks - Global Hosting Solutions Since 1995
http://www.tellurian.com | 888-TELLURIAN | 973-300-9211
"Well done is better than well said." - Benjamin Franklin

Robert Boyle wrote:

Either your firewall/router or the customer's firewall/router is blocking PMTUD packets..... I suspect an overzealous firewall admin

> is blocking all icmp.

Which you can't do anything about if the overzealous firewall admin
is at the other end of the connection. My repeated, first-hand
experience has been that several of the better-known web sites out
there will happily send out 1500-byte packets with DF set, then
ignore the DEST_UNREACH/FRAG_NEEDED icmp responses they get. If you're
on the client end of this, you're sunk unless you initiate the
connection specifying a lower MSS.

Linux has a nifty iptables option (clamp-mss-to-pmtu) to rewrite the
MSS in TCP SYN packets when forwarding a packet onto a link with
a lower MTU than the MSS in the packet. Works like a charm. If every
packet forwarding device on the Internet did this, PMTUD would not be
needed. As is, PMTUD is simply broken, due to widespread firewall
misconfiguration. As in so many other cases of Internet misbehavior,
you can avoid being part of the problem, but you can't be the solution.

Jim Shankland

.. non-TCP traffic?


Adrian Chadd wrote:

AIX 5.2 and earlier supported it for UDP (we're getting out of the AIX
business, so I can't speak to what 5.3 does). Basically, it would send out a
gratuitous 64K ICMP Echo Request with DF set, and waited to see what came back.
I ended up turning it off all over, simply because we didn't have enough
UDP-based services that actually hit frag issues to make a difference.

AIX 5.3 changes the whole PMTU scheme to work more like the RFCs and not depend on out of band ICMP packets.

From http://www.ibm.com/developerworks/aix/library/au-aix5l-me.html :

PMTU discovery in AIX 5L Version 5.3
The current Path Maximum Transmission Unit (PMTU) discovery implementation uses ICMP Echo Request and ICMP Echo Reply packets to discover PMTU. Some system administrators set up their firewall to drop ICMP Echo packets, resulting in the above method of PMTU discovery to fail. The PMTU discovery mechanism in AIX 5L Version 5.3 is implemented with TCP packets and UDP datagram instead of ICMP Echo packets.