As I'm attempting to lay out in my posts, there are are a plethora of problems, end-to-end in the network. Would that there was only one problem.
Excessive, unmanaged buffers afflict the user's OS's (Windows, Mac and Linux alike), particularly on recent hardware. Home routers and the broadband connections (as shown by netalyzr) all have problems.
The bottleneck may be anywhere in the path; with the (sometimes) exception of Windows XP, all edge equipment now routinely congests the edge.
Multiple seconds of latencies, in both directions, are dismaying commonplace.
Retail operators have had a hidden major support problem: how many of the "bad service" calls have been due to the problem? It tends to be transient in behavior, and I've chased the problem personally at least 5 times in the last 3 years. I've placed service calls I now believe likely due to bufferbloat. I've caught problems with crash dumps being uploaded to the net; backup and downloads can all cause trouble.
Courtesy of the Netalyzr team, I've been able to post color versions of their results on http://gettys.wordpress.com/2010/12/06/whose-house-is-of-glasse-must-not-throw-stones-at-another/; they first reported results at the NANOG meeting last summer.
Disentangling broadband data from home router and operating system bufferbloat is difficult; I've found bufferbloat is present in all of them. The power of two bufferbloat sizes are almost certainly all broadband gear (since the OS buffer sizes are quantised in packets, not bytes).
In the downstream direction, one of the possible causes is be failure to run any AQM in the broadband head-ends; this is certainly also the case in home routers.
Also, as outlined in: Characterizing Residential Broadband Networks
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.65.6825&rep=rep1&type=pdf we have good reason to believe this is taking place.
Since I became aware that bufferbloat might have become a generic problem last summer from anecdotal data and personal experiments, I've probed networks wherever I've travelled. Some of what I've seen was clearly broadband bufferbloat; but more disturbingly, I've also seen other evidence further into several of the networks I've probed (from hotels *not* using broadband for their service), further confirming the initial anecdotal data I was given that queue management is far from universal (and essentially unheard of in the home).
If the idea that the buffers have destroyed congestion avoidance doesn't scare you, I don't know what will.
It's a major problem.
- Jim Gettys