Not an entirely whacky concept actually. The way hot potato routing works,
this could actually be a "purer" test I suspect of the network internally
and a purer test of connectivity of any network to all others cum Internet.
At least in the other direction, complementing what we are probably
measuring. Unfortunately, it would require the cooperation of the backbones
themselves, and a number of them would not find it in their interest,
consider it proprietary, etc. etc. ad nauseum. The problem is, that many
of these companies benefit in the marketplace from this game of liars
poker, and don't WANT ****ANY**** testing or results.
Keynote does a "Top 40" study of 40 popular web sites and I believe they
make the results available on their web site. It is interesting to observe
performance variations of the network as a whole over time. Other than
that, we don't have much interest in it. It is indicative of no specific
network, but of the Internet in general.
Jack Rickard
==>Not an entirely whacky concept actually. The way hot potato routing works,
==>this could actually be a "purer" test I suspect of the network internally
==>and a purer test of connectivity of any network to all others cum Internet.
The article says that you're measuring backbone provider performance, yet
you're including:
* carrier's web server location
* carrier's web server performance
==>Keynote does a "Top 40" study of 40 popular web sites and I believe they
==>make the results available on their web site. It is interesting to observe
==>performance variations of the network as a whole over time. Other than
==>that, we don't have much interest in it. It is indicative of no specific
==>network, but of the Internet in general.
/cah
for an exmple of somewhat more complete and better designed benchmarks of
this type:
http://www.inversenet.com/about/backgrounder.html#2
note that they understand the numerous factors that contribute to overall
performance. only a marketing droid could think downloading 50k worth of
web pages is somehow an indicator of overall performance.
b3n
Where does my network end and the Internet in general begin?
Where does your testing equipment end and the Internet
in general begin?
Along those lines, I'm happy to announce that I'm prepared
to offer one megabyte of FREE web space to any backbone.
It will be hosted on www.cybernothing.org, a FreeBSD
machine running Apache, currently co-located with Erol's
and soon to move to the Priori office.
Every backbone whose web page is hosted on my server is
almost guranteed to have equal network performance in the
eyes of Keynote, Boardwatch, and anybody nieve enough to
believe them.
I'll even put the same 10k file in each directory. *grin*
for an exmple of somewhat more complete and better designed
benchmarks of this type:
http://www.inversenet.com/about/backgrounder.html#2
We make web measurements from over 800 points (POPs) on the Internet for
roughly 20 ISP's using a standard basket of 10 popular web pages (porn
pages excluded given there is the believe it will drive down the
productivity of our operation staff 
We control the test by having every ISP runs through the same set of
pages at roughly the same time. (By the way, to closely emulate
consumer end user's experience, we specifically use a [painful] test
harness to drive Netscape 3.0 on Windows95 over boring 33.6 modem line
... would have love to use a UNIX script with our own custom browser
over our direct link into the net instead ... but then it is not what
consumer users are doing ... oh well).
Even with the extensive matrix of measurements, the only claim we dare
to make is that it approximate a end user experience index a la Dow
Jones's basket of companies. (By the way, with few interesting
exceptions, mostly limited by the size of the final pipe - the modem
link).
We think there are way too much variables involved to allow us to
defintively partition out the components such as the page structure (how
many other embedded objects we have to retrieve, compressibility by
modem of those objects etc.), the Web server load at the time the
measurement was made, the load of the web server's LAN, the load and
size of that LAN's pipe into the server's ISP, that ISP's network
performance, its peering performance with the end user's ISP (and
transit links if any), the user's ISP's network performance .... etc.
etc. etc. Even with extensive analysis, it will be difficult for us to
derive the backbone performance component from our end to end
measurement.
If it is the underlying network performance one wants I think a
different measurment infrastructure such as those being developed by
Paxon/Mathis/Jamshid (NIMI), Almes/Barber (Surveryor), and Graig
Labovitz (Merit) ... using tools such as Jacobson's Pathchar will be
much more appropriate.
Regards,
John Leong