Thanks for the suggestions. We're considering doing a follow-up and perhaps making
this a regular feature every 2 or 4 months thereafter. We've received a few suggestions
for methodology changes/enhancements, and also several emails so far denouncing our
methodology but not explaining why (which is typical of people in many areas -- politics,
the environment, economics, whatever -- who disagree emotionally but not intellectually
with the conclusions of a study).
The current methodology generally shows how a web site connected to a particular
backbone appears to the general internet population of users. The results are intended
to be a guide (but not the only one) for helping web sites select or evaluate a collocation,
hosting, or access provider.
Your methodology suggestion would be useful to include because its results would also help
end-users select their dial-up ISPs based on the backbone that those ISPs are connected
to.
Gene Shklar
Gene Shklar wrote:
Thanks for the suggestions. We're considering doing a follow-up and perhaps making
this a regular feature every 2 or 4 months thereafter. We've received a few suggestions
for methodology changes/enhancements, and also several emails so far denouncing our
methodology but not explaining why (which is typical of people in many areas -- politics,
the environment, economics, whatever -- who disagree emotionally but not intellectually
with the conclusions of a study).
My disagreement isn't based on emotion. If you had followed the thread
you would have
picked up on the point that under _most_ circumstances, the
ISP/NSP/IXP's web servers
are typically _not_ on the fastest part of their network. They leave
this space for
revenue generating customers.
The current methodology generally shows how a web site connected to a particular
backbone appears to the general internet population of users. The results are intended
to be a guide (but not the only one) for helping web sites select or evaluate a collocation,
hosting, or access provider.
Your going to need cooperation from these sites to put a resource in
their colo space
not use their web server. A better way to tackle this would be to
solicit the cooperation
of one or two of their actual customers who provide a service from that
backbone.
Your methodology suggestion would be useful to include because its results would also help
end-users select their dial-up ISPs based on the backbone that those ISPs are connected
to.
I highly doubt this will really help anyone pick an ISP correctly. It
doesn't show
the most annoying side of the equation; getting connected. What is the
ISP user to
modem ratio? What is their customer to bandwidth ration? etc. I doubt
the average
joe-user will even think or ask this.
I don't have an issue with somebody trying to provide QoS info about
ISP/NSP etc, but
you can't do it just by downloading 56k of data. There is a lot more to
it. How
do each segments behave with differing packet sizes, etc. Fragmentation
will take its
toll. I could go on but I should do some real work.
Thanks for the suggestions. We're considering doing a follow-up and
perhaps making
this a regular feature every 2 or 4 months thereafter. We've received a
few suggestions
for methodology changes/enhancements,
I just thought of another solution to the problem of too many variables.
The numbers that you came up with this time are essentially meaningless
because there are so many variables involved that shift from time to time,
and by taking one month of measurements and boiling it down to a single
number, you simply don't learn anything useful. However, if you were to do
a similar test and come up with a single number for each day and then plot
that number for one month, then it would show some useful info. If a
provider has one bad day, it doesn't blow their average but it is still
clearly visible. And if a provider has 10 bad days, then that is also
visible and is arguably more useful than a single number which might be
seen as implying a slower network. Personally, I would rate an erratic
network as far worse than a slower one.