Has anyone else exerienced problems with AOL's web proxy servers? We
switched the IP address of a web site about 5 months ago, and we are still
getting about 10,000 hits a day from AOL on the old address. We dropped a
note to them a ways back but didn't get much of a response.
Brian Horvitz
Shore.Net
On a related note.
I would be curious to the liability issues surrounding the use of web
proxy servers and Cisco Cache engines. ISP/NSPs traditionally only
forwards packets or are at least responsible to a certain degree for
resolution of IP addresses from DNS and routers. The use of these web
cache technologies allows ISP/NSPs to now *intelligently determine* what
data the requesting client receives. With web proxy server, at least the
client is aware(we hope) that data is being cached but the Cisco Cache
engine makes that process transparent. As we are all well aware, these
intelligent implementations don't always work. It seems that we are on
slippery grounds making content decisions for end user requests. And
doesn't it become even messier with the suggested/proposed web caching at
the MAEs or at the NSP level?
Regards,
Turnando Fuad
NSNet
Brian Horvitz wrote:
Has anyone else exerienced problems with AOL's web proxy servers? We
switched the IP address of a web site about 5 months ago, and we are still
getting about 10,000 hits a day from AOL on the old address. We dropped a
note to them a ways back but didn't get much of a response.
they've been problems for a long time; some of their proxy servers
hold cache data for far too long. i have some web discussion pages
that are effectively useless for aol subscribers because aol even
caches cgi generated stuff with obvious cgi related extensions like
".cgi" and ".pl".
they appear to have little organizational control over the software
running on their server farm, either for software versions or
configuration files; there are clearly several different variants of
sendmail running, in addition to several different versions of
their proxy software. it's kind of a mess, and they've never returned
any of my email notes.
sigh,
richard
It may well be worse than that, depending on judicial interpretations
of the Cubby and Stratton-Oakmont cases. I suspect that S-O wouldn't
apply to this circumstance, but who can tell what a judge will think.
Cheers,
-- jr 'oh, yeah: Happy New Year!'
Nope this post can't be configured into your router Randy.
Brian Horvitz wrote:
> Has anyone else exerienced problems with AOL's web proxy servers? We
> switched the IP address of a web site about 5 months ago, and we are still
> getting about 10,000 hits a day from AOL on the old address. We dropped a
> note to them a ways back but didn't get much of a response.they've been problems for a long time; some of their proxy servers
hold cache data for far too long. i have some web discussion pages
that are effectively useless for aol subscribers because aol even
caches cgi generated stuff with obvious cgi related extensions like
".cgi" and ".pl".
I'm taking a wild guess that your CGI responses include neither
Last-Modified or Expires headers. HTTP/1.0 (rfc1945) doesn't define what
a proxy cache is to do in this situation, so AOL is compliant. HTTP/1.1
(rfc2068) section 13.2.4 gives more details on this, but that's not what
AOL implements.
Not to say that AOL's caches aren't littered with other standards
violations.
The DNS thing isn't something that only AOL is innocent of. Netscape
navigator up through the 3.x versions (I haven't tested the 4.x versions,
they may have fixed it) caches DNS responses for the lifetime of the
browser. Given that some folks on stable unix machines are able to keep
their browser open for months this sucks. One might argue it has
something to do with the lack of timeout information in the
gethostbyname(3) API. I've always taken the opinion that DNS changes for
webservers regardless of your ttl values, will take at least two weeks and
you better plan to either lose some of the hits or run extra IP aliases
with a tunnel from the old addresses or whatnot.
Dean
I think I would have to categorize this 3 ways. No 1, caching in
principle and, working properly, is probably not breaching any lines of
legality, even as they stand now, and commen sense tells us that's it's a
good thing (not to say that common sense would help you in, at least an
american, court of law.) No 2, what AOL has is what I refer to as
'caching negligence'. They run a cache because it's good idea, but they
run it poorly, and fail to notice or respond when it screws up. Probably
the only people who have any legal recourse here are the AOL members,
and not web site operators, since it's due to the negligence of AOL that
the service for the members suffers. No 3 would be 'caching with
prejudice'. This would get you in a whole lot of trouble if a provider
was manipulating the cache, for any of a number of reasons. Regulating
the content would probably be ok, but is available in many other products.
Manipultaing the content would get you into a lot of trouble.
Just my $0.02.
Nick Bastin
System Administrator - World Trade Internet Communications
Dean Gaudet wrote:
The DNS thing isn't something that only AOL is innocent of. Netscape
navigator up through the 3.x versions (I haven't tested the 4.x versions,
they may have fixed it) caches DNS responses for the lifetime of the
browser. Given that some folks on stable unix machines are able to keep
their browser open for months this sucks.
In current versions of Netscape, IP addresses are cached for fifteen minutes. To
tweak this, add or change the following line in your prefs.js file
user_pref("network.dnsCacheExpiration", 900) // The integer is the timeout in
seconds
One might argue it has something to do with the lack of timeout information in
the
gethostbyname(3) API.
This is, in fact, why it was implemented that way -- gethostbyname is available on
pretty much every platform. I've pointed the folks responsible for this code to
res_query() related documentation and code, so this may make it into a future
version.
Dan
nbastin@mail-2.wtic.net wrote:
> On a related note.
>
> I would be curious to the liability issues surrounding the use of web
> proxy servers and Cisco Cache engines. ISP/NSPs traditionally only
> forwards packets or are at least responsible to a certain degree for
> resolution of IP addresses from DNS and routers. The use of these web
> cache technologies allows ISP/NSPs to now *intelligently determine* what
> data the requesting client receives. With web proxy server, at least the
> client is aware(we hope) that data is being cached but the Cisco Cache
> engine makes that process transparent. As we are all well aware, these
> intelligent implementations don't always work. It seems that we are on
> slippery grounds making content decisions for end user requests. And
> doesn't it become even messier with the suggested/proposed web caching at
> the MAEs or at the NSP level?I think I would have to categorize this 3 ways. No 1, caching in
principle and, working properly, is probably not breaching any lines of
legality, even as they stand now, and commen sense tells us that's it's a
good thing (not to say that common sense would help you in, at least an
american, court of law.) No 2, what AOL has is what I refer to as
'caching negligence'. They run a cache because it's good idea, but they
run it poorly, and fail to notice or respond when it screws up. Probably
the only people who have any legal recourse here are the AOL members,
and not web site operators, since it's due to the negligence of AOL that
the service for the members suffers. No 3 would be 'caching with
So, go and get an aol account and start bitching. That's done easily
enough and is a small price to pay to ensure that the kajilions of
aol'ers see one's site properly. Granted, it's still hard not to
envision aolusers as ebola victims on a bus on the info super duper
highway.