Inktomi (now Yahoo!) sends it's spiders all over the Internet. Lately some of our systems are reporting that they open many HTTP connections to our web sites, without ever sending any data and immediately disconnecting. This is getting to a level where it disturbs us.
Is something broke over there? I can't seem to be able to reach them and this is becoming a real annoyance.
Anyone else observing this?
Inktomi (now Yahoo!) sends it's spiders all over the Internet. Lately
some of our systems are reporting that they open many HTTP connections
to our web sites, without ever sending any data and immediately
disconnecting. This is getting to a level where it disturbs us.
I have heard previous stories of inktomi ignoring robots.txt (not seen
this for myself though). And there are threads like this -
Quoting from http://www.webmasterworld.com/forum11/1968-1-15.htm
back in 1999 inktomi hammered our nameserver (which never has, and never
will run http. ever.) After _weeks_ of complaining to them and to their
upstream exodus (hah!) I finally got them to stop. Only to have them
start up again a month later.
not suprising to see them up to their old antics again.
time to nullroute i guess?
-Dan
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
not sure if spiders falls under spam or ddos bracket when they
repeatedly start hammering one's network. you could possible report to
spamcop (*grin*) to get a quicker response. spamcom hasn't been accurate
in some instances 
do you remember this incident, http://www.cs.wisc.edu/~plonka/netgear-sntp/
regards,
/vicky
Dan Hollis wrote:
not sure if spiders falls under spam or ddos bracket when they
repeatedly start hammering one's network. you could possible report to
spamcop (*grin*) to get a quicker response. spamcom hasn't been accurate
in some instances
Er.. just what would you report to spamcop, and what would spamcop do with your
reports?
do you remember this incident, http://www.cs.wisc.edu/~plonka/netgear-sntp/
Not very new .. broken apps which keep hammering on a resource for some reason
are a fairly regular "feature" of the internet.
srs
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
in-line:
Suresh Ramasubramanian wrote:
not sure if spiders falls under spam or ddos bracket when they
repeatedly start hammering one's network. you could possible report to
spamcop (*grin*) to get a quicker response. spamcom hasn't been accurate
in some instancesEr.. just what would you report to spamcop, and what would spamcop do
with your
reports?
- ------------------
that's why i asked, this type of behavior falls under what abuse terms?
do you remember this incident,
http://www.cs.wisc.edu/~plonka/netgear-sntp/
Not very new .. broken apps which keep hammering on a resource for
some reason
are a fairly regular "feature" of the internet.
- -----------------
doesn't mean that it shouldn't be blocked/reported.
regards,
/vicky
Block - you have enable on your routers and can do everything from
access list 101 deny to something fancier like NBAR to block just
these queries.
All Inktomi/Yahoo crawling is done from 68.142.248.0/22; the whois
entry for that block says to report issues to slurp@inktomi.comHave you tried alerting them to the problems yet?
If yes, and if you didn't receive a response, please forward me the mail
that you sent, and I'll see to it that the right people see it.
Thank you. I am now in contact with them. I must admit it is the first time I ever got any kind of abuse response from Yahoo!, but the people who contacted me seem serious. I will know more later on as to how well they handle it, but they seem like people who want to help.
Gadi.