We have a research project at OCLC that does web crawling. We created an
email account (netsecurity@oclc.org) that admins could contact and posted
the email address at the web site from which the crawling was done. That
way when admins see our IP address in their firewall logs (from attempted
http connects), they could browse to that address and see an explanation of
the project and send email to the contact address to get more info or
request removal from the IP space being crawled.
See http://132.174.5.40 for our set up.
...Kevin O'Neil