I'm in a disagreement with ARIN about my application for bulk whois
data. I've got a software program that needs resolve AS numbers to the
Company Name of the owner. The software app has need to do this on a
very high volume. E.g. I run a report that returns the top 100 AS
destinations for my network and I want to resolve the numbers to the
names as part of the report generation. Since ARIN throttles the number
of queries that you can execute against their servers I seems to "just
make sense" that you would do the processing using local data.
That is all fine and good, but the problem comes when I distribute the
software to users. ARIN's AUP for bulk whois states:
"Redistributing bulk ARIN WHOIS data is explicitly forbidden. It is
permissible to publish the data on an individual query or
small number of queries at a time basis, as long as reasonable
precautions are taken to prevent automated querying by
database harvesters."
My original AUP application stated that I would transfer the data to the
users using an XML file on a regular basis. Clearly in violation of the
first point. Fine. But now after a phone conversation they are telling
me that I can not operate a server to distribute the data on a "per
query" basis too. Providing a server that answers whois queries just
like ARIN seems to be clearly permissible based on the remaining AUP
verbage. At this point the only thing I can get out of the guy/gal on
the phone is "NO!".
Does anyone have any experience doing something like this? How about a
sanity check? Am I completely wrong in how I'm interpreting the AUP?
altho arguably its not up to arin to provide processing power for all
these
deployments.
if you can get a local copy why not have your clients resolve back to
that?
that is the point of his post actually - arin told him that he can't do that
without pointing out where this is prohibited in the aup. i can see their
point - they're trying to restrict the practicality of attempting to harvest
the data and an open to the public whois server with no access restrictions
would defeat that. perhaps asking arin if they would consent to you running
a server open to registered users of your app behind authentication of some
sort is worth a try?
Interestingly, the Caida data lacks AS numbers issued in the last several months, while the data is present in the arin.net and potaroo.net listings. Not sure where Caida gets its data, but the other sources would seem to be better for the project the original poster had in mind.
It's also more than a bit odd, IMO, to assign a latitude and longitude to AS numbers. While some networks are well localized, many others span continents. As such, the value of that part of the caida data set seems nebulous.