Completewhois New Features - RBL Lookup and Search Utilities

Hello everyone,

Over the past month several new features and utilities have been added
that are likely to be of interest here. In this post I'll focus on RBL lookup related utilities which have to do with rbl data from about 30 lists (with one or two exceptions, pretty much covers 25 most used
free lists) that we're collecting and aggregating in the database (updates once per day) for analysis and allow users to check on based
on individual queries and ip ranges.

I. First of there is now direct whois lookup facility to check if ip address or domain is in one of those lists - you can simply do
  whois -h ip-address/domain
OR (to show all lists that were checked)
  whois -h RBL_INCLUDENOMATCH=ON ip-address/domain
OR (to include RBL data with normal whois lookup)
  whois -h RBL ip-address/domain

The queries are completed in average 1/2 second, so results are always fast.
Note that no ip ranges are accepted (or going to be) on the whois interface.

II. The web interface and light documentation for RBL Lookup (individual
queries interface to our system) is available at

The website utility has two types of output display - one user-friendly (now default) table showing lists that matched and did not as red and green and including links to the list pages (good for less RBL-familiar
users who want to know what to do) and simple format based on whois (can be easy to cut-paste from) and which is used when you want to also combine
query with whois and dns data.

The website lookup CGI can also be refernced directly (already is and used quite heavily) from other places and applications, do it as in this example:

There is also real-time RBL check utility for 200 lists (not using our database and so quite slow) available on the bottom of the page and you
even have a choice there to use several dns libraries (ADNS, FireDNS, BIND
resolver) and compare how fast/slow they work...

III. Another utility on the website allows to do IP range searches and
is intended to be used primarily by ISPs and network operators to check
on the listings that cover their own ip blocks (this is to help make operators aware of the extent of possible abuse coming from their network).

The interface to this is available at:

The search utility is restricted to maximum /24 range as allowing more
then that could in my opinion (and others I consulted) facilitate abuse
rather then help stop it. To be able to do more then /24 lookup on your
ip block(s), you will need to register and get username and password in
our system (its still all free - registration is just making sure only
ISP who is assigned the ip block can do query on entire block). Also note
that use of this utility is covered by separate AUP.

The results from ip range search also come in several formats, including
simple list on the website, comprehensive webtable format as well as an
option to produce CSV file for export to spreadsheet. The queries and searches are typically done in 1-2 seconds for /24 and about 4-8 seconds for /16 (with 1000 matches from various RBLs).Webtable adds additional couple seconds for large (500+) matches.

In the future if there is an interest, further work will be done on the
search interface ISP features to allow not only one-time lookups but reports that can be generated and sent automatically. Options will also include ability to do query based on specified time range (i.e. only new RBL entries that appeared in last 7 days). You will need to tell me what you want and how its to be presented, if you expect new features and note that any further work on this will be done end August or later when I come
back from IETF conference.

P.S. For those who like statistics, there are currently 1.8 million
individual RBL entries which as an aggregate cover about 2 /8s (not
all fair comparison because spews level2 covers large ip ranges where
as many other lists are more specific). About 100 thousand (varies,
very low on weekend but can be lot more some days) get updated every
day and most active as far as updates is Spamhaus XBL.