Good morning and happy holidays all. I'm in the process of creating an
automated filtering application and would like to know if anyone can
point me to the right place. I'd like to be able to query a
site/db/etc., and pull out specific netblocks to create fw rules. Since
IP space is always changing, it would be helpful if my queries can be
tailored to something like:
wget site | Parse IP space | grep Company | create rule
Or:
wget site | Parse IP space | grep {EDU_IP_SPACE,MIL_SPACE,GOV_SPACE} |
create rule
Follow?
Right now I am using potaroo with something like :
wget -qO -
http://bgp.potaroo.net/ipv4-stats/allocated-{apnic.html,ripe.html, etc}
But this just gives me entire blocks, not who is behind them. Is there
any site I could use to query specifics? E.g., for a gov client: wget
-qO - this.site.org | grep "\.gov" | parse_with_awk '{print "fw_rule"}'
Thanks in advance and Happy New Year to everyone.
You may want to look at Capirca (http://code.google.com/p/capirca/) for
creating policy files from which to generate your firewall rulesets. I am
not aware of a simple categorization of netblocks. My first thought is that
an agreement with every RIR for bulk whois data and writing code to parse /
categorize would be quite difficult and may not get you a reasonable result
after all that work - maybe there is something commercially available.
-Ryan
Good morning and happy holidays all. I'm in the process of creating an
automated filtering application and would like to know if anyone can
point me to the right place. I'd like to be able to query a
site/db/etc., and pull out specific netblocks to create fw rules.
[...]
But this just gives me entire blocks, not who is behind them. Is there
any site I could use to query specifics? E.g., for a gov client: wget
-qO - this.site.org | grep "\.gov" | parse_with_awk '{print "fw_rule"}'
Given the current IPv4 climiate, this sounds like a terrible idea. The
landscape has changed dramatically from what it once was. Large
volumes of mobile carriers use NAT, many IPv6 to IPv4 gateways are out
there routing traffic, and we'll soon see a time in which entire
countries are transiting over small chunks of IPv4 space. Never mind
the fact that applications on services like Google App Engine have a
different IP nearly every time they connect because of outbound proxy
pools.
I think you're going to have a very difficult time resolving an IP to
the appropriate owner. Coarse calculation of who might be in charge of
a block is possible but fine-grained discovery and classification of
an owner is a difficult task.
That being said, the tools that I'm using on a daily basis to figure
out who actually owns an IP block (or is sending traffic over it) are:
- Senderbase (Cisco)
- cymru whois (whois.cymru.com - good for fast bgp lookups and geo)
- http://multirbl.valli.org/dnsbl-lookup (multi-rbl lookup , good for
finding abusers and other issues)
- SmartViper (Website ownership) http://www.markosweb.com/
-John
Good morning and happy holidays all. I'm in the process of creating an
automated filtering application and would like to know if anyone can
point me to the right place. I'd like to be able to query a
site/db/etc., and pull out specific netblocks to create fw rules.
[...]
But this just gives me entire blocks, not who is behind them. Is there
any site I could use to query specifics? E.g., for a gov client: wget
-qO - this.site.org | grep "\.gov" | parse_with_awk '{print "fw_rule"}'
given an ASN you can query their announcements from RouteViews DNS no?
(or rsync that and do the lookup locally in whatever form you feel is
helpful)
That probably has some whois data easily tied to it as well...
Given the current IPv4 climiate, this sounds like a terrible idea. The
landscape has changed dramatically from what it once was. Large
if you are updating filters 'quickly' it shouldn't matter, right?
you'll catch things (presuming whois is updated and/or BGP is and you
can tie things back through asn/netblock relationships, oh...
RPKI...) pretty quickly as they move.
volumes of mobile carriers use NAT, many IPv6 to IPv4 gateways are out
there routing traffic, and we'll soon see a time in which entire
countries are transiting over small chunks of IPv4 space. Never mind
I don't recall the OP saying 'ipv4' only?
the fact that applications on services like Google App Engine have a
different IP nearly every time they connect because of outbound proxy
pools.
it's probably not 'every time they connect' there's probably some
sensible reasoning behind the decision process.. like your query that
triggers it comes into "METRO-X" and thus outbound queries come from a
netblock for NAT things inside "METRO-X", my query goes to "METRO-Y"
so ... diff netblock.
Inside a set of queries (10-100?) you'll see a repeated set of ips, I suspect.
-chris