Folks,
Have a gremlin we have been chasing around for several months now and it’s becoming a major issue as we are getting tighter on IPV4 and needing to give some provider assigned space back.
In June we received a /22 from ARIN. As is my workflow I started announcing it but waited a month while I checked out the geolocation databases for correct info, did testing ,etc. All this time our test accounts could browse web-sites, etc.
We put one of the pools into production and things ran good for awhile. Then we started getting the occasional web-site was not working. After several of these we started assigning the customer an IP out of one of our other ARIN blocks and the web-site would be fine and reachable. The issue seems to reside just on this /22. We have other blocks from ARIN and they are just fine. We can assign an IP out of this new block and can’t reach certain web-sites. We turn around and assign out of another block and web-site works just fine.
We have two upstreams and an IX on this network. We have tried withdrawing the route on this particular /22 and isolating to one upstream alone and the problems still persist.
Many of the web-sites in question are government (both state and local), online universities, and the occasional local news station. They are diverse enough to not be traced down to a common point, except the IP block.
We announce the IP block via BGP the same exact way we announce the other blocks. Traceroutes show the path going the same way no matter what IP block the customer has.
It acts like the IP block was blacklisted at some point and got on some bad lists but I don’t want ti limit myself to that theory. I have opened up a ticket with ARIN asking for any guidance. Has anyone ran into this with new space assigned? Any tools, sites, etc. I can use to do further troubleshooting. The IP block does not appear to have any blacklisted IPs according to MX toolbox, and some others.
The block in question is 134.195.44.0/22. It has been RPKI certified and has IRR entries.
One common cause of this issue is entities out there that have very old ‘bogons’ filters in place for the larger block, as an entire /8, /12 to /16 size of space that, many years ago, was unallocated space. Without getting the end point organizations running the httpd, firewalls or whatever to fix their broken configuration, it’s a hard issue to fix from your end.
On a longer term time scale like multiple years, the reachability of an IP block like yours will gradually increase as people with broken services are contacted by additional persons to say “hey, this really is valid ARIN IP space”.
It acts like the IP block was blacklisted at some point and got on
some bad lists but I don’t want ti limit myself to that theory.
I have opened up a ticket with ARIN asking for any guidance. Has
anyone ran into this with new space assigned? Any tools, sites, etc. I
can use to do further troubleshooting.
Is there any specific IP address in the range that should always respond
to ICMP Echo Requests? This will help others see if they can reach you
or not.
We have had this with recent ARIN assignments, too. When we’d get reports from customers, we would reach out to the site admin contacts (either domain WHOIS or IP address WHOIS), explain the situation, and in every case, they were either blocking it because the prefix formerly originated from outside the US, or their GeoLocation database was not updated, in spite of us having contacted all the known GeoLocation providers on the TBW page.
Unfortunately, I've moved on from that job and don't have any of the code that I developed for not69box/69box (and AFAIK, the box itself is long gone), but you can get an idea from the above page what I did. i.e. The two names resolved to an IP in 69.28.64/19 or an IP in 209.208/17. One of the cooler (at least at the time) features was a dual-frame traceroute that visitors could run and watch the box traceroute to a destination from a each of it's IP's, thus showing where in the path their traceroute broke, if it did, from the "69 space".
