I just became aware that AWS has a list of hosting IP providers and that list is blocked by their WAF? (!?!?). None of my VM or colo customers can reach anything in AWS, such as Docker, Twilio, etc. I confirmed through source routing that when I access it using one of my peering partners as a source IP it is reachable, but using one of my net blocks, it is not reachable and times out. Checked all of my routing tables and those AWS blocks are definitely visible. Also confirmed from looking glass that my IP ranges are showing up.
Has anyone else encountered that? If so, is there a way to get removed from that list? I have a very curated list of clients and I know all of them personally and none of them have been abusing AWS, so I was wondering if it was some kind of blanket ban?
If you’re internal to AWS, my ASN is 54380, IP ranges affected are 199.33.244.0/24, 199.79.202.0/24, 199.188.96.0/22, 45.59.144.0/22 and 206.197.110.0/24
You can try opening a case yourself, and hope it gets to someone with a clue. If you don’t have a support contract with them, your chances are almost 0. If you do, your chances are slightly higher, but not by much. most likely they will just tell you to ‘contact the owner of the thing you’re trying to access and have them customize their WAF rules’.
AWS is doing some REALLY dumb things. For example, if your ASN announces a single prefix that a 3rd party provider classifies as ‘hosting provider’ , AWS will flag EVERY prefix from that ASN as ‘hosting provider’, which are all blocked in the default managed WAF rules. They also won’t tell you in any circumstance (even if you’re a customer who is paying for support) who that 3rd party provider IS.
Expect a lot of hassle to get this fixed, if you ever can.
our network is a mix of content and eyeballs and they listed the whole thing. This has prevented the local school district from using their text to speech application (for their deaf students) as well as others. This has also affected the local library and residences.
Endured a similar problem last year. The company I was with migrated prod to Global Accelerator, ALBs, and WAF with Shield Advanced on an accelerated timeline. Several of our customer networks were blocked in AWS's default blocklists.
The path to resolution was meeting with our AWS account executive and their internal WAF team to expedite the removal of the addresses.
Our experience from last month was quite similar to what others have
already described. Based on our tests all the allocations associated
with our ASN were added to "HostingProviderIPList". At first we tried
to reach the AWS by e-mail, but received no response. Once we opened a
ticket(we are both the AWS customer and a Direct Connect Delivery
partner) and explained that our address space is not solely used for
hosting services, then all our networks were whitelisted within few
days. We even had a call with AWS support team and they seemed to be
well aware about false positives in the "HostingProviderIPList". In
other words, we were clearly not a first ISP complaining.
Final update to our ticket from the AWS Support Engineering department
was following:
"I would like to inform you that as per the update I got from our
internal team, the IP Data which we use for our rules is provided by a
third party and changes frequently. Moving ahead I would like to
request you to kindly check your setup and see if you can separate the
network ranges reserved for any hosting type services in your ASN and
provide that to us so that we can check if we can allow the range
which is not used for hosting type services."
It's unfortunate that Amazon continues to offer this service despite
their underlying IP address data having a low level of granularity,
and they are well aware of the problems it causes for address owners.
AWS Web Application Firewall for testing purposes can be build for
example like this:
1) created a VPC("VPC only") with "IPv4 CIDR" 10.0.0.0/24
2) created a subnet 10.0.0.0/28 in one availability zone(for example
"eu-north-1a") and a subnet 10.0.0.16/28 in another availability
zone(for example "eu-north-1b")
3) launched an Ubuntu EC2 instance("t3.micro", free tier eligible).
Auto-assigned the public IP. Allowed SSH and HTTP traffic from
anywhere.
4) created an "Internet gateway" and attached it to VPC
5) added a default route to route table and pointed it to Internet gateway
6) installed(sudo apt update && sudo apt install apache2 -y) a web
server in Ubuntu VM
7) tested with "curl http://51.20.94.186" where 51.20.94.186 is the
public IP address automatically assigned to EC2 instance
8) created a "Target group" for "Instances" type target which is
associated with VPC created earlier. Protocol:Port configuration has
to be HTTP:80.
9) created an Internet-facing "Application Load balancer" and mapped
it with two subnets created previously in two different availability
zones. Load balancer has to forward to target group created in the
previous step.
10) created a WAF and associate it with Application Load Balancer
11) used an AWS managed rule group named "Anonymous IP list" in WAF.
This includes the notorious "HostingProviderIPList" with a default
rule action of "Block".
12) "curl http://test-213161342.eu-north-1.elb.amazonaws.com" returns
"HTTP/1.1 403" from the networks included in the
"HostingProviderIPList". DNS name is seen on the load balancer details
page.