Gi Firewall for mobile subscribers

Hello NANOG,

We are discussing internally and wanted to get more opinions and especially more data on what are people actually doing.

We are running an ISP network with about 150K fixed broadband users, running dual stack (IPv4 behind CGNAT).

On the ISP network IPv6 is simply routed, and is firewalled on the CPE.

This network added mobile services about a year ago, also dual stack (we have no control on the mobile devices so we were too concerned to choose IPv6 only access).

We have an ongoing discussion about Gi firewall (adding a firewall between the subscribers and the internet, allowing only subscriber initiated connections), for the IPv6 traffic.

The firewall is doing very little security, the ruleset is very basic, allowing anything from subscribers to the internet and blocking all traffic from the internet towards the subscribers.

We have a few rules to limit the number of connections per subscriber (to a relatively high number) and that is it.

One of the arguments in favor of having the firewall is that unsolicited traffic from the internet can “wake” idle mobile devices, and create signaling (paging) storms as well as drain user batteries.

On the other hand, allowing only subscriber initiated traffic is mostly achievable using ACLs on the mobile core facing routers, or is it with the growing percentage of UDP traffic ?

BTW – I don’t mention IPv4 traffic on the mobile network as it’s all behind CGNAT which don’t allow internet initiated connections.

Anyway, we are very interested to know hear more opinions, and especially to hear what are other mobile operators do.

Regards

Amos

Hello NANOG,

We are discussing internally and wanted to get more opinions and especially more data on what are people actually doing.

We are running an ISP network with about 150K fixed broadband users, running dual stack (IPv4 behind CGNAT).

On the ISP network IPv6 is simply routed, and is firewalled on the CPE.

This network added mobile services about a year ago, also dual stack (we have no control on the mobile devices so we were too concerned to choose IPv6 only access).

We have an ongoing discussion about Gi firewall (adding a firewall between the subscribers and the internet, allowing only subscriber initiated connections), for the IPv6 traffic.

The firewall is doing very little security, the ruleset is very basic, allowing anything from subscribers to the internet and blocking all traffic from the internet towards the subscribers.

We have a few rules to limit the number of connections per subscriber (to a relatively high number) and that is it.

One of the arguments in favor of having the firewall is that unsolicited traffic from the internet can “wake” idle mobile devices, and create signaling (paging) storms as well as drain user batteries.

On the other hand, allowing only subscriber initiated traffic is mostly achievable using ACLs on the mobile core facing routers, or is it with the growing percentage of UDP traffic ?

BTW – I don’t mention IPv4 traffic on the mobile network as it’s all behind CGNAT which don’t allow internet initiated connections.

Anyway, we are very interested to know hear more opinions, and especially to hear what are other mobile operators do.

Regards

Amos

Step outside the theoretical and model your real threats. Attack yourself of pay someone to do a real pentest.

  1. Does a hacker know the ipv6 of your subs? How frequently does the sub get a new 128 bit address?

  2. What does the hacker get from a paging storm? Economic benefit ? Lolz? Has a malicious paging storm ever happened in the real world? What level of effort would be required to trigger that? Is that level of effort more or less than it would take to tip over a stateful firewall (session exhaustion, pps attack, alg bugs, vulns in the fw
    https://www.zdnet.com/article/cisco-removed-its-seventh-backdoor-account-this-year-and-thats-a-good-thing/
    )

  3. Assuming the hacker gleans the address of the sub, what ports are open in the real world? What can a hacker connect to and accomplish?

I think the traffic Amos is referring to is random traffic hitting the devices causing them to “wake up”. Everyone here knows a simple dump on port 22 will show traffic. We have a /22 that gets an avg of 1-2 mbit of random traffic (mainly 22 and 3389).

I think the traffic Amos is referring to is random traffic hitting the devices causing them to “wake up”. Everyone here knows a simple dump on port 22 will show traffic. We have a /22 that gets an avg of 1-2 mbit of random traffic (mainly 22 and 3389).

I believe he was talking about ipv6.

Does this backscatter happen in ipv6 given how impractical scanning ipv6 is ?

I don’t v6 stats yet but it would be interesting to see. I did a tcpdump on one v6 IP and saw hundreds of requests to port 25.

In a previous job we did have a stateful Gi firewall and experienced
first hand what backscatter does to the radio network. By accident we
allowed icmp from the Internet to the subcribers and paging went up by
30%. We all agree that the average amount of backscatter on IPv6 is much
less than what we see in IPv4. However active IPv6 adresses are exposed
(for instance on IRC!) and will be targeted by attackers. Also half-open
TCP sessions can be very long running - for instance a mobile goes
offline while downloading a file. Some webservers will keep trying to
send data for a long time, and having a stateful device with agressive
timeouts on half open sessions will definately reduce paging

Also keep in mind that most GGSN/PGW will assign a /64 (and not a /128)
so if someone does a scan targeting that specific /64 you might see a
lot of traffic to the device. I would strongly suggest deploying a
stateful device - purely to protect the radio and signaling network -
not the terminal/phone

- Jan

We have an ongoing discussion about Gi firewall (adding a firewall between the subscribers and the internet, allowing only subscriber initiated connections), for the IPv6 traffic.

The firewall is doing very little security, the ruleset is very basic, allowing anything from subscribers to the internet and blocking all traffic from the internet towards the subscribers.

We have a few rules to limit the number of connections per subscriber (to a relatively high number) and that is it.

What would be the process for a subscriber who wishes to allow inbound connections?

If you are simply saying that as a customer of your ISP you simply can’t allow inbound IPv6 connections at all, then you are becoming a very poor substitute for an ISP IMHO.

One of the arguments in favor of having the firewall is that unsolicited traffic from the internet can “wake” idle mobile devices, and create signaling (paging) storms as well as drain user batteries.

There are lots of ways to configure alerts and reduce this problem space. If you want to provide a checkbox on the my.t-mobile page for the user to turn this firewall on or off on a per device basis, then sure, I could see that as viable. Even if it annoyingly defaults to on.

On the other hand, allowing only subscriber initiated traffic is mostly achievable using ACLs on the mobile core facing routers, or is it with the growing percentage of UDP traffic ?

Is it even desirable to allow only subscriber initiated traffic?

Case in point, I will occasionally end up tethering my laptop (mobile hot spot) and want certain authorized individuals to be able to VNC into it via that tethering connection.

There have been other times when I’ve had things on the other side of a tether that I wanted to ssh into.

There are also things like Particle IONs where it is desirable to be able to push firmware updates OTA. I realize that Particle is sadly lagging on IPv6 support, but it will, hopefully, one day become a valid use case as well.

BTW – I don’t mention IPv4 traffic on the mobile network as it’s all behind CGNAT which don’t allow internet initiated connections.

Yes, but IPv6 is supposed to hope us recover from this travesty.

Anyway, we are very interested to know hear more opinions, and especially to hear what are other mobile operators do.

As is tradition, most operators screw the customer in one way or another in this regard. Some haven’t thought about screwing customers in this particular way in IPv6 yet and so IPv6 sometimes works as one would hope.

Owen

Owen,

Let me clarify a few points:

  1. I am in favor of end to end connectivity and IPv6 can help restore this.

  2. In the fixed broadband portion of the network this is the case.
    IPv6 is routed to the subscriber CPE.
    Firewall on the CPE is turned on by default, but can be turned off by the user.

  3. In the mobile portion life are a bit more complicated.
    Unsolicited traffic from the internet towards an idle subscriber triggers a signaling process called paging.
    Extra paging is expensive in terms of signaling resource utilization, as well as on device battery.

Amos

I agree with Owen that an always-on firewall which blocks all inbound traffic would be very frustrating to some users. I do understand your need as a provider to prevent expensive signaling operations. I think the suggestion of a toggle in a web portal to disable the firewall is a good compromise. Most users will not need to make inbound connections to their cellular devices, so the impact to your network would likely be small.

Also keep in mind that most GGSN/PGW will assign a /64 (and not a /128)

All 3GPP devices assign /64 per bearer because that's what's in the 3GPP spec. I've been told 3GPP went to IETF and asked what to do, IETF said "assign /64 per device" and that's what ended up in the specs.

so if someone does a scan targeting that specific /64 you might see a lot of traffic to the device. I would strongly suggest deploying a stateful device - purely to protect the radio and signaling network - not the terminal/phone

If they scan the /64 then this won't cause excessive paging traffic as the device will already be out of low power mode.

The balanced solution is to have a stateful device that typically does nothing but has some kind of "abuse detection" which triggers filtering certain Internet sources when it decides that this device is performing scans of larger IP spaces. This protects the mobile network from paging storms but also allows users to be reachable from the Internet.

* Owen DeLong

What would be the process for a subscriber who wishes to allow inbound connections?

If you are simply saying that as a customer of your ISP you simply can’t allow inbound IPv6 connections at all, then you are becoming a very poor substitute for an ISP IMHO.

I have to agree with this.

We've been wanting to replace our all of our ad-hoc OOB links with a
standardised setup based on LTE connectivity to an embedded
login/console server at each PoP. IPv6 would be perfect due to no
CGNAT and infinitesimal levels of background scanning.

Unfortunately Telenor has decided to deploy a central firewall that
drops all inbound connections, making their service totally unusable
for our use case. I guess they don't want our money.

Maybe with EU RLAH I could simply find another more suitable provider
abroad. Maybe I'd even get vPLMN redundancy that way. Hmm...

Tore

Also keep in mind that most GGSN/PGW will assign a /64 (and not a /128)

All 3GPP devices assign /64 per bearer because that's what's in the 3GPP spec. I've been told 3GPP went to IETF and asked what to do, IETF said "assign /64 per device" and that's what ended up in the specs.

so if someone does a scan targeting that specific /64 you might see a lot of traffic to the device. I would strongly suggest deploying a stateful device - purely to protect the radio and signaling network - not the terminal/phone

If they scan the /64 then this won't cause excessive paging traffic as the device will already be out of low power mode.

If they scan the entire /64, I’ll be impressed.

Let’s assume a maximum packet rate of 10,000 packets per second.

A /64 contains 18,446,744,073,709,551,616 addresses.

If we ping continuously and only count one of the two packets required for each ping attempt, that’s 184,467,440,737,096
seconds. Putting this in perspective, that’s 3,074,457,345,619 minutes or 51,240,955,761 hours or 2,135,039,824 days
or 5,849,424 years.

I’m pretty sure that no matter how good your power management is, any cell phone’s battery will die long before its /64 can be scanned.

The balanced solution is to have a stateful device that typically does nothing but has some kind of "abuse detection" which triggers filtering certain Internet sources when it decides that this device is performing scans of larger IP spaces. This protects the mobile network from paging storms but also allows users to be reachable from the Internet.

+1

Owen

Owen,

Let me clarify a few points:

  1. I am in favor of end to end connectivity and IPv6 can help restore this.

  2. In the fixed broadband portion of the network this is the case.
    IPv6 is routed to the subscriber CPE.
    Firewall on the CPE is turned on by default, but can be turned off by the user.

  3. In the mobile portion life are a bit more complicated.
    Unsolicited traffic from the internet towards an idle subscriber triggers a signaling process called paging.
    Extra paging is expensive in terms of signaling resource utilization, as well as on device battery.

That’s a problem I leave for the developers of the platform to improve/solve in the design of 5G or subsequent protocols.

It should not be worked around by permanently and irrevocably disabling end user functionality.

Owen

And that might be the point of the scan - not to find the addresses in use, but to deplete the battery. That would have the effect of service denial.

Why bother scanning in that case vs. just repeatedly hammering the same address?

Owen

Sounds like the console server will need to "phone home". That a workaround might be possible doesn't make a firewall which the user cannot control to some degree less annoying. Though it might be that Telenor just needs to be notified/reminded that power users and business customers exist.

/mark

* Mark Milhollan