We are starting to look at CGNAT solutions. The primary motivation at the moment is to extend current IPv4 resources, but IPv6 migration is also a factor.
We’ve been in touch with A10. Just wondering if there are some alternative vendors that anyone would recommend. We’d probably be looking at a solution to support 5k to 15k customers and bandwidth up to around 30-40 gig as a starting point. A solution that is as transparent to user experience as possible is a priority.
Thanks
I recommend you to take a look at DANOS.
https://danosproject.atlassian.net/wiki/spaces/DAN/pages/416153601/Carrier+Grade+NAT+CGNAT
I and some good friends here in Brazil had some good experiences with it.
Marcelo Gondin wrote this tutorial in pt_BR, mentioning about a case with:
26Gbps / 1.5Mpps / 11502 simultaneous clients / 192 used Públic IPv4 addresses.
https://wiki.brasilpeeringforum.org/w/CGNAT_Bulk_Port_Allocation_com_DPDK
Not the Cheapest option out there but the most rock solid one I have found is to install the extended service/multi service cards in the BNG and do it locally there. We are currently using both Juniper MX480/960 with MS-MPC cards and Nokia 7750 SR with ISA or ESA cards. Its also well worth running dual stack IPv6 as you can bypass 40%+ traffic from the CGN process for all that CDN traffic.
Why not go whole hog and provide IPv4 as a service? That way you are not waiting for your customers to turn up IPv6 to take the load off your NAT box.
Yes, you can do it dual stack but you have waited so long you may as well miss that step along the deployment path.
Because then a large part of the Internet won’t work…
Why not go whole hog and provide IPv4 as a service? That way you are not waiting for your customers to turn up IPv6 to take the load off your NAT box.
I’m sure the large parts of the world already doing this would disagree.
Hey, look on the bright side: customers won't be able to use Twitter to
complain! 
Ofc, IPv4aaS has many good success stories out there; Sky Italia are
running MAP-T, many, many mobile ISPs are running 464XLAT with great
success.
We're in a situation where making IPv6 a *prerequisite* of your IPv4
connectivity can realistically improve your margins when some sort of
CGNAT gateway is a requirement.
Yes it requires looking at your CPE support, but if you're doing even
00,000's of subs, I'm sure the benefits aren't trivial when viewed
through the lens of the number of connections that a single Chrome tab
can happily chew through.
We are starting to look at CGNAT solutions. The primary motivation at the moment is to extend current IPv4 resources, but IPv6 migration is also a factor.
IPv6 Migration is generally not aided by CGNAT.
In general, the economics today still work out to make purchasing or leasing addresses more favorable than CGNAT.
It’s a bit dated by now, but still very relevant, see Lee Howard’s excellent research presented at the 2012 Rocky
mountain v6 task force meeting:
https://www.rmv6tf.org/wp-content/uploads/2012/11/TCO-of-CGN1.pdf
Owen
While I don't doubt the accuracy of Lee's presentation at the time, at least two base factors have changed since then:
- Greater deployment of IPv6 content (necessitating less CGN capacity per user)
- Increased price of Legacy IP space on the secondary market (changing the formula) -- strictly speaking, this presentation was still in "primary market" era for LACNIC/ARIN/AFRINIC
IPv6 migration is not generally aided by CGNAT, but CGNAT deployment is generally aided by IPv6 deployment; to reiterate the earlier point, any ISPs deploying CGNAT without first deploying IPv6 are burning cash.
- Jima
Hi Steve
We are looking at implementing a similar solution with A10 for CGNAT.
We’ve been in touch with A10. Just wondering if there are some alternative vendors that anyone would recommend. We’d probably be looking at a solution to support 5k to 15k customers and bandwidth up to around 30-40 gig as a starting point. A solution that is as transparent to user experience as possible is a priority.
The numbers below are for a similar target of subscriber’s and peak bandwidth.
We assumed a couple of numbers:
Current Peak Bandwidth = 40G
Remaining IPv4 traffic after migration = 20% (Seen references to 10% or 20% on this forum)
Future Bandwidth Growth = 2x (no data behind this assumption)
Future CGNAT’ed bandwidth = 15Gbps
Equipment & budget lifecycle = 7Yr
Getting that data led us to this price comparison:
That’s provably not true if the IPv4AAS implementation is done carefully.
Owen
IPv4AAS will also work easily for any ISP on the planet.
CGNAT requires IPv4 address space between the CGNAT and the customer CPE which doesn’t overlap with that on the Internet nor that behind the CPE (no you can’t use RFC 1918). 100.64/10 gives you ~4M addresses which fit this criteria but that isn’t enough without reuse for the larger ISPs.
IPv4AAS uses no IPv4 addresses between the B4/NAT64/… and the CPE.
You should also be able to out source IPv4AAS to specialist providers. There is no requirement to run your own hardware. You just populate your DHCPv6/RA/IPV4ONLY.ARPA with the correct information and the traffic will be delivered to the specialist providers. I’m not sure if there are any out there yet but it is a business opportunity for anyone that is already running such boxes.
Mark
While I don't doubt the accuracy of Lee's presentation at the time, at least two base factors have changed since then:
- Greater deployment of IPv6 content (necessitating less CGN capacity per user)
This is only true if the ISP in question is implementing IPv6 along side their CGN deployment and only if they get a significant uptake of IPv6 capability by their end users.
- Increased price of Legacy IP space on the secondary market (changing the formula) -- strictly speaking, this presentation was still in "primary market" era for LACNIC/ARIN/AFRINIC
While that’s true, even at current prices, IPv4 addresses are cheaper to buy and/or lease than CGN.
IPv6 migration is not generally aided by CGNAT, but CGNAT deployment is generally aided by IPv6 deployment; to reiterate the earlier point, any ISPs deploying CGNAT without first deploying IPv6 are burning cash.
Yep.
I still think that implementing CGN is a good way to burn cash vs. the alternatives, but YMMV.
Owen
I did this "economics" exercise for a customer having 25.000.000 customers (DSL, GPON and cellular). Even updating/replacing the CPEs, the cost of 464XLAT deployment was cheaper than CGN or anything else.
Also, if you consider the cost of buying more IPv4 addresses instead of investing that money in CGN, you avoid CGN troubles (like black listening your IPv4 addresses by Sony and others and the consequently operation/management expenses to rotate IPv4 addresses in the CGN, resolve customers problems, etc.), it becomes cheaper than CGN boxes.
It's easy to predict that you will buy now CGN and tomorrow you will need to buy some new IPv4 addresses because that black listening.
Regards,
Jordi
@jordipalet
El 24/2/21 3:13, "NANOG en nombre de Owen DeLong via NANOG" <nanog-bounces+jordi.palet=consulintel.es@nanog.org en nombre de nanog@nanog.org> escribió:
>
> While I don't doubt the accuracy of Lee's presentation at the time, at least two base factors have changed since then:
>
> - Greater deployment of IPv6 content (necessitating less CGN capacity per user)
This is only true if the ISP in question is implementing IPv6 along side their CGN deployment and only if they get a significant uptake of IPv6 capability by their end users.
> - Increased price of Legacy IP space on the secondary market (changing the formula) -- strictly speaking, this presentation was still in "primary market" era for LACNIC/ARIN/AFRINIC
While that’s true, even at current prices, IPv4 addresses are cheaper to buy and/or lease than CGN.
> IPv6 migration is not generally aided by CGNAT, but CGNAT deployment is generally aided by IPv6 deployment; to reiterate the earlier point, any ISPs deploying CGNAT without first deploying IPv6 are burning cash.
Yep.
I still think that implementing CGN is a good way to burn cash vs. the alternatives, but YMMV.
Owen
P.S.: Forking thread from CGNAT.
Hello Jordi!
Since our last heated talk about transitions methods(Rosario, 2018?), I must recognize that the intolerance to other scenarios other than dual-stack had reduced(mostly because of improvements on the applications in generral). I’m even considering the possibility of using 464Xlat on some scenarios.
But I’m still, as it was in 2018, primarily concerned to avoid end-user support tickets.
And I’m still hooked on some specific issues… For example:
For that, with 464Xlat we still keep in pain…
But using DualStack with Good Quality CGNAT, the support tickets statistics are reduced to less than 5%.
So, the question here is:
How not use Dual-Stack and keep the support tickets as low as possible?
P.S.: Forking thread from CGNAT.
Hello Jordi!
Since our last heated talk about transitions methods(Rosario, 2018?), I must recognize that the intolerance to other scenarios other than dual-stack had reduced(mostly because of improvements on the applications in generral). I’m even considering the possibility of using 464Xlat on some scenarios.
But I’m still, as it was in 2018, primarily concerned to avoid end-user support tickets.
And I’m still hooked on some specific issues… For example:
- SIP/Voip Applications, that almost all the providers do not work correctly on when those streams and connections pass over some v6 only paths.
- Applications with some source-based restrictions(some Internet Banking, some Compan-VPNs).
- Games (this is the champion of support tickets).
For that, with 464Xlat we still keep in pain…
Is this pain you have lived or verified with first hand testing?
I am operate 464xlat broadband network, and do not have this pain in the general case. That said, there are cpe specific qa concerns, but that is always the case, regardless of tech
Hi Douglas,
I’ve done a lot of testing in several countries and customer networks and I’ve never got a single failure because 464XLAT.
If anything failed, we tested it with a pure IPv4 network and dual-stack network. They failed as well.
For example, I recall, in a customer deployment, that PlayStation 4 was not working … surprise. It was a specific problem at that specific time, so it was also not working with IPv4-only. Retested a couple of days after, and it worked.
I talk very frequently with other engineers which have also deployed 464XLAT in both cellular and wireline, and I’ve never heard any complain about any specific application or service not working because 464XLAT, so I’m not alone on this.
So, I think experience talk. Probably the question is about the same as you’re indicating “good quality” (whatever, including experience in the matter), makes things work without issues!
Regards,
Jordi
@jordipalet
P.S.: Forking thread from CGNAT.
Hello Jordi!
Since our last heated talk about transitions methods(Rosario, 2018?), I must recognize that the intolerance to other scenarios other than dual-stack had reduced(mostly because of improvements on the applications in generral). I’m even considering the possibility of using 464Xlat on some scenarios.
But I’m still, as it was in 2018, primarily concerned to avoid end-user support tickets.
And I’m still hooked on some specific issues… For example:
SIP/Voip Applications, that almost all the providers do not work correctly on when those streams and connections pass over some v6 only paths.
Applications with some source-based restrictions(some Internet Banking, some Compan-VPNs).
Games (this is the champion of support tickets).
For that, with 464Xlat we still keep in pain…
But using DualStack with Good Quality CGNAT, the support tickets statistics are reduced to less than 5%.
So, the question here is:
How not use Dual-Stack and keep the support tickets as low as possible?
Is this pain you have lived or verified with first hand testing?
Yep! A lot!
LOL gamers can be pretty much insistent…
(haha.jpg + haha-crying.jpg)
And Specifically on SIP/Voip over the Internet, with deep analysis at all the parts involved.
The most common issue is incoming Calls to SIP endpoints behind 464Xlat using IPv4 with unidirectional audio.
And several types of causes:
After hours and hours fighting against the lions, we decided:
“Let’s keep those clients in Dual-Stak and CGNAT” and it just worked.
And after that, the obvious conclusions:
Well then use one of the encapsulating IPv4AAS mechanisms rather than 464XLAT (DS-Lite, MAP-E). They don’t involve translating the payload between IPv4 and IPv6. That said what you are reporting below are implementation bugs. Did you report them to the vendor? Did you install the fix? Rewriting is required as you may have native IPv6 clients rather than clients behind a CLAT on the customer side.
Hello Mark…
Yes, until when I was decided to Fight Agins IPv4, I tried the Fixes.
But after some time, I saw that very little of the problems were due to inadequacies of the ISP’s responsibility equipment.
Most of the difficulties stemmed from:
A) Choices of end-users in their networks.
(Something that the ISP may even try to influence, but that ends up bringing more “childrens” to the support queue, as customers said, “Your company that recommended me to use software X instead of Y, so you have to teach me how to use software X”.)
B) Lack of adequate support for IPv6 by the companies that provided the service on the internet (eGames, IPTV, SIP-VOIP).
After some time beating the dead horse, and mainly seeing that these problems did not happen with Dual-Stack, I decided to do what I was able to do well.
Since 1-2 years ago, things have improved a lot in these two points, pointed out as problems that do not concern the ISP.
Perhaps it is time to review this approach.