Sincere apologies if this e-mail is inappropriate for this audience,
We are (going to be) a startup ISP building a new network from the ground up. I was hoping I could get an opinion, or two, on how everyone feels about 464XLAT. I saw what everyone was saying about it in the 'Android doesn't support DHCPv6' discussion, but what about in the wireline side of things? The main reason we are even considering 464XLAT as opposed to dual-stack (the latter is, in my ignorant opinion, the better option.) is the fear of IPv4 depletion that we think might hit ARIN between now and the start of next year; causing us to pay a premium for IPv4 in the gray market. So I guess the real question here would be: is our fear real, or is it just bug on the wall? If our fear is real, what should we implement so that our users can still get to the v4 internet, are we even thinking soberly by suggesting 464XLAT?
Thanks,
- Nich
Yes, your fears about IPv4 are correct.
If you have a look at ARIN PPML lately, you can see some pretty intense
"discussion" about companies exporting ARIN addresses to CCNIC and so on.
As a greenfield, you should definitely be focused on IPv6-only to the edge
solutions. DS-lite, MAP-E, and 464XLAT come to mind.
DS-lite is the oldest and most common in wireline. 464XLAT is more common
in mobile. MAP-E and MAP-T have not yet been deployed at the same scale as
DS-lite and 464XLAT yet AFAIK, not sure if they will be.
You could also simply do dual-stack with private space and CGN to the end
user using RFC1918 (10.0.0.0/8, 100.64.0.0/10)
Regards,
CB
464XLAT is a abomination that grew from NAT64/DNS64 despite DNS64
not working with DNSSEC. NAT64/DNS64 was pushed as a "short term
solution" as DNS64 cuts off IPv4 access if there is a IPv6 address
for the remote site as the client only asks for AAAA addresses.
In practice the address selection rules move most traffic from IPv4
to IPv6 anyway so there is no need to have DNS64.
Actually , there is no better audience that I know of to ask this
question. And my information might be more marketing related and hardware
skeptical.
My IPv6 direction choice was much easier than yours. You need to figure
out how to build an IPv4 network today from scratch in a world where the
IPv4 bus ride seats have largely assigned.
When we setup our IPv6 ability, I chose to build a native IPv6 network.
Tunneling and translation devices left me wondering about packet flow at
those gateway points. Aside from verbal sales assurances, I still had the
feeling that under loads these devices would break momentarily or cause
latency issues. For web and email services it's not a big issue. Sure
everyone could show me a twitch game playing well or a video conference
call, but what happens when the device is under load or attacked ? Will
service latency be detected by a cleaver well known gamer ? One that
points to the issue as a flaw that makes others think our network is
unusable for all kinds of services ? Overcome issues like "this ISP forces
you to use IPv6" ? The hardware costs can be small compared to consumer
perceptions marketing dollars. So you might position to pitch upfront your
new world Internet service from day one.
European and Comcast has been implementing NAT 6 related things for years.
My son made me move his connection to the smallest bandwidth DSL on ATT
for his games. However, our Comcast has been fine perfectly for watching
Amazon and Netflix streaming (most of the time).
Thank You
Bob Evans
CTO
A network needs users or it is useless. I am curious as to how your native IPv6 network communicated with (if at all) the v4 world. Has anyone confronted you about your network being IPv6? I might have problems with reading comprehension, but in your statement " So you might position to pitch upfront your new world Internet service from day one.", do you mean pitch as in, setup; or pitch as, into the trash.
Thank you,
- Nich Warren
Actually , there is no better audience that I know of to ask this
question. And my information might be more marketing related and hardware
skeptical.My IPv6 direction choice was much easier than yours. You need to figure
out how to build an IPv4 network today from scratch in a world where the
IPv4 bus ride seats have largely assigned.When we setup our IPv6 ability, I chose to build a native IPv6 network.
Tunneling and translation devices left me wondering about packet flow at
those gateway points. Aside from verbal sales assurances, I still had the
feeling that under loads these devices would break momentarily or cause
latency issues. For web and email services it's not a big issue. Sure
everyone could show me a twitch game playing well or a video conference
call, but what happens when the device is under load or attacked ? Will
service latency be detected by a cleaver well known gamer ? One that
points to the issue as a flaw that makes others think our network is
unusable for all kinds of services ? Overcome issues like "this ISP forces
you to use IPv6" ? The hardware costs can be small compared to consumer
perceptions marketing dollars. So you might position to pitch upfront your
new world Internet service from day one.European and Comcast has been implementing NAT 6 related things for years.
My son made me move his connection to the smallest bandwidth DSL on ATT
for his games. However, our Comcast has been fine perfectly for watching
Amazon and Netflix streaming (most of the time).
I have been running across more and more and more people that are
actually doing that - using two isps - one for games and voice, and
the other for streaming and web. I had no idea how prevalent it was.
Still don't, all my evidence is anecdotal (and it all points at
bufferbloat related problems).
Thank You
Bob Evans
CTOSincere apologies if this e-mail is inappropriate for this audience,
We are (going to be) a startup ISP building a new network from the ground
up. I was hoping I could get an opinion, or two, on how everyone feels
about 464XLAT. I saw what everyone was saying about it in the 'Android
doesn't support DHCPv6' discussion, but what about in the wireline side of
things? The main reason we are even considering 464XLAT as opposed to
dual-stack (the latter is, in my ignorant opinion, the better option.) is
the fear of IPv4 depletion that we think might hit ARIN between now and
the start of next year; causing us to pay a premium for IPv4 in the gray
market. So I guess the real question here would be: is our fear real, or
is it just bug on the wall? If our fear is real, what should we implement
so that our users can still get to the v4 internet, are we even thinking
soberly by suggesting 464XLAT?
Thanks,
- Nich
I so would not want to be a new entrant in this market. When last I
tried various ipv4 over ipv6 tunnelling methods (in nicaragua 2009) it
was very hard to get devices that could do it at all. I think common
tools I use now (like openwrt) are really far along on every possible
encapsulation method, but the state of other gear still lagging.
I mean marketing/salesman like pitch. When you have something so new and
familiarity is always the desire of the day by IT managers (hence, all the
cisco only fans), it's better to be upfront and pitch it as new and
improved before others decide to call it something else and choose a
different network.
We began with IPv4. Then many of us members at both ARIN and NANOG all
agreed to push IPv6. I looked at all the methods available and decided we
would build native IPv6 network and give the customer both. Soooo, the
networks are separate from each other and provided to customers on via
separate ports. There is no place in our network where you can hop from
IPv6 to IPv4 and visa versa. The customer can install such gear in their
LAN and make routing those decisions at their end. (Now years later, a
very tiny percentage of customers have link on their IPv6 port.) If anyone
complains, it's the customers choice of gear or routing issues at their
end, as nothing in our network is NATed. Thereby, reducing our potential
service labor costs of dealing with a customers understanding of trace
routes in NAT space - and other similar issues that they try to make your
staff's problem.
Thank You
Bob Evans
CTO
Sincere apologies if this e-mail is inappropriate for this audience,
Hi Nich,
Looks like the correct audience to me.
We are (going to be) a startup ISP building a new network from the
ground up. [...] The main reason we are even considering 464XLAT
as opposed to dual-stack (the latter is, in my ignorant opinion, the
better option.) is the fear of IPv4 depletion that we think might hit
ARIN between now and the start of next year; causing us to pay a
premium for IPv4 in the gray market.
Your customers will require end-to-end IPv4 for the foreseeable future.
464XLAT can provide natted IPv4 using an internal IPv6 infrastructure
in special circumstances. Specifically: you must have sufficient
control of the customer equipment to compel it to employ 464XLAT to
provide IPv4 services to the customer. If your customers lease phones
from you and your phone vendors build in 464XLAT support, T-Mobile has
demonstrated that this is practical. If your customers bring generic
Macs and PCs with the odd Linux user in the mix (their equipment, not
yours), you may be asking for extensive support headaches with
464XLAT.
Dual stack with carrier NAT would also handle your IPv4 needs. You'll
have an additional expense maintaining both protocols within your
infrastructure. Nevertheless, this approach alleviates the need to
control the customer premises equipment.
Regardless of your approach, DS+NAT or 464XLAT, you will require a
comparable number of global IPv4 addresses. Neither technology
eliminates your need for IPv4 addresses facing the public Internet.
Regardless of your approach, you will need to make provisions to
support customers who require a global and/or static IPv4 address
without NAT. It need not be part of your basic package, but if it's
unavailable at any price you can be sure of getting a PR black eye at
some point.
Regards,
Bill Herrin
I figured that duel-stack would be the way to go, but I worry that ARIN might not give us space for duel stack out of their reserved pool (https://www.arin.net/policy/nrpm.html#four10), and that this .13 of a /8 won't make it to next year. I suppose that would be a question for the ARIN mailing list?
Thank you,
- Nich Warren
I figured that duel-stack would be the way to go, but I worry that ARIN might not give us space for duel stack out of their reserved pool (https://www.arin.net/policy/nrpm.html#four10), and that this .13 of a /8 won't make it to next year.
Hi Nich,
The absolute most you can get out of that pool is a /24. The same
amount as a regular allocation at current gray market prices would
only cost $2500-$5000 (see e.g. http://www.ipv4auctions.com/). There
have been enough directed transfers at this point to expect the
process to go smoothly, though you might want to contract someone to
help walk you through it if you haven't extensively worked with ARIN
in the past. There are some fiddly bits with needs assessments and the
like where you want to make sure the words you're saying are the ones
ARIN staff expects to hear. It can be tough to walk it back if you say
the wrong thing the first time.
From reading the ARIN PPML, I understand that the 4.10 pool is largely
untouched due to confusion (staff and registrant) on what does and
doesn't qualify. The regular free pool (not the 4.10 pool) is on its
last dregs and will not likely have any addresses come January.
Regards,
Bill Herrin
I am thinking now that our best option would be to go duel-stack lite (RFC6333), after reading what you fellows have to say about 464XLAT. I feel as though I should add that our peer networks (one was started at the end of 2013) are implementing IPv4 only networks; they are pressuring management into thinking that IPv6 is too experimental to deploy, and that IPv4 (only) is the only way to go.
Thank you,
- Nich Warren
Hi,
The price for IPv4 is about $10 per address. I do not expect that to become
much more expensive in the short term, especially not in the Arin region
where there is such abundance of allocated address space that could be
freed for a quick dime.
So is $10 one time fee for new users too much?
We are a young network in the RIPE region and we have had to buy most of
our address space. We had the first 1024 for free as that is the RIPE
policy. Our strategy is to buy what we need, but use the space as efficient
as possible. Users are on a shared subnet (a /32 per user) instead of a /30
per user etc.
I believe the other sane option is dual stack with carrier nat. You can
sell non-NAT access at a premium.
We decided against the NAT option because it is seen as inferior by the
users and because it is not free either. Instead of spending money on
buying address space, you will spend it on NAT servers. You will also have
more fun with denial of service attacks killing your NAT server etc.
The high tech solution is stuff like MAP where you move the cost out to the
CPE. But then you need to control the CPE - if you have that then great.
You would still want to sell a non-NAT (and MAP is NAT) to users that
require a public IPv4 address, so you still need to go dual stack or use
some tunnelling for that.
That's my 2c.
Regards,
Baldur
* Baldur Norddahl <baldur.norddahl@gmail.com>
The high tech solution is stuff like MAP where you move the cost out
to the CPE. But then you need to control the CPE - if you have that
then great. You would still want to sell a non-NAT (and MAP is NAT)
to users that require a public IPv4 address, so you still need to go
dual stack or use some tunnelling for that.
Hi Baldur,
MAP is *not* NAT; that's what's so neat about it. The users do get a
public IPv4 address (or prefix!) routed to their CPE's WAN interface,
towards which they can accept inbound unsolicited connections.
The public IPv4 address could be port-restricted if the operator wants
address sharing, but it does not have to be. You could do both at the
same time, e.g., giving your "premium" users a /32 or /28, while the
standard subscription includes a /32 with 4k ports.
I will grant you that MAP-T performs NAT (i.e., protocol translation)
internally, but the translations that happens when a packet enters the
MAP domain are reversed when it exits. So the IPv4 addresses are
transparent end-to-end.
MAP-E (and lw4o6 for that matter), on the other hand, has no form of
NAT anywhere. (Unless you count the NAPT44 that sits between the
subscriber's RFC1918 LAN segment and the CPE's WAN interface, but
that's not exactly something that's unique to MAP.)
Nicholas: If I were you, before going down the 464XLAT route, I'd first
look closely at these technologies, in the order given:
1) MAP (because it is fully stateless)
2) lw4o6 (because it is mostly stateless, i.e., no session tracking)
3) DS-Lite (which, like 464XLAT, is stateful, but you'll have way more
CPEs to choose from than with 464XLAT, which is mostly for mobile)
Tore
True if you are only doing MAP because you do not like pesky IPv4 packets
in your backbone (ie. do not like dual stack backbone).
But for us that are in the "have to buy IPv4 addresses" boat, the
interesting thing about MAP is that it can be used instead of carrier NAT.
You will have multiple users sharing the same IP address. Each user has a
port range routed to him. While he does get the public IP directly on his
CPE, he is restricted from using it freely. He will not be able to run ssh
on port 22 or a webserver on port 80/443.
In this sense it is carrier NAT implemented on the CPEs. And with it comes
some of the evil of carrier NAT.
If I ever go down the carrier NAT route I would like a MAP solution. It is
clever. The only problem is that I do not know of any equipment that will
actually do MAP (besides possible Cisco which is outside my price range).
The RFC is not even done yet.
Regards,
Baldur