202203071610.AYC Re: Making Use of 240/4 NetBlock

It doesn’t take any OS upgrades for “getting everything to work on
IPv6”. All the OS’s and routers have supported IPv6 for more than a
decade.

There are lots of vendors, both inside and outside the networking space, that have consistently released products with non-existant or broken IPv6 implementations. That includes smaller startups, as well as very big names. An affirmative choice is often made to make sure v4 works , get the thing out the door, and deal with v6 later, or if a big client complains.

To be completely fair, some of those vendors also mess up IPv4 implementations as well, but in my experience , v4 stuff is more often ‘vanilla’ coding issues, whereas v6 mistakes tend to be more basic functional errors, like handling leading zeros correctly.

In about two years time, IPv4 addresses will be worth on the order of $100/IP, assuming current trends hold.

That's a lot of revenue in leasing IPv4 to the business customers that refuse to think about IPv6 because $reason.

It's also a lot of unit cost to add to a consumer-grade service, where your margins are distastefully thin already (well, in some markets) and set to get thinner when you need to buy a swathe of CGNAT boxes to keep routing IPv4.

Even at todays's dollar price, this dilemma holds true, but I largely suspect that there are too few fixed-line ISPs that have been forced into CGNAT yet - the more that are, the more will wonder why they're buying so many of them.

This a thousand times. Don't believe the claims of IPv6
support until you have fully tested it. Almost no vendor is including
any IPv6 testing in their QA process and nobody is including it in any
of their support staff training. Their labs may not even have v6
capability. Some of our biggest vendors who have supposedly supported
v6 for over a decade have rudimentary, show-stopping bugs. The support
staff at these vendors have often never even seen a customer using v6,
and they have no idea what it looks like on their own gear.

  A subset of these vendors will listen to you and fix the
problems. Give them your support and loyalty. I want to name names so
bad...

--TimH

Tim,

Some of our biggest vendors who have supposedly supported
v6 for over a decade have rudimentary, show-stopping bugs.

Not disagreeing (and not picking on you), but despite hearing this with some frequency, I haven’t seen much data to corroborate these sorts of statements.

  A subset of these vendors will listen to you and fix the
problems. Give them your support and loyalty. I want to name names so
bad…

Perhaps the right approach would be similar for network operators to move to a timed full disclosure model (like Google’s Project Zero for security issues)? In the software security world, this model seems to have had a positive impact in getting fixes out. If a vendor who claims v6 support doesn’t actually support v6 (or, if a vendor fixes a known lack of v6 support), it would seem to me that this is information that folks on NANOG (and elsewhere) would find useful.

Regards,
-drc

Sounds like an excellent reason not to try to use it for global unicast.

Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

So basically no solution is offered to what is the showstopper for this proposal, only a hand wave that it "should be" easy to fix (but that's everyone else's problem). I mean, I believe this has been discussed to death many times over in the past and yet here we still are.

When did squatting become a justification for not allocating addresses?

Regards,
-drc

It appears that David Conrad <drc@virtualized.org> said:

-=-=-=-=-=-

Major networks are already squatting on the space internally, because they tried it and it works.

Sounds like an excellent reason not to try to use it for global unicast.

When did squatting become a justification for not allocating addresses?

Um, when can I register my .corp and .home domains?

R's,
John

Isn't this essentially the same thing as the DNS name collision problem
ICANN has been studying and discussing? Perhaps scale and potential
for harm is different, but I think the essence of the challenge is
essentially the same. In the name space we are unlikely to see
.home (for example) allocated any time soon if ever thanks to what some
might describe as a form of squatting. I would look to that work for
some perspective on when or why something like this might, or might not
be justified.

  <ICANN Publishes Name Collision Analysis Project (NCAP) Study 2 Documents>

John

Hi Seth,

AFAICT, the core of Abraham's proposal is to deploy 240/4 as an
addition to RFC1918 space, to be used as folks' equipment permits.
Activity beyond that (associated with IoT) appears to have no
inter-domain application that need fall within the standards
development space at this time.

Would you care to articulate the showstopper problem you see for the
standards-relevant portion of the proposal?

Regards,
Bill Herrin

Fine. We could start at the top, with protocols that are defective
by design, such as OSPFv3, which lack built-in authentication and
rely on IPsec. That's great if you have a system where this is all
tightly and neatly integrated, but smaller scale networks may be
built on Linux or BSD platforms, and this can quickly turn into a
trainwreck of loosely cooperating but separate subsystems, maintaining
IPsec with one set of tools and the routing with another.

Or ... FreeBSD's firewall has a DEFAULT_TO_DENY option for IPv4 but
not for IPv6. Perhaps not a show-stopping bug, granted. But, wait,
if you really want end-to-end IPv6 (without something like NAT in
between doing its "faux-firewalling") endpoints, wouldn't you really
want a firewall that defaults to deny, just in case something went
awry? If I've got a gateway host that normally does stateful
firewalling but it fails to load due to a typo, I'd really like
it to die horribly not packet forwarding anything, because someone
will then notice that. But if it fails open, that's pretty awful
because it may not be noticed for months or years. So that's a
show-stopper.

As exciting as it would be to go all-in on v6, it's already quite a
bit of a challenge to build everything dual-stack and get to feature
parity. The gratuitous differences feel like arrogant protocol
developers who know what's best for you and are going to make you
comply with their idea of how the world should work, complexity be
damned.

I really never thought it'd be 2022 and my networks would be still
heavily v4. Mind boggling.

... JG

My team ran into a bug in Cisco IOS-XR a few years ago wherein IPv6 connected BGP had problems when using on-link IPs that it did not have when using off-link IPs. E.g. on-link (LL or GUA) vs off-link (GUA on loopback / different interface).

Heh :slight_smile: This is kind of funny for me since I am usually far
more likely to be accused of not being able to shut up about it. I
don't like to throw vendors under the bus publicly, but I will instead
name some positive actors whose efforts I appreciate:
  Juniper fixed their DHCPv6 relay server when I was able to show
them it was not working correctly; they have also had other bugs that
were DHCPv6 related and provided work-arounds and fixes (they still
have a few bugs that appear to be the result of incomplete fixes for
related issues). I'm still not sure if they ever fixed the fact that
SRX routers can't dynamically add the default v6 route (I still had to
hard code this last time I tested on SRX320; the bug is documented).
  Of course, it still feels like not-that-long-ago when Juniper
wanted me to buy a special license for each switch that I wanted to use
OSPFv3 on. That's no longer a thing, but it made me pretty mad at the
time.

  ZyXel has been very responsive to adding support, fixing
functionality, and working with me on in-depth and long term testing of
issues. I'm not sure I have ever worked with a vendor that was so
responsive to my requests and reports. This is in sharp contrast to
one of their competitors who basically told me they have no interest in
adding the IPv6 support they had previously told me they could add.

  Adtran is the only vendor I know of that has DHCPv6 option-18
support in their gpon/xgspon OLT cards. They have also fixed v6 DNS
issues that we have reported on their 5000 chassis. If it weren't for
their OLTs I would probably not know what "working" looks like. They
are also the only vendor that has ever handed me a Broadcom based CPE
to test that worked out of the box. Sadly, I have problems with most
of their current gen; the 515ac is a bright spot - someone important
must be buying these who insisted on v6 support. There are still some
minor issues with MAC tables on the OLTs that have only revealed
themselves over time, but nothing show-stopping (just requires minor
manual intervention in some cases).

  The ISC's Kea DHCP server is a must have for us; this org
deserves more recognition and support from the community. We use it
for prefix delegation and option-18 (basically all dhcpv6 is relayed
there).

  I have still not seen an ACS that has equiv v6 support, but I
will say that both Adtran and Calix appear to be making /very recent/
efforts and have shown progress in this regard.

  Ubiquiti Edge routers have pretty decent IPv6 support at the
CLI, but the web GUIs expose little to none of it.

  Of course shout out to Microsoft Windows for great support
going back a long way; I wouldn't use it, but the efforts help us all.
The OpenBSD team are also rock stars.

--TimH

I am going to attend the WISPA conference in New Orleans next week.
(anyone going?). I don't see any topics related to ipv6 there, nor as
requirements for broadband grants.

I first tried to deploy ipv6 at my wisp 14 years ago, and failed
utterly. Since then, I've kept track of that market, and most of the
ipv6 support from the vendors that serve it, has been faltering. In
particular, I've been working on improving mikrotik's fq_codel and
cake support only to be completely stymied by several bugs in their
ipv6 implementation. ( TTI Vanguard: Dave Taht, CEO, Bufferbloat.net; March 2022 - YouTube ).

This week, I've been trying to get a cerowrt box with a hurricane ipv6
tunnel upgraded to modern openwrt, but am running into issues with the
default deny firewall.

Cisco Meraki only recently got a first release of ipv6 out the door.

Tomorrow I hope to take a look at a new ubnt 60ghz outdoor radio,
which I'm told "HAS WORKING IPV6!!", and hoping for the best, because
it has a SDN stack I have no visibility into.

John,

John,

It's not just equipment vendors, it's ISPs. Here in Oregon, Frontier was recently acquired by Ziply. They're doing massive infrastructure work and recently started offering symmetrical gigabit FTTH. This is a brand new greenfield PON deployment. No IPv6. It took being transferred three times to reach a person who even knew what it was.

Likewise the Wave Broadband cable operator. No IPv6, no plans for it.

Um, are you suggesting there is sufficiently heavy use of 240/4 to result in a significant security/stability issue if the address space is allocated? I thought you were arguing too many systems would have to be updated to even send/receive packets with 240/4 in the source or destination field.

Both, actually. A few messages back someone said a cloud provider was using 240/4 for private address space, which is easy to believe since they have their own versions of the system software they run, e.g., Amazon's own distro of linux for use within AWS.

So first you have to figure out all of the random computers that need network stack upgrades, and then, oh, you can't talk to Google or whoever. At this point, either buying a chunk of real IPv4 space or getting IPv6 working looks quite attractive.

Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

It appears that David Conrad <drc@virtualized.org> said:

isn’t very far), 240/4 isn’t sourcing or sinking significant traffic on the Internet.

FWIW, my tiny server sees about 20 packets/day from that range. It's not very much but it's
hard to imagine why I'm seeing any at all.

It's more than I see from 0/8, less than what I see from 192.168.0.0/16.

R's,
John

It's not just equipment vendors, it's ISPs.

I completely agree.

I get why line of business applications; e.g. billing, provisioning, repair, haven't been updated to support IPv6.

But I believe that any network equipment vendor that is (or has been for the last 1-2 decades) selling /new/ equipment really has no excuse for not IPv6 not having feature parity with IPv4.

Here in Oregon, Frontier was recently acquired by Ziply. They're doing massive infrastructure work and recently started offering symmetrical gigabit FTTH. This is a brand new greenfield PON deployment. No IPv6. It took being transferred three times to reach a person who even knew what it was.

I've had similar lack of success with my municipal GPON provider. At least the people answering support tickets know what IPv6 is and know that it's on their future list without even being in planing / testing phase.

Likewise the Wave Broadband cable operator. No IPv6, no plans for it.

....