Some advice on IPv6 planning and ARIN request, please

Mailman List Mirror (Read Only)
1

Hello,

If anyone out there could provide some input or advice on how to best
handle our upcoming leap into IPv6, it would be much appreciated. I want to
make sure we're playing nicely and not causing anyone any unnecessary grief
before we deploy. We're currently in the planning stage and can make
whatever changes we need to.

Situation:

We're an end-user org and qualify for a /40 assignment because we operate
over 60 sites and some of those are/will be multihomed. We manage hotels in
Canada only, but from coast to coast to coast and everywhere in between.
Our corporate network and org structure is optimized for three regions. We
also have, and continue to grow into, cloud infrastructure and foresee
wanting to bring our own addresses (.e.g., to AWS VPC when that option
becomes available). As such, an obvious design strategy would be to break
the /40 into 4 x /42's. However, due to an imbalance in national site
distribution, 50% of our sites are located in one region (Region A).
Additionally, historical and forecasted growth indicates that it's
perfectly reasonable for us to expect growth of an additional 16 sites in
that same region over the next 3-5 years.

We would prefer to summarize at the /42 level, announced from our last-mile
providers. There are 3 primary last-mile providers so this strategy would
help significantly reduce the number of global routes being injected. If we
split regions evenly at /42 and if we follow the /48-per-site best practice
(which I believe is justifiable in our situation - see below), Region A
will be at 50% usage immediately. Adding 16 more sites brings it to 75%
usage in only a few years. The other regions would be at ~33% usage (Region
B) and 15% usage (Region C) and will see moderate growth in 3-5 years.
Cloud will initially be at 2-4% usage (Region D) but will also grow quickly
within 3-5 years.

Ideal situation: ARIN assigns us a /36 and we don't need to worry about
re-addressing. Even if they can offer us contiguous space with a second
request to increase our assignment, we would likely need to re-address a
significant portion of our sites which would be painful and time-consuming.
Less ideal situation #1: Split the first level of subnets unevenly at 1 x
/41 and 2 x /42 and hope we can carve out some of that space for use in our
cloud infrastructure. This strategy would solve our Region A problem and
would keep Regions B and C from going to 68% and 34% utilization
immediately but it would mess up Region D and impact Regions B and C.
Less ideal situation #2: Split the first level of subnets unevenly at 1 x
/41, 1 x /42, and 2 x /43. This strategy would solve our Region A and
Region B problems but would constrain Region C and Region D future growth
options somewhat.
Less ideal situation #3: Drop the /48 per site default to somewhere between
a /49 and /53 and hope we don't bust out of those. This strategy would
allow us to keep top-level aggregation at /42's but would move the site
assignments off the nibble boundaries.
Less ideal situation #4: Keep 4 x /42's and hope we don't expand out of
them in Region A. This strategy would imply we don't wish for our business
to grow and is pretty risky.

I feel the /48 site default is justifiable because of the various
applications and services that are currently, or could likely be offered at
hotels. E.g., each room gets a /64 for all guest-internet devices
registered to that room. + IoT could result in the same rule (each room
gets a /64) or, perhaps a bit simpler, each class of IoT device is on a /64
or each IoT vendor is on a /64. There will also be new applications that
come out with similar possible needs. With some of our hotels in the
500-1000 room range, we're looking at as many as 2000 /64's per site in the
next 5 or so years. Plus backoffice/admin subnets.

I think the ideal situation is out as ARIN policy wouldn't allow them to
assign us a /36 at this time. Unless someone knows something that can help
us here.

Assuming we can't get a /36, my feeling is that less ideal situation #2 is
better than #3 is better than #1 is better than #4, assuming we're
following the following design best-practices:

a) assign top-level aggregations evenly (which we'd be breaking a bit with
option #2)
b) reduce global routes as much as possible
c) stay on the nibble boundary as much as possible
d) default to /48 per site

Any thoughts or advice would be much appreciated.

Thanks in advance,
Oliver

2

60 sites? Just ask for a /32.

/kc

3

We would prefer to summarize at the /42 level, announced from our last-mile
providers. There are 3 primary last-mile providers so this strategy would
help significantly reduce the number of global routes being injected. If we
split regions evenly at /42 and if we follow the /48-per-site best practice
(which I believe is justifiable in our situation - see below), Region A
will be at 50% usage immediately. Adding 16 more sites brings it to 75%
usage in only a few years. The other regions would be at ~33% usage (Region
B) and 15% usage (Region C) and will see moderate growth in 3-5 years.
Cloud will initially be at 2-4% usage (Region D) but will also grow quickly
within 3-5 years.

If you're backhauling each region (even effectively via your upstream), I'd take a look/listen to these two slides: https://www.youtube.com/watch?v=rWJZfShWE6g&t=12m46s (Honestly, that entire video is worth watching if you're preparing to make your initial IPv6 PI space request -- it's a very informative presentation, and is fairly authoritative.)

Net-net, if "hub 1" is supporting 30-ish sites, with projected growth to 46-ish, you could possibly make the case for allocating a /40 per hub, and a /38 (or maybe even /36) overall. (There's only one /38 assignment in ARIN region, FWIW.)

I feel the /48 site default is justifiable because of the various
applications and services that are currently, or could likely be offered at
hotels.

If it's a site, /48 is justified as per ARIN requirements, period.

I think the ideal situation is out as ARIN policy wouldn't allow them to
assign us a /36 at this time. Unless someone knows something that can help
us here.

Might. I'd file the request, as long as you have a logical addressing plan to justify it.

      Jima

4

Thanks, Jima. I'll review the slides.

Without complicating the issue, we're trying to address a number of
challenges at the same time. There's no regional backhauling at this time.
Each site will be reachable via the internal network but will also
independently announce it's assignment to its ISP(s). There are many
reasons for this model, some of which I like and others I don't! We do plan
to coordinate address assignments/aggregations with the ISPs to reduce
global routes and unwanted conflicts/overlap.Unfortunately, there's no real
hub in each region and the ISPs are not region-specific. We inherit a bunch
of stuff and then need to find a way to jam it into something that isn't
completely broken... we've done a lot of cleanup and re-org of services but
there's still a long way to go. IPv6 should help us get there, however.

Agreed with the /48 but ARIN doesn't appear to agree with our justification
for a /36 thus far.

5

Hi Oliver,

I second Ken's notion. You're trying to be an ISP under the end-user rules.
However transient, your users are mostly customers rather than staff. Just
register as an ISP and get the default /32.

IIRC, ARIN sparsely allocates IPv6 so if you go back for more addresses
there is a high probability they'll just increase your netmask.

Finally, /56 or /60 per guest, not /64. IPv6 can do nifty IoT things like
collecting all of a guest's devices behind his personal firewall but it
doesn't work if you've only assigned a /64.

Regards,
Bill Herrin

6

Bill,

Thanks for the input. I don't consider us an isp, though i suppose i can
see how that argument could me made. Hotels are both simple and
complicated. There is a mix of our staff and equipment, guests and their
equipment, and brands with their equipment. But really it's just one
operating entity that ultimayely isn't that much different than any other
enterprise out there. Now multiply that by 60-65 sites spread across the
country and we need to manage our 6000 staff and networks accordingly. We
operate 100% of the hotel, top to bottom, not just the technology.

I wouldn't want ARIN or anyone else thinking we were an ISP if we aren't.
Particulary if that creates problems in the future as rules (and possibly
costs) change.

However, if what you are saying is that registerong as an ISP is actually
the correct way to go about this in ARIN's eyes as well, then that's a
different story.

Thanks for the tip on IoT sizing. That's precisely the kind of thing i am
concerned about being constrained with in the future if we size sites too
small.

Oliver

We're an end-user org and qualify for a /40 assignment because we operate
over 60 sites and some of those are/will be multihomed.

Hi Oliver,

I second Ken's notion. You're trying to be an ISP under the end-user rules.
However transient, your users are mostly customers rather than staff. Just
register as an ISP and get the default /32.

IIRC, ARIN sparsely allocates IPv6 so if you go back for more addresses
there is a high probability they'll just increase your netmask.

Finally, /56 or /60 per guest, not /64. IPv6 can do nifty IoT things like
collecting all of a guest's devices behind his personal firewall but it
doesn't work if you've only assigned a /64.

Regards,
Bill Herrin

7

Oliver,

I’ll mostly second what Bill has said here. However, I encourage you to actually
consider a /48 per guest room as well as a /48 per hotel for the hotel itself.

Yes, this is excessive, but IPv6 was designed with these types of excesses in mind.

We don’t yet know the scope and breadth of what will come out of IoT development,
but one thing we do know for sure… Development tends to get stymied by whatever
turns into the lowest common denominator among available technologies out there,
so if 10 hotel chains give their guests /48s and 2 give out /60s and one gives
out /64s, development may well lock everyone into nothing better than what can
be done with a /64 even if better is available.

We’ve seen this time and again with products that depend on autodiscovery processes
that rely on everything being on the same LAN and assume that they can just trust
the NAT router to protect that LAN from anything else. This is clearly a very bad
strategy to anyone who understands networking, yet if you walk into your local
Best Buy, more of the “internet enabled” products on the shelf have this behavior
than don’t… Far more.

Owen

8

On 7/7/17, 1:07 PM, "NANOG on behalf of Oliver O'Boyle"

We're currently in the planning stage and can make
whatever changes we need to.

I always say to just expect you’ll change your address plan three times.
Some people say, “I’ve only changed the address plan twice. . . so far.”

Situation:

We're an end-user org and qualify for a /40 assignment because we operate
over 60 sites and some of those are/will be multihomed. We manage hotels
in
Canada only, but from coast to coast to coast and everywhere in between.
Our corporate network and org structure is optimized for three regions. We
also have, and continue to grow into, cloud infrastructure and foresee
wanting to bring our own addresses (.e.g., to AWS VPC when that option
becomes available). As such, an obvious design strategy would be to break
the /40 into 4 x /42's. However, due to an imbalance in national site
distribution, 50% of our sites are located in one region (Region A).
Additionally, historical and forecasted growth indicates that it's
perfectly reasonable for us to expect growth of an additional 16 sites in
that same region over the next 3-5 years.

Even assuming, as you said: a /48 per hotel, it sounds like you’re
planning for:
Region A, 45 sites, minimum /42
Region B, <20 sites, minimum /44
Region C, <20 sites, minimum /44
Cloud stuff, minimum /48, but that might need more

However, as others have suggested, you might want to start from the
bottom, deciding the allocations within each hotel. It may be that you
need multiple /48s for HotelGuest, HotelLobby, HotelConference, and
HotelStaff SSIDs.
A /64 per WiFi AP is an aboslute minimum, but a prefix per room (or guest)
would be better, and there are reasons to consider /56 and /48 per “end
user” in the hotel. Even if you can’t assign it with current WiFi
technology, your address plan should allow for an evolution to a better
way of doing things.
If my math works right, and you have between 127 and 255 rooms in a hotel:
255 * /56 = /48 just for HotelGuest. You may need a /44 per hotel, if
there are four separate networks.
Or:
255 * /48 = /40 just for HotelGuest. You may need a /36 per hotel.

As others have said, I’m assuming you treat guests to whom you provide
Internet service as customers.

I think the ideal situation is out as ARIN policy wouldn't allow them to
assign us a /36 at this time. Unless someone knows something that can help
us here.

Try calling ARIN. Ask a hostmaster whether the End User or ISP category
makes more sense in your case. It’s also possible they’ll say “slow start”
and give you a /40 for your first hotel, and tell you to return in a week
when you need more.

But also, take into account [NRPM 6.5.8.2] "Requests for larger
initial assignments, reasonably justified with supporting
documentation, will be evaluated based on the number of sites in an
    organization’s network and the number of subnets needed to support any
           extra-large sites defined below.” There’s a lot of room within
policy to do sensible things with IPv6.

Assuming we can't get a /36, my feeling is that less ideal situation #2 is
better than #3 is better than #1 is better than #4, assuming we're
following the following design best-practices:

a) assign top-level aggregations evenly (which we'd be breaking a bit with
option #2)
b) reduce global routes as much as possible
c) stay on the nibble boundary as much as possible
d) default to /48 per site

Yes, all good goals. But none is critical to the success of your network
(except c, only if you plan to delegate reverse DNS). “As much as
possible” also implies “and no more than is possible.”

Thanks in advance,
Oliver

btw, I can’t wait to stay in your hotels once they have IPv6! I hope
you’ll be able to tweet or post here when it’s deployed, so we can
congratulate you, and maybe get some conferences to consider you as a
venue.

Lee

9

I think the classfull madness of "/48 everywhere" should stop at some
point; the "every subnet is a /64" is enough already.

A /48 is 65536 *subnets*, with each subnet having space for what can be
considered "unlimited" number of devices.
A /56 already is 256 *subnets*.
Now please show be a hotel room that has close to 65536 items in it
(also tell me how much does a night in such a room cost).
Then how many rooms may host close to 256 devices that can transmit and
receive data ?
And then again, at the end of the day a hotel is *NOT* and ISP, a hotel
is a hotel. Internet access is just an extra service that became
mandatory lately in order to remain "competitive".

10

Radu,

Are you assuming that a goal of IPv6 is to efficiently fill subsets? I submit that it is not. There are advantages to sparse address spaces, among them easy mapping of MAC addresses for transition purposes and the security that discourages malefactors from quickly enumerating active devices in a subnet.

But that's not the main reason for /64 basic subsets. One of the guiding principles of IPv6 was to not make the mistake of underestimating the future applications of IP addresses. Thus your question "what hotel room has 65536 items in it?" has no meaning in terms of future applications. As you point out, we're not talking about hotel rooms. We don't, by definition, know what we're talking about for future applications.

I tell people in my IPv6 classes that we have to stop thinking of ourselves in a spacesuit with a limited air supply that must be rationed, and instead recognize that we're now in a wide-open planet-sized atmosphere where we can breathe freely, and without apportionment.

That open atmosphere was by design. It's why IPv6 uses 128-bit addresses, and not 48- or 64-bit. In the exponential space of integers, IPv6 selected a maximum integer that was many orders of magnitude greater than we could ever imagine needing at the time.

They're just integers. Not lumps of gold. And there's more where those came from :slight_smile:

-mel beckman

11

Hi Oliver, et al, I recall from when I attended an ARIN on the Road meeting in Austin last year ( https://www.arin.net/ontheroad/ ), that the folks at ARIN seemed to be open to discussing with you about getting the right size address space into your hands for what you needed to accomplish....within reason...and within justification. I won't speak for ARIN, but I just seem to remember that they were open to talking about it. I don't recall if you said you have actually had dialogue with ARIN about getting the "right" amount of address space to accomplish what you are looking to do. If not, please reach out to them. They've always been helpful and responsive when I've discussed IPv4 and also now, v6 with them.

Also, I recall in a v6 online class I did that one point that was made was to not take too much time analyzing, but to get moving with v6. I think Lee just said you should plan on readdressing a few times. Ok, fine. I see that as being possible. You live and you learn. I did find myself last year and earlier this year spending A LOT of time going over and over and over again, the "best" way to carve up my /32 of v6 addresses with fellow engineers. We stopped talking about it for a while... then I came back recently and said guys we gotta settle on something and go for it ! Well, we did and I'm glad. I'm not saying be willy nilly about your v6 space, but settle on something sensible and go for it... then be open to course correcting along the way, and readdress where you must. I've dual staked a few of my cdn public caches, and am talking about dual-stacking 7,000 DSL customers that are currently doing NAT444.

v6 is fairly early in my deployment and going fine so far. Btw, I will add that I love my 6VPE. Dang MPLS xVPN's make my life so nice and manageable. You geniuses out there that invent technology are incredible. Keep it up.

-Aaron Gould

12

Radu,

Are you assuming that a goal of IPv6 is to efficiently fill subsets? I

No, but I assume IPv6 is still subject to common-sense.

among them easy mapping of MAC addresses for transition purposes and the
security that discourages malefactors from quickly enumerating active
devices in a subnet.

I do get all those points. And by the way, try to explain the same to
security people.

But that's not the main reason for /64 basic subsets. One of the guiding
principles of IPv6 was to not make the mistake of underestimating the
future applications of IP addresses. Thus your question "what hotel room

... so it went directly to over-estimating ....

has 65536 items in it?" has no meaning in terms of future applications.
As you point out, we're not talking about hotel rooms. We don't, by
definition, know what we're talking about for future applications.

All this by forgetting today's applications.
And no, you can't possibly treat the same way a hotel room and a 4 floor
site with a server room.

I tell people in my IPv6 classes that we have to stop thinking of
ourselves in a spacesuit with a limited air supply that must be rationed,
and instead recognize that we're now in a wide-open planet-sized
atmosphere where we can breathe freely, and without apportionment.

Well, by having 64 bits for each subnet, I start lacking bits for other
things (like inter-devices connections, ....). I'm not in a space-suit,
but I'm on top of Kilimanjaro, where air pressure is only half of what
we're used to.

That open atmosphere was by design. It's why IPv6 uses 128-bit addresses,

That's for hosts. When you care more about subnets, it's shortened to 64
bits.

They're just integers. Not lumps of gold.

Be careful, IPv4 got upgraded from numbers to gold a number of years
ago.

And there's more where those came from :slight_smile:

Hopefully. I'm just curious if 8000::/4 will obey today's rules or not.

Back to the original question, I find it delirious to treat a small
entity the same as a big one, especially when the size difference
between the two is several orders of magnitude. Even if we consider
"future applications", there's still a very high chance that the size
will still matter. Get "the IT guy" of a small company to get used with
a /48 for his 20 people, 5 printers and 2-3 servers set-up, then
imagine what happens with a design of a "site" 10 or 100 times bigger.
This is something that you already see with VLAN ids and RFC1918 space.
Even if you think you gave people "as much as they will ever need", they
will still end up needing more.

13

Well, as I sit here, my apartment edge router gets a /60 from Comcast, and
burns through them pretty quick. A subnet for the 4 wired devices,
another for the 2.4Ghz wireless, another for 5ghz wireless, and if I
enabled them another 2 guest wireless subnets.. and then more for any
VLAN I might set up. If I lived in a large enough house, I'm *already*
out of enough address space to easily prefix-delegate to a second router
at the far end of the house.

And yes, this *is* a setup where there's only 1 or 2 devices per most subnets.

So no, the idea is *not at all* to see how we can cram as many devices as
possible onto a subnet. The idea is to set up a networking environment where
it's as easy as possible for even fairly stupid devices to be able to
auto-configure and join in. And there's *really* good security reasons
for your FizzBin 5000 that wants to be a IoT device but you don't really
trust, to end up on a different subnet from your laptop....

14

Hi Oliver,

You read to me like a borderline case. It comes up a lot with universities:
are they end users or ISPs to their students? ARIN will generally accept
either explanation. You'll get the larger number of IPv6 addresses you want
if you tell them you're an ISP.

The cost difference is likely to remain minimal. The major issue is that as
an ISP you'll be expected to enter SWIP records so read up on that.

Regards,
Bill

15

"Radu-Adrian Feurdean" <nanog@radu-adrian.feurdean.net> writes:

No, but I assume IPv6 is still subject to common-sense.

I don't see how you can make that assumption.

If common sense had been applied, then people would have realized that
there are more important parameters than address conservation to
consider when it comes to IPv6 planning/design/discussions/whatever.

Bjørn

16

Agreed with the /48 but ARIN doesn't appear to agree with our justification
for a /36 thus far.

I am not sure how you have been communicating with ARIN, my experience with them strongly suggest that after you put in your request, pickup the phone and call them, speak to the analyst assigned to your request..

Have a polite conversation with them, and ask them .. how to go about accomplishing what you are needing...

You are going to be in for a very pleasant surprise.

Regards.

Faisal Imtiaz
Snappy Internet & Telecom

17

Indeed. Especielly if you do hierarchical delegation within your
organization, you will often have sparse allocations at several
levels. A /48 for an organization might seem like reasonably lots
with 65536 subnets, but if that organization in turn delegates to
departments, and they have more than 16 departments, each department
might only get a /56 (256 subnets). Try to delegate that one step
further, and you can't do any reasonable allocation strategy, but
have to allocate subnet by subnet.

I managed to get a full /52 from our host university, but they
initially wanted to give us only a /56. Of course, they can only
give out a few /52:s; other departments will have less structured
address plans than us.

- --
Thomas Bellman, National Supercomputer Centre, Linköping Univ., Sweden
"Life IS pain, highness. Anyone who tells ! bellman @ nsc . liu . se
differently is selling something." ! Make Love -- Nicht Wahr!

18

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

>> That open atmosphere was by design. It's why IPv6 uses 128-bit addresses,
>
> That's for hosts. When you care more about subnets, it's shortened to 64
> bits.

Indeed. Especielly if you do hierarchical delegation within your
organization, you will often have sparse allocations at several
levels. A /48 for an organization might seem like reasonably lots
with 65536 subnets, but if that organization in turn delegates to
departments, and they have more than 16 departments, each department
might only get a /56 (256 subnets).

If I had 32 departments and were wanting to give them equal sized
allocations then I'd give them a /53 each which is 2064 subnets
each. It isn't that hard to do 8 delegations in the reverse tree
for each of the 32 departments. Delegation on nibble boundaries
is for convience and nothing else.

Or you could start with a /56 each and reserve the next 7 /56s for
each department to grow into.

Try to delegate that one step
further, and you can't do any reasonable allocation strategy, but
have to allocate subnet by subnet.

Of which the only downside is that it is harder to do internal
firewalls between departments or if you want to do cost recovery
of external traffic to departments. The DHCPv6 servers also need
to learn where to get additional subnets from to fulfill PD requests.

Remember a site can have more than one /48. Thats just the recommended
starting point.

19

Mark Andrews <marka@isc.org> writes:

If I had 32 departments and were wanting to give them equal sized
allocations then I'd give them a /53 each which is 2064 subnets
each. It isn't that hard to do 8 delegations in the reverse tree
for each of the 32 departments. Delegation on nibble boundaries
is for convience and nothing else.

I believe you under-estimate the importance of sysadmin convenience...

Yes, you *can* do 8 delegations. And you are of course right - it's not
even hard. But it does come with a "less convenient" price tag, so
you'd better get something back. What was that, exactly? Right, you
saved a /48. Big deal.

Bjørn

20

For comprehensibility which nets convenience. Consistently delegate on
nibble boundaries and your power users don't have to understand Boolean
algebra to make sense of the network.

Regards,
Bill Herrin