Using /126 for IPv6 router links

Mailman List Mirror (Read Only)
21

First off, it was 5, not 18. :slight_smile:

Second, there's not much evidence that TJ Watson actually said it.

http://en.wikipedia.org/wiki/Thomas_J._Watson#Famous_misquote

Third, given that IBM had already been shipping accounting units with limited
plugboard programmability (the model 405) for almost a decade at that point,
it's reasonable to conclude that TJ was intentionally and specifically talking
about high-end "if you have to ask you can't afford it" systems. And if you
look at the Top500 list now, 65 years years later, it's still true - there's
always 2-5 boxes that are *way* out in front, then a cluster in spots 5-20 or
so, and then a *really* long tail on the way down to #500.

http://www-03.ibm.com/ibm/history/exhibits/vintage/vintage_4506VV4006.html

22

I remember a day when 18 was the largest number of computers that would
ever be needed.

First off, it was 5, not 18. :slight_smile:

Second, there's not much evidence that TJ Watson actually said it.

Thomas J. Watson - Wikipedia

I think the 18 was a UNIVAC blunder (don't remember who supposedly said it). Given their corporate history, I can believe it,

Third, given that IBM had already been shipping accounting units with limited
plugboard programmability (the model 405) for almost a decade at that point,
it's reasonable to conclude that TJ was intentionally and specifically talking
about high-end "if you have to ask you can't afford it" systems. And if you
look at the Top500 list now, 65 years years later, it's still true - there's
always 2-5 boxes that are *way* out in front, then a cluster in spots 5-20 or
so, and then a *really* long tail on the way down to #500.

http://www-03.ibm.com/ibm/history/exhibits/vintage/vintage_4506VV4006.html

It may surprise some folks to learn that there were several computer makers--IBM was not the first, not the best, and not the stupidest.

23

64 bits is enough networks that if each network was an almond M&M,
you would be able to fill all of the great lakes with M&Ms before you
ran out of /64s.

Did somebody once say something like that about Class C addresses?

The number of /24s in all of IPv4 would only cover 70 yards of a football
field (in a single layer of M&Ms). Compared to the filling the
three-dimensional full volume of all 5 great lakes, I am hoping you can
see the vast difference in the comparison.

Of course--I was asking about the metaphorical message implying "More than we can imagine ever needing".

I remember a day when 18 was the largest number of computers that would ever be needed.

Do not make the mistake of assuming that just because I support using IPv6
as designed (at least for now) I am too young to remember those things myself.

While I wasn't born early enough to remember the demand for 18 computers
projection of T.J. Watson in the first person, I am quite familiar with the quote
and the environment that fostered it. I am also familiar with the history of
the internet and it's 8-bit address precursor.

Yes, your point about demand expanding beyond expectation is well taken.
However, I believe that the scale of the IP address space will accommodate
at least a couple of orders of magnitude expansion beyond any anticipated
amount of address space demand. Further, the current IPv6 addressing
scheme does come with a safety valve if people like me turn out to be wrong.
If we're wrong, it will only affect 1/8th of the address space and we can do
something different with the other nearly 7/8ths, possibly setting a 5-10 year
horizon for renumbering out of the first 1/8th into more condensed addressing
schemes so that the original 1/8th isn't wastefully allocated.

Finally, we come to another key difference between IPv4 and IPv6 which
is one of its best features and one of the things that has created the greatest
controversy among legacy IPv4 holders. There is no IPv6 globally routable
unicast space which is not issued by an RIR under contract with the recipient.
Unlike in IPv4 where the ability to reclaim addresses (whether abandoned,
underutilized, or otherwise) is murky at best, all IPv6 addresses are subject
to a nominal annual fee and a contract which allows the RIRs to maintain
proper stewardship over them.

If I were designing IPv6 today, would I reserve 1/2 the bits for the host
address? No, I wouldn't do that. However, I do think there is benefit
to a fixed-sized host field. However, the design we have is the design
we have. It's too late to make fundamental changes prior to deployment.
Stack implementations all have the ability to adapt to non-fixed-size
networks if necessary down the road, but, for now, /64s are the
best way to avoid surprises and move forward.

Owen

24

>
>>>> 64 bits is enough networks that if each network was an almond M&M,
>>>> you would be able to fill all of the great lakes with M&Ms before you
>>>> ran out of /64s.
>>>
>>> Did somebody once say something like that about Class C addresses?
>>>
>> The number of /24s in all of IPv4 would only cover 70 yards of a football
>> field (in a single layer of M&Ms). Compared to the filling the
>> three-dimensional full volume of all 5 great lakes, I am hoping you can
>> see the vast difference in the comparison.
>
> Of course--I was asking about the metaphorical message implying "More than we can imagine ever needing".
>
> I remember a day when 18 was the largest number of computers that would ever be needed.
>
Do not make the mistake of assuming that just because I support using IPv6
as designed (at least for now) I am too young to remember those things myself.

While I wasn't born early enough to remember the demand for 18 computers
projection of T.J. Watson in the first person, I am quite familiar with the quote
and the environment that fostered it. I am also familiar with the history of
the internet and it's 8-bit address precursor.

Yes, your point about demand expanding beyond expectation is well taken.
However, I believe that the scale of the IP address space will accommodate
at least a couple of orders of magnitude expansion beyond any anticipated
amount of address space demand. Further, the current IPv6 addressing
scheme does come with a safety valve if people like me turn out to be wrong.
If we're wrong, it will only affect 1/8th of the address space and we can do
something different with the other nearly 7/8ths, possibly setting a 5-10 year
horizon for renumbering out of the first 1/8th into more condensed addressing
schemes so that the original 1/8th isn't wastefully allocated.

Finally, we come to another key difference between IPv4 and IPv6 which
is one of its best features and one of the things that has created the greatest
controversy among legacy IPv4 holders. There is no IPv6 globally routable
unicast space which is not issued by an RIR under contract with the recipient.
Unlike in IPv4 where the ability to reclaim addresses (whether abandoned,
underutilized, or otherwise) is murky at best, all IPv6 addresses are subject
to a nominal annual fee and a contract which allows the RIRs to maintain
proper stewardship over them.

If I were designing IPv6 today, would I reserve 1/2 the bits for the host
address? No, I wouldn't do that.

Actually, from what Christian Huitema says in his "IPv6: The New
Internet Protocol" book, the original IPv6 address size was 64 bits,
derived from Steve Deering's Simple Internet Protocol proposal.
IIRC, they doubled it to 128 bits to specifically have 64 bits as the
host portion, to allow for autoconfiguration.

25

Actually, Scott Bradner and I share most of the credit (or blame) for the change from 64 bits to 128.

During the days of the IPng directorate, quite a number of different alternatives were considered. At one point, there was a compromise proposal known as the "Big 10" design, because it was propounded at the Big Ten Conference Center near O'Hare. One feature of it was addresses of length 64, 128, 192, or 256 bits, determined by the high-order two bits. That deal fell apart for reasons I no longer remember; SIPP was the heir apparent at that point. Scott and I pushed back, saying that 64 bits was too few to allow for both growth and for innovative uses of the address. We offered 128 bits as a compromise; it was accepted, albeit grudgingly. The stateless autoconfig design came later.

    --Steve Bellovin, http://www.cs.columbia.edu/~smb

26

I don't remember the details of Big 10, but I do remember the general objection
to variable-length addresses (cf. some of the OSI-influenced schemes) was the
perceived difficulty of building an ASIC to do hardware handling of the
address fields at line rate. Or was Big 10 itself the compromise to avoid
dealing with variable-length NSAP-style addresses ("What do you mean, the
address can be between 7 and 23 bytes long, depending on bits in bytes 3, 12,
and 17?" :slight_smile:

27

Precisely. The two bits could feed into a simple decoder that would activate one of four address handlers; depending on your design, they could all run in parallel, with only the output of one of them used. There were only four choices, all a multiple of 8 bytes.

But my goal is not to revisit the design issues, but rather to clarify the historical record.

    --Steve Bellovin, http://www.cs.columbia.edu/~smb

28

I too prefer /112s. I can take the first /64 in any assignment or allocation and set it aside for networking infrastructure.
The first /112 is for loopbacks, the remaining /112s are for linknets.

Then I can filter this /64 at my border, and it's easy.

You can do the same thing with /64 linknets, but then you have to set aside a block of them, and that might get hard if you have a /48 or something. Maybe not. What if you have a /56?

Maybe there is some value in linknets being effectively disposable so you never have to worry about problems coming from re-use. A single /64 full of /112s gives you 281 trillion.

For links to customers and other networks, I like /64s, because they are right now the standard so you're not going to run in to compatibility problems. If you've got links to customers you should have a /32, so setting aside a /48 or a /44 or something for those customer links is no huge drama.

29

I'd suggest using a /126. For two reasons.

1) Using EUI-64 addresses on router-router links is an error, the
    consequences of which you encounter the first time you replace
    some faulty hardware.[1] I have made this error :frowning: If you
    are not using EUI-64 then assigning a non-autoconf /64 is no
    less or more work than assigning a non-autoconf /126.

2) Having a block of addresses for router-router links (and other blocks
    for other infrastructure such as loopbacks and unicast) makes ACLs
    much simpler. Using a /126 means that this block can last for a long,
    long time, reducing configuration maintenance.

Cheers, Glen

[1] Basically, interface addresses end up scattered through the
     router's configuration (some manufacturers are better than
     others in this regard). Tracking down all the references to
     an address and changing the config merely as the result of a
     hardware swap is painful and adds complexity at a time when
     it is not desired.

30

I think there's a lot of value in knowing why things are the way they
are. It's common enough to see things that at face value appear to be
overly complicated e.g. classes or subnets in IPv4 compared to IPX's
simple, flat network numbers, or appear unrealistic or "ridiculous"
like IPv6's 128 bit addresses.

However I've found once I know the problem that was trying to be
solved, and what options there were to solve it, I then usually
understand why the particular solution was chosen, and most of the time
agree with it. The value I got out of Christian's book was not the going
through the mechanisms of IPv6, but his perspective on what options
there were to solve certain options, and why the choices were made.

Regards,
Mark.

31

> Use the /64... It's OK... IPv6 was designed with that in mind.

I'd suggest using a /126. For two reasons.

1) Using EUI-64 addresses on router-router links is an error, the
    consequences of which you encounter the first time you replace
    some faulty hardware.[1] I have made this error :frowning: If you
    are not using EUI-64 then assigning a non-autoconf /64 is no
    less or more work than assigning a non-autoconf /126.

So all that's really saying is that you shouldn't use node addresses
derived from link layer addresses, due to the risk of them changing
when you swap out an interface, which makes sense. I don't think that
argument really supports not using /64s though, as <blah>::1/64 and
<blah>::2/64 would achieve that too.

2) Having a block of addresses for router-router links (and other blocks
    for other infrastructure such as loopbacks and unicast) makes ACLs
    much simpler. Using a /126 means that this block can last for a long,
    long time, reducing configuration maintenance.

With a /48, the recommended size for an end-site, giving you 65
536 subnets, you could reserve the top quarter 16 384 subnets for your
point to point links and loopbacks (or split that in half to separate
loopbacks and p-to-p links), and then cover them with ACL them with
<blah>:c000::/50, and still have 49 152 subnets for your edge/services
LANs.

Regards,
Mark.

32

During the days of the IPng directorate, quite a number of different alternatives were considered. At one point, there was a compromise proposal known as the "Big 10" design, because it was propounded at the Big Ten Conference Center near O'Hare. One feature of it was addresses of length 64, 128, 192, or 256 bits, determined by the high-order two bits. That deal fell apart for reasons I no longer remember; SIPP was the heir apparent at that point. Scott and I pushed back, saying that 64 bits was too few to allow for both growth and for innovative uses of the address. We offered 128 bits as a compromise; it was accepted, albeit grudgingly. The stateless autoconfig design came later.

           \-\-Steve Bellovin, http://www.cs.columbia.edu/~smb

This historical record finally made me understand why we have up to
/128 prefixes with /128 addresses instead of what would suit best
stateless autoconfig: up to /64 prefixes with /128 addresses.

Rubens

33

If you have link nets, you probably shouldn't have just a /48 and you CERTAINLY shouldn't have just a /56.

Owen

34

No. There are only 2,097,152 Class C networks.

Assuming all M&Ms are spheroids of uniform oblate nature, radius major
axis=6mm, minor axis=3mm. Volume is (4/3)Pi (Major^2) Minor
(http://en.wikipedia.org/wiki/Spheroid#Volume)

They will be poured into a great lake of your choice, and we will assume
random close packing (agitation mechanisms are probably best discussed
off-list) and a (generous, but the article insists) void fraction of 32%.
(http://en.wikipedia.org/wiki/Random_close_pack)

Volume of m&m = 0.452cm^3, occupies 0.665cm^3.

Lake Erie is 484km^3
http://www.epa.gov/glnpo/factsheet.html

1 km^3 = 1,000,000,000,000,000 cm^3

484,000,000,000,000,000 * 0.665 = 321,860,000,000,000,000 m&ms needed to
fill this lake.

There are 4,294,967,296 /64s in my own /32 allocation. If we only ever
use 2000::/3 on the internet, I make that 2,305,843,009,213,693,952
/64s. This is enough to fill over seven Lake Eries. The total amount
of ipv6 address space is exponentially larger still - I have just looked
at 2000::/3 in these maths.

THE IPv6 ADDRESS SPACE IS VERY, VERY, VERY BIG.

** Can we please now just go ahead and roll out some ipv6 services ? **

Thanks
Andy

35

As I mentioned in my lightning talk at the last NANOG, we reserved a
/64 for each
PtP link, but configured it as the first /126 out of the /64. That
gives us the most
flexibility for expanding to the full /64 later if necessary, but
prevents us from being
victim of the classic v6 neighbor discovery attack that you're prone
to if you configure
the entire /64 on the link. All someone out on the 'net needs to do
is scan up through
your address space on the link as quickly as possible, sending single packets at
all the non-existent addresses on the link, and watch as your router CPU starts
to churn keeping track of all the neighbor discovery messages, state table
updates, and incomplete age-outs. With the link configured as a /126, there's
a very small limit to the number of neighbor discovery messages, and the amount
of state table that needs to be maintained and updated for each PtP link.

It seemed like a reasonable approach for us--but there's more than one way to
skin this particular cat.

Hope this helps!

Matt

36

Don't get carried away with all of that "IPv6 is huge" math, it quickly
deteriorates when you start digging into it. Auto-configuration reduces
it from 340282366920938463463374607431768211456 to 18446744073709551616
(that's 0.000000000000000005% of the original 128 bit space). Now as an
end user you might get anything ranging from a /56 to a /64, that's only
between 1 - 256 IPs, barely a significant increase at all if you were to
actually use a /64 for each routed IP rather than as each routed subnet.
As a small network you might get a /48, so that even if you gave out
/64s to everyone it would be only 16 bits of space for you (the
equivalent of getting a class B back in IPv4 land), something like a
8-16 bit improvement over what a similar sized network would have gotten
in IPv4. As a bigger ISP you might get a /32, but it's the same thing,
only 16 bits of space when you have to give out /48s. All we've really
done is buy ourselves an 8 to 16 bit improvement at every level of
allocation space (and a lot of prefix bloat for when we start using more
than 2000::/3), which is a FAR cry from the 2^128 "omg big number, we
can give every molecule an IPv6 address" math of the popular
imagination. :slight_smile:

37

That's an attack vector you have to worry about even today with IPv4
space. There are quite a few vendors who you can make fall over with CPU
trying to do arp resolution and/or cam exhaustion just by doing "ip
address x.x.x.x/16" as a directly connected block on an Ethernet
interface. One redeeming quality to the whole autoconfig thing is it'll
do a great job cutting down on port scanning for vulnerabilities on end
hosts (or else make the flood of port scanning traffic 2^64 times worse,
it remains to be seen I suppose :P). My personal theory is it will
result in a black market of buying and selling people's active IPv6
addresses from various website logs and the like, so hax0rs will have
something to scan. In a few years time it will probably be popular with
end users to periodicially "rotate the shield frequencies" with their
final 64 bits, or maybe even use them on a per-transaction basis for
extra security. :slight_smile:

38

Good Morning!

From: Richard A Steenbergen [mailto:ras@e-gerbil.net]
Sent: Monday, January 25, 2010 05:45
To: Andy Davidson
Cc: nanog@nanog.org
Subject: Re: Using /126 for IPv6 router links

> There are 4,294,967,296 /64s in my own /32 allocation. If we only ever
> use 2000::/3 on the internet, I make that 2,305,843,009,213,693,952
> /64s. This is enough to fill over seven Lake Eries. The total amount
> of ipv6 address space is exponentially larger still - I have just looked
> at 2000::/3 in these maths.
>
> THE IPv6 ADDRESS SPACE IS VERY, VERY, VERY BIG.

Don't get carried away with all of that "IPv6 is huge" math, it quickly
deteriorates when you start digging into it. Auto-configuration reduces
it from 340282366920938463463374607431768211456 to 18446744073709551616
(that's 0.000000000000000005% of the original 128 bit space). Now as an
end user you might get anything ranging from a /56 to a /64, that's only
between 1 - 256 IPs, barely a significant increase at all if you were to
actually use a /64 for each routed IP rather than as each routed subnet.
As a small network you might get a /48, so that even if you gave out
/64s to everyone it would be only 16 bits of space for you (the
equivalent of getting a class B back in IPv4 land), something like a
8-16 bit improvement over what a similar sized network would have gotten
in IPv4. As a bigger ISP you might get a /32, but it's the same thing,
only 16 bits of space when you have to give out /48s. All we've really
done is buy ourselves an 8 to 16 bit improvement at every level of
allocation space (and a lot of prefix bloat for when we start using more
than 2000::/3), which is a FAR cry from the 2^128 "omg big number, we
can give every molecule an IPv6 address" math of the popular
imagination. :slight_smile:

While I agree with parts of what you are saying - that using the "simple
2^128" math can be misleading, let's be clear on a few things:
*) 2^61 is still very, very big. That is the number of IPv6 network
segments available within 2000::/3.
*) An end-user should get something between a /48 and a /56, _maybe_ as low
as a /60 ... hopefully never a /64. Really.
**) Let's call the /48s enterprise assignments, and the /56s home
assignments ... ?
**) And your /56 to /64 is NOT 1-256 IPs, it is 1-256 segments.
**) And, using the expected /48-/56, the numbers are really 256-64k subnets.
**) Each segment supporting as many hosts as you want it to. Probably
nowhere near 2^64, but that isn't the point :).
*) _Any_ ISP gets a /32 by default, a "bigger ISP" can readily get more.

So, actually, we have 'bought' ourselves much more space.
*) The standard registry allocation is a /12. So within the /3 we have 512
of those. Note: We currently have 5 RIRs.
*) A /12 yields 20 bits of /32s. So within any given /12, we have ~1M ISPs.
*) The "standard ISP /32" can support 64K Enterprises or 16.7M Homes.
**) Oh, and if you need more = just ask.
*) Even allowing for inefficiency / room to grow / summarization - I think
we are good for quite some time.
*) And this is just the first /3.

Note: "All we've really done is buy ourselves an 8 to 16 bit improvement at
every level of allocation space"
*) And you don't think 8-16 bits _AT EVERY LEVEL_ is a bit deal??
**) Remembering that the original address space was 'only' 32bits.
**) I guess only supporting 256-64k more registries, each of which can
support 256-64k more ISPs, each of which can support 256-64k more customers
just isn't that useful to you?
*) Additionally - the number of IPs per segment, which is not the same as
the number of hosts per segment, is much vaster. The quite common IPv4 /24
being analogous to an IPv6 /64 ...

/TJ
PS - We also get much more multicast space, Which Is Nice(TM).

39

The gazillions of one-time-use nanomachines used to scrape away plaque in just a single patient's bloodstream,

Ah, now if only was hadn't deprecated site-local... :slight_smile:

40

Ok let's summarize:

/64:
+ Sticks to the way IPv6 was designed (64 bits host part)
+ Probability of renumbering very low
+ simpler for ACLs and the like
+ rDNS on a bit boundary

<> You can give your peers funny names, like 2001:db8::dead:beef :wink:

- Prone to attacks (scans, router CPU load)
- "Waste" of addresses
- Peer address needs to be known, impossible to guess with 2^64 addresses

/126
+ Only 4 addresses possible (memorable, not so error-prone at configuration-time and while debugging)
+ Not prone to scan-like attacks

- Not on a bit boundary, so more complicated for ACLs and …
- … rDNS
- Perhaps need to renumber into /64 some time.
- No 64 bits for hosts

/127
Like /126 but there's an RFC not recommending it and an RFC (draft) which revises that non-recommendation.

Hi
In reference to the discussion about /31 for router links, I d'like to know what is your experience with IPv6 in this regard.

I use a /126 if possible but have also configured one /64 just for the link between two routers. This works great but when I think that I'm wasting 2^64 - 2 addresses here it feels plain wrong.

So what do you think? Good? Bad? Ugly? /127 ? :wink:

Cheers

Mathias Seiler
MiroNet GmbH, Strassburgerallee 86, CH-4055 Basel
T +41 61 201 30 90, F +41 61 201 30 99
mathias.seiler@mironet.ch
www.mironet.ch

As I mentioned in my lightning talk at the last NANOG, we reserved a
/64 for each
PtP link,
but configured it as the first /126 out of the /64. That
gives us the most
flexibility for expanding to the full /64 later if necessary, but
prevents us from being
victim of the classic v6 neighbor discovery attack that you're prone
to if you configure
the entire /64 on the link.

I think I will go this way. Since we've got the usual /32 assignment I have plenty of /64 to "waste".
If I continue assigning a /48 to every customer I can set apart a /64 for each PtP link and still have room to grow for a very long time (I'm not taking into account the assignment of IPv6 addresses to high amounts of M&Ms so far :wink: )

This way the configuration and addressing plan is simple and understandable to anyone.

All someone out on the 'net needs to do
is scan up through
your address space on the link as quickly as possible, sending single packets at
all the non-existent addresses on the link, and watch as your router CPU starts
to churn keeping track of all the neighbor discovery messages, state table
updates, and incomplete age-outs.

Well I could filter that in hardware with an interface ACL but a /126 seems much easier to maintain.

With the link configured as a /126, there's
a very small limit to the number of neighbor discovery messages, and the amount
of state table that needs to be maintained and updated for each PtP link.

It seemed like a reasonable approach for us--but there's more than one way to
skin this particular cat.

Hope this helps!

Yes it does. Thanks!

Mathias Seiler

MiroNet GmbH, Strassburgerallee 86, CH-4055 Basel
T +41 61 201 30 90, F +41 61 201 30 99

mathias.seiler@mironet.ch
www.mironet.ch