IPV6 in enterprise best practices/white papaers

NDP multicast has scaling issues, and I'd not be surprised if switches
will soon stop learning it and flood all NDP multicasts to save space
for the users' higher-traffic multicast groups.

This is very reasonable, because end-host Ethernet chipsets have been
discarding useless frames since the beginning. Even unicast frames were
flooded in the times of coax and hubs; ethernet chipsets will drop
disinteresting frames on the floor.

The problem with ARP and other broadcasts was that they were never
dropped by any ethernet chipset, because there was no way for it to know
if it is interesting. NDP multicast addresses, on the other hand, allow
for the device to program only the multicast MACs it is interested about
in the ethernet chipset, so the CPU will never see the useless packets.

This is a very good compromise for most cases; you haul some useless
packets, but they are dropped by the ethernet chipset, so even the most
measly print server or internet controlled coffee maker CPU will not be
unduly burdened. You will also not need to burden your network with
multicast groups (=state) to save hauling a few useless packets around.

* * *

There are some cases where it actually is expensive to flood ARP/NDP
requests, like 802.11 WLANs where bandwidth can be limited and
multicast/broadcast is implemented by transmitting at a very low bitrate
to hope everyone can hear it, taking up airtime on access points,
instead of transmitting at high rates with an ACK mechanism like unicast
frames. (*)

If the WLAN implements MLD snooping, an NDP broadcast is unlikely to be
listened to by more than one host; a smart AP could deliver it like a
unicast frame at a high rate to said single client. The other APs in the
same L2 network can drop the frame on the floor altogether, or never see
it if the wired network has MLD snooping. But even in this case it
scales better to have access points throw away a small amount of frames
than have the whole wired switch network learn a large amount of
multicast groups that churn each time the client roams to a new AP.

* I am aware this is a simplification, and many modern WLANs are
smarter than this; many also do proxy ARP to eliminate the problem with
flooded ARP broadcasts altogether.

> Also, if a switch does not do MLD snooping, it will flood multicast to
> all ports. You lose one of the major benefits of IPv6 multicast - less
> admin traffic.
NDP multicast has scaling issues, and I'd not be surprised if switches
will soon stop learning it and flood all NDP multicasts to save space
for the users' higher-traffic multicast groups.

Can you be more specific about these scaling issues? Seems to me that
each node is in relatively few multicast groups - one per interface (all
link-local hosts), plus one per address (solicited node multicast), less
if SLAAC is being used, because one SNMA is used for both the link local
address and the SLAAC address. Some nodes may be participating in other
groups - routers, for example, will also be in the all link-local
routers group, and maybe things like the DHCPv6 all servers and relays
group. If the node is doing temporary addressing, there will be an
additional solicited node multicast address in play during the
changeover. So a typical node in a subnet will be in three, maybe four
groups. I'm guessing that's NOT the scalability problem you are talking
about.

if it is interesting. NDP multicast addresses, on the other hand, allow
for the device to program only the multicast MACs it is interested about
in the ethernet chipset, so the CPU will never see the useless packets.

Yep - belt and braces. But that multicast packet still went over the
wire as far as the NIC, and while it was doing that, other traffic was
not able to use the wire. So getting that multicast traffic off the wire
altogether is a Good Thing, and the place for that filtering to happen
is in the switch.

unduly burdened. You will also not need to burden your network with
multicast groups (=state) to save hauling a few useless packets around.

As long as it's a few, true. But one of the aims of moving to multicast
was to enable larger subnets. That "few useless packets" can turn into a
LOT of useless packets when there are a few hundred or a few thousand
nodes on the subnet.

If the WLAN implements MLD snooping, an NDP broadcast is unlikely to be
listened to by more than one host; a smart AP could deliver it like a
unicast frame at a high rate to said single client.

How does the behaviour of this AP differ in principle from the behaviour
of a switch doing MLD snooping and delivering multicast packets only to
listeners in the particular group (and for the same reason)?

than have the whole wired switch network learn a large amount of
multicast groups that churn each time the client roams to a new AP.

Why is it a large amount? See above - it's probably three or four per
host. And they only churn when a client moves into or away from a
connection point (AP or switch port). Most things connected to switch
ports won't churn that much.

Regards, K.

High density virtual machine setups can have 100 VMs per host. Each VM
has at least a link-local address and a routable address. This is 200
groups per port, 9600 per 48 port switch. This is a rather large amount
of state for what it's worth. If you have mld snooping on a switch
aggregating multiple racks like this, you start hitting limits on some
platforms. There is a similar situation with a WLAN that has large
amounts of clients; a single AP, on the other hand, should not see that
many groups.

Multicast always requires state in the whole network for each group, or
flooding. In the case of ndp, flooding may very well be the better
option, especially if you view this as a DoS to your Really Important
multicast groups - some virtual hosters give /64 per VM, which brings
about all kinds of trouble not limited to multicast groups if the client
decides to configure too many addresses to his server.

High density virtual machine setups can have 100 VMs per host.

OK, I see where you are coming from now.

Hm. If you have 100 VMs per host and 48 hosts on a switch, methinks you
should probably invest in the finest switches money can buy, and they
will have no problem tracking that state. While it is certainly a hefty
dose *more*, it is geometrically, not exponentially more, so not a
scalability issue IMHO. An ordinary old IPv4 switch tracks 4800 L2/port
mappings in the scenario you describe. If each VM had just two
addresses, it would be 9600...

I wonder if there is hard information out there on how many multicast
groups a modern switch can actually maintain in this regard. Are you
saying you have seen actual failures due to this, or are you supposing?
Serious question - is this a possible problem or an actual problem?

multicast groups - some virtual hosters give /64 per VM, which brings
about all kinds of trouble not limited to multicast groups if the client
decides to configure too many addresses to his server.

There is always some way to misconfigure a network to cause trouble.
It's a bit unfair to make that IPv6's fault.

As a matter of interest, what is the "all kinds of trouble" that a
client can cause by configuring too many addresses on their server?
Things that are not the client-side problems, obviously :wink:

Regards, K.

What make+model switches would these be, did you say?

Nick

Oooh, you've got me there :slight_smile:

My point was just that the additional state did not, as described, seem
to me like such a massive amount more than that presently required. I
thus doubted that lack of state capacity in switches would be a real (as
distinct from a possible, likely or supposed) problem. If there *are*
actual problems, then obviously I stand corrected.

I don't mind being corrected, but I would be sad to see this aspect of
IPv6 go by the board. I suspect that even if it *is* an actual problem
now, it will turn out to be a transient one - switches will get more
capacity and will just deal with it. Even if low-end switches don't, I
expect that high-end switches will.

But if it turns out that even the high end of the market doesn't care
about traffic reduction but does want cheaper switches, then I guess
we'll see a lot of non-MLD-snooping switches.

Regards, K.

um - let's compare apples to apples here - 100 VMs per host, 9600 per
48 port switch, is a problem for IPv4 also...

A trident+ or trident2 switch can't support that, a tor would do something like 8k arp entries or 4k ndp cache entries, probably less.

The level of IPv6 support in firewalls has been all over the place, even from vendors who have known IPv6 was coming for a long time :wink:

I published a minimum IPv6 firewall ruleset for Cisco ASAs a while back on some other lists and got only a little feedback, so for the benefit of the NANOG community, I offer up:

http://www.cluebyfour.org/ipv6/

I will be testing the transition from 8.x to 9.x code in my lab as soon as this week, so I should have some updated to publish very soon.

Likewise, I'm in the process of getting a DHCPv6 server spun up as well, so I'll have some updates to publish there as well.

As always, suggestions and constructive feedback are always welcome.

jms

The smarter way to do this is to assign a /64 to each host and route
to it instead of exporting any L2 issues beyond the TOR switch.

In general, WLANs don't scale to large numbers of clients particularly
well for a variety of reasons that have little to do with ND. More
APs with smaller range are almost always a better solution.

Owen