IPv6 Wow

Mailman List Mirror (Read Only)
1

I'm in LA with Time Warner Cable - didn't know they rolled out an IPv6
link to AMS-IX.

HOST: macbook.local Loss% Snt Last Avg Best Wrst StDev
  1. 2002:4ca6:18c5::21b:63ff:fef 0.0% 10 0.8 1.5 0.7 4.6 1.2
  2. 2002:82f4:21::1 50.0% 10 185.4 188.3 185.4 197.1 5.0
  3. 2a00:800:0:1::2:2 0.0% 10 219.8 215.3 205.8 229.9 8.6
  4. ams-ix.he.net 0.0% 10 200.9 192.0 187.0 201.9 5.6
  5. 10gigabitethernet1-4.core1.l 0.0% 10 195.7 198.1 192.8 214.1 6.5
  6. 10gigabitethernet2-3.core1.n 0.0% 10 275.4 266.4 261.7 275.4 3.8
  7. 10gigabitethernet3-1.core1.s 0.0% 10 344.4 345.1 342.4 351.0 2.4
  8. 10gigabitethernet3-2.core1.p 0.0% 10 350.8 350.0 342.4 364.5 7.5
  9. ??? 100.0 10 0.0 0.0 0.0 0.0 0.0

2

Hello,

That is 6to4. You can tell due to the (2002::slight_smile: prefix.

William Pitcock
SystemInPlace

3

And using a relay at Tele2 AB (Sweden).

So really Max, all your outbound IPv6 packets are going via Sweden, even if they're going to elsewhere in the US.

In order to get around this, encourage your ISP to build a 6to4 relay, which is a couple of commands on a spare Cisco router. For extra points, get them to build out a Teredo relay as well, which is a few commands on a spare Linux box.

4

And using a relay at Tele2 AB (Sweden).

So really Max, all your outbound IPv6 packets are going via Sweden, even if they're going to elsewhere in the US.

Well, actually, Tele2 has 6to4 relays in Stockholm, Paris, Amsterdam and Frankfurt. But correct, it's a relay in Europe that is being used.

In order to get around this, encourage your ISP to build a 6to4 relay, which is a couple of commands on a spare Cisco router. For extra points, get them to build out a Teredo relay as well, which is a few commands on a spare Linux box.

We have very good experience with 7600 for 6to4 relay usage.

This brings up an interesting question, should we stop announcing our 6to4 relays outside of Europe? Is there consensus in the business how this should be done? I have heard opinions both ways.

5

Mikael Abrahamsson wrote:

This brings up an interesting question, should we stop announcing our 6to4 relays outside of Europe? Is there consensus in the business how this should be done? I have heard opinions both ways.

I can understand why some folks would say stop, but unfortunately Europe has the closest public 6to4 relays to the US, and our own providers don't seem to want to put any up. That means 6to4 will break for a great many folks who _are_ trying to use IPv6 (like developers trying to get ahead of the curve and make sure their apps don't break when the transition finally happens) but whose providers haven't clued in yet.

(My traceroutes to 192.88.99.1 have a next-to-last hop in Amsterdam, and I'm on one of the largest ISPs in the US, which apparently hasn't figured out 6to4, much less native IPv6.)

S

6

Mikael Abrahamsson wrote:

This brings up an interesting question, should we stop announcing our 6to4 relays outside of Europe? Is there consensus in the business how this should be done? I have heard opinions both ways.

I can understand why some folks would say stop, but unfortunately Europe has the closest public 6to4 relays to the US, and our own providers don't seem to want to put any up. That means 6to4 will break for a great many folks who _are_ trying to use IPv6 (like developers trying to get ahead of the curve and make sure their apps don't break when the transition finally happens) but whose providers haven't clued in yet.

I'm sure I sound like a broken record to some, but whenever I see these comments I feel the need to step up and correct them, until I don't see them anymore.

By far the biggest end users of IPv6 are non-experimenters. Real end users, many of whom do not know what an IP address is.

6to4 is enabled by default in Vista - any Vista machine with a non-RFC1918 address will use 6to4. It is also available in some linksys routers, and is enabled by default in Apple Airport Extreme.

(My traceroutes to 192.88.99.1 have a next-to-last hop in Amsterdam, and I'm on one of the largest ISPs in the US, which apparently hasn't figured out 6to4, much less native IPv6.)

There are public 6to4 relays in the US, I guess your provider just has a shorter ASPATH to somewhere in Europe. Unfortunately BGP had no idea of geography :slight_smile:

Re. whether to advertise outside your continent, it really depends whether you're trying to achieve 'good enough' connectivity for 100% of people, or really good connectivity for 95% of people.

Perhaps a good way to do it is advertise outside Europe, but have the providers that get your advertisement out there prepend their AS a few times as it leaves. That way, US providers will still prefer US 6to4 relays (ie lower latency) but any who don't get a 192.88.99.0/24 route from the US will us your relay in Europe. Kinda gets you best of both worlds.

7

There are quite a few 6to4 relays outside Europe. I think the problem more is that a lot of providers haven't gotten the hang of routing to an anycasted prefix that so many people are announcing. Or making sure 6to4 is fast is so far down their priority list they just don't care. Either way, peering seems to be a lot more dense in Europe, so they're more likely to be peering with someone announcing a 6to4 relay there. Prefer sending traffic to someone you're peering with (even if it's further away) to someone local that you have to pay for, and you don't always get sensible routing on a prefix like this.

From the best that I can tell, here's a reasonably complete list of who are running 6to4 relays... Its not easy to tell what country someone's relay is in, but this is probably close.

Oceania/Asia:

Australia:
1221 Telstra

Korea:
17832 NISA

Europe:

Denmark:
1835 FSK Net

Estonia:
3327 Linxtelecom

Finland:
1741 FUNET

Germany:
286 kpn.de
5430 Freenet
8767 m-net.de
12816 mwn
15598 IP Exchange
20640 Titan
29259 IABG Teleport
35244 kms.de

Italy:
12779 itgate.net

Netherlands:
1101 SURFNet
8954 InTouch
26943 Your.Org
31383 Computel

Portugal:
1930 FCCN

Spain:
16206 Abared

Sweden:
1257 Tele2
16150 GlobalTransit

Switzerland:
559 switch.ch

United Kingdom:
5400 BT

North America:

US:
59 University of Wisconsin
109 Cisco
1239 Sprint
3344 Kewlio
5050 Pittsburgh Supercomputing Center
6175 Sprint
7019 NTT
10533 Ottawa Internet Exchange
19255 Your.Org
19782 Indiana University
25795 ARP Networks

8

Not to rain on anyone's parade, but it'd be interesting (and difficult, unfortunately) to know how many Vista machines are actually on non-RFC1918 addresses. Corporate users are in many cases staying with XP for a while, but they're more likely to have public space than most. A great many home users have a cheap NAT box that provides RFC1918 addresses.

I do wonder whether where the Vista machines on public IPs really are. I also have to wonder if performance is really better when those users are routed over 6to4 in Europe from, say California, or whether they'd actually get better performance if they stuck in a NAT box, resulting in their using IPv4 instead?

9

Don't worry, you're not raining on my parade if that's what you're concerned about. I don't like Vista/XPSP2 having 6to4, Teredo is the protocol designed to connect end hosts to the IPv6 network. That works through NAT, and is enabled by default on Vista.
6to4 should existing in CPE devices, etc. not in end hosts. Cue religious war.

Also, Windows boxes that are part of a domain will only try ISATAP and native IPv6 - they will not attempt to tunnel IPv6 over IPv4 using public relays (ISATAP is an internal thing).

I did a bit of stats, and roughly 95% of packets leaving an ISP's aggregation layer were from hosts behind NAT (look at TTL, make assumptions based on initial TTL). So, 6to4 is only on 5% of customers, assuming that % of packets and % of customers are roughly equal.

Here's a mini-rant I had about Teredo traffic offlist when someone said they had very little 6to4 traffic. I thought it was on-list.

begin.
I suspect you'll find that Teredo contributes to a very large amount of it, but you won't be seeing it as you don't have a local Teredo relay (in my understanding of your network, anyway :slight_smile:
Even then you won't see Teredo<->Teredo, or Teredo<->NonTeredo when NonTeredo is on another network.

An interesting way to get a rough idea of how much Teredo<->NotTeredo is going on is to look at the packets going to teredo.ipv6.microsoft.com port 3544/UDP.
Every Vista/XPSP2 Teredo client will send a UDP packet there every 30 seconds (IIRC), and then another packet for every new NonTeredo host it wants to talk to. Source UDP port is generally static and unique for each client host, so you can get an idea for unique number of hosts.

The periodic packets are going to be 68b (of IPv4+UDP+IPv6 = 68b), whereas the new-connection packets are going to be at least 76b (IPv4+UDP+IPv6+ICMPv6+Echo Request = 76b, then there's also the ICMPv6 Echo Request payload). Obviously you want to add 14b if you've got ethernet headers and what not.

If you have netflow anywhere, you should be able to ask it an appropriate question with the above info.

That'll tell you number of end-to-end connections there are which may give you some insight there.
If you've got a netflow exporter, I'd be more than happy to run stats over the data to figure out what amount of Teredo there is.
end.

10

6to4 is enabled by default in Vista - any Vista machine with a non-RFC1918 address will use 6to4. It is also available in some linksys routers, and is enabled by default in Apple Airport Extreme.

I've been told there is a difference between OEM and non-OEM Vista machines when it comes to Teredo being activated or not.

Perhaps a good way to do it is advertise outside Europe, but have the providers that get your advertisement out there prepend their AS a few times as it leaves. That way, US providers will still prefer US 6to4 relays (ie lower latency) but any who don't get a 192.88.99.0/24 route from the US will us your relay in Europe. Kinda gets you best of both worlds.

Yeah, that's been one option as well, prepending 2-3 times to our peers/transit in the US is probably a good middle way.

Regarding some numbers on 6to4 and Teredo usage, I'd like to point people to this thread on the ipv6ops IETF list:

<http://www.ops.ietf.org/lists/v6ops/v6ops.2008/msg01582.html&gt;

If someone has some nice code that'll take a list of IPv6 addresses and break it down to geographical distribution of native/teredo/6to4, I'd be more than happy to run it on my data.

11

I'd say it's very rare where IPv6 will give you better performance than IPv4 right now.

Regarding where they are, I'd say all over the place. It's very common in my regional market to hand out one or more public IPs, and if the customer doesn't put their own NAT box there, then their Vista computer(s) will have public IPs and will use 6to4.

Regarding 6to4 or Teredo, I've done some testing of my own and the statelessness of 6to4 makes it avoid some of the session setup/NAT travesal magic of Teredo that slows Teredo down. I'd much rather see the NAT boxes do 6to4 and run native on their local LAN segment, than having end hosts do Teredo to get thru the NAT. It'll give the end user a better IPv6 experience.

12

Long term I agree, but short term I prefer Teredo for regular end users' experience. Where regular end user means an end user communicates with a relatively small number of remote hosts.

Several reasons:
1) 6to4 currently lacks a testing mechanism to ensure that it is functioning at startup, and that it is still functioning. Packets are sent and blackholed by the network, resulting in a 90s timeout waiting for a response to the three SYN packets in a TCP connection set up. 90s is a long time for users today, and my experience shows that they consider a service to be 'broken' before they wait for the timeout to expire.
2) If Teredo relays are deployed close to the service (ie. content, etc.) then performance is almost equivalent to IPv4. 6to4 relies on relays being close to both the client and the server, which requires end users' ISPs to build at least *some* IPv6 infrastructure, maintain transit, etc. When you consider that this infrastructure and transit is quite likely to be over long tunnels to weird parts of the world, this is a bad thing. Putting relays close to the content helps for the reverse path (ie. content -> client), however the forward path (client -> content) is likely to perform poorly.

13

I do wonder whether where the Vista machines on public IPs really are. I also have to wonder if performance is really better when those users are routed over 6to4 in Europe from, say California, or whether they'd actually get better performance if they stuck in a NAT box, resulting in their using IPv4 instead?

I'd say it's very rare where IPv6 will give you better performance than IPv4 right now.

Rare = Absolutely Yes. Impossible = No :slight_smile:

Regarding where they are, I'd say all over the place. It's very common in my regional market to hand out one or more public IPs, and if the customer doesn't put their own NAT box there, then their Vista computer(s) will have public IPs and will use 6to4.

Regarding 6to4 or Teredo, I've done some testing of my own and the statelessness of 6to4 makes it avoid some of the session setup/NAT travesal magic of Teredo that slows Teredo down. I'd much rather see the NAT boxes do 6to4 and run native on their local LAN segment, than having end hosts do Teredo to get thru the NAT. It'll give the end user a better IPv6 experience.

Fully agree. Unfortunately not every NATbox/cheap-consumer-router is happy to pass on 6to4 packets to its next hop :frowning:

--
Mikael Abrahamsson email: swmike@swm.pp.se

Cheers,

14

6to4 is enabled by default in Vista - any Vista machine with a non-RFC1918 address will use 6to4. It is also available in some linksys routers, and is enabled by default in Apple Airport Extreme.

I've been told there is a difference between OEM and non-OEM Vista machines when it comes to Teredo being activated or not.

I've not heard that, I'll be interested if someone can confirm?

Perhaps certain vendors disable IPv6 because of the lack of relays etc. in the network causing performance problems.

Perhaps a good way to do it is advertise outside Europe, but have the providers that get your advertisement out there prepend their AS a few times as it leaves. That way, US providers will still prefer US 6to4 relays (ie lower latency) but any who don't get a 192.88.99.0/24 route from the US will us your relay in Europe. Kinda gets you best of both worlds.

Yeah, that's been one option as well, prepending 2-3 times to our peers/transit in the US is probably a good middle way.

Regarding some numbers on 6to4 and Teredo usage, I'd like to point people to this thread on the ipv6ops IETF list:

<http://www.ops.ietf.org/lists/v6ops/v6ops.2008/msg01582.html&gt;

If someone has some nice code that'll take a list of IPv6 addresses and break it down to geographical distribution of native/teredo/6to4, I'd be more than happy to run it on my data.

Teredo and 6to4 is easy - translate the addresses to IPv4 and geolocate with maxmind geoip or something.
IPv6 is harder, you have to build a geolocation database, which I suppose you'd have to build from either origin AS location, or whatever location data the RIR has, for the prefix an address is in.

I have been intending on building a map from Teredo and 6to4 using IPv6 addresses from my bittorrent population stuff, and have that as one output of a periodic study. I'm not sure how to do non-Teredo/6to4 addresses though, so if you've got some ideas there I'll whip something up, the Teredo/6to4 stuff is very simple.

15

The problem is that every tunneling mechanisms is selecting detination without the real knowldege about the underlying technology/distance etc. It was horrible during the 6bone - documented by Pekka Savola. We are not learning, from the past... 6to4 can generate same amount of problem....

Basically if they would obey the default address selection rules they would use 6to4 addresses only if there would be no global addressess and a resource would be acessible only from IPv6.

This is the intended and recommended behaviour which is implemented by Windows (XP, Vista), *BSD systems and recent Linux systems.

Unfortunately there is a broken idea of Apple to not implment RFC 3484 style Default Address Selection into the protocol stack, however it is implemented in its ancestors (*BSD + KAME) for more than 4 years now.

Best Regards,

Janos Mohacsi
Network Engineer, Research Associate, Head of Network Planning and Projects
NIIF/HUNGARNET, HUNGARY
Key 70EF9882: DEC2 C685 1ED4 C95A 145F 4300 6F64 7B00 70EF 9882

16

If someone has some nice code that'll take a list of IPv6 addresses and
break it down to geographical distribution of native/teredo/6to4, I'd be
more than happy to run it on my data.

I have some code for doing breakdowns of IPv6 addresses which runs
on web logs. It was written for my own consumption, but I've put
some instructions on how to use it here:

  Scripts for analysing IPv6 addresses

If anyone is interested in playing with it, I'd be interested to
see how it does.

  David.

17

Nathan Ward wrote:
...

2) If Teredo relays are deployed close to the service (ie. content,
etc.) then performance is almost equivalent to IPv4. 6to4 relies on
relays being close to both the client and the server, which requires
end users' ISPs to build at least *some* IPv6 infrastructure, maintain
transit, etc. When you consider that this infrastructure and transit
is quite likely to be over long tunnels to weird parts of the world,
this is a bad thing. Putting relays close to the content helps for the
reverse path (ie. content -> client), however the forward path (client
-> content) is likely to perform poorly.

Not quite correct. 6to4 does not require transiting a relay if the target is
another 6to4 site. What this means is that a clueful content provider will
put up a 6to4 router alongside whatever native service they provide, then
populate the dns with both the native and 6to4 address. A properly
implemented client will do the longest prefix match against that set, so a
6to4 client will go directly to the content provider's 6to4 router, while a
native client will take the direct path. The only time an anycast relay
needs to be used is when the server is native-only and the client is
6to4-only.

Tony

18

Not quite.
Say the server has native IPv6 address 2001::1 and 6to4 IPv6 2002::X.
Say the client has native IPv6 address 2003::1 and 6to4 IPv6 2002::Y.
Longest prefix match will choose 6to4 over native IPv6. Not good.

  - Alain.

19

Surely the longest prefix only wins over prefixes that cover the long prefix.

What you seem to be suggesting is that a client will somehow be able to deduce the prefix length for two bare v6 addresses received from the DNS, and will choose the one with the longest prefix first. That seems unlikely to be the case in general.

Joe

20

Alain Durand wrote:

A properly
implemented client will do the longest prefix match against that set, so a
6to4 client will go directly to the content provider's 6to4 router, while a
native client will take the direct path.
    
Not quite.
Say the server has native IPv6 address 2001::1 and 6to4 IPv6 2002::X.
Say the client has native IPv6 address 2003::1 and 6to4 IPv6 2002::Y.
Longest prefix match will choose 6to4 over native IPv6. Not good.

Not quite. A properly implemented client will use the policy table first which by section 2.1 and 10.3 of RFC 3484 depref's 2002::/16 below 0::/0.
It's only if two addresses are very similar (as far as the OS can determine) that the "longest match" rule comes into play.

You should also be able to configure your operating system to depref or pref source/destination addresses as local site policy requires (avoiding tunnels,
preferring v4 for some sites, using 6to4 for other sites, and avoiding v6 all together for others and so on).