NANOG36-NOTES 2006.02.15 talk 1 ipv6fix (and boy, does it need it)

Morning intro notes–don’t forget to fill out
your SURVEYS!!!

six lightening talks signed up, should be very
cool. If you have slides, get them to Steve
Feldman start with!

Wireless movie after break should be cool to watch.
Ren? Steve mistakenly introduces her, she corrects
them. Don’t forget to give feedback via the Survey

2006.02.15 v6fix: Wiping the Slate Clean for IPv6
Kenjiro Cho, WIDE/IIJ, Ruri Hiromi, WIDE/Intec NetCore

Will be talking about their efforts to deploy
IPv6, called v6fix.

v6fix is an effort to solve problems in the current
v6 deployment.
focuses on v4/v6 dual stack environments.
it’s a technical analysis of real world problem
Kenjiro will talk about tools and measurements.

deployment status
majority of equipment out there is v6 available
from major vendors
still many applications and appliances just work
with v4
v6 is starting to get into various business fields
Many people lack knowledge/experience with v6.
when non-experts hit problems, they’re clueless.

Example: illiteracy.
Hotel internet systems have instructions for guest.
troubleshooting: if you have IPv6 enabled, please
disable IPv6–brochure in guest room.
Cause of problem: combination.
DNS redirection returns specific A record for AAAA
clients stub-resolver accepts the A for AAAA, can’t
get out.

Wiping the slate clean for the v6
faulty behaviours only 1% and combinatorial often, but
could be fatal to deployment.
slow fallback to v4 after v6 errors
misbehaving DNS resolvers
filtering of ICMPv6
DNS misconfigurations
poorly configured tunnels
lack of peering or v6 paths

v6fix activities (research group)
identify/analysze/solve real-world tech problems
in v6 deployment.
Enemy: “after disabling v6, my problems went away”
Cooperation needed between researches, implementers, ops.

v6fix topics
harmful effects of the on-link assumption.
misbehaving DNS servers and resolvers
slow fallback to v4 after v6 failures

case 1: DNS loop at hotel
real story of hotel internet system–went to same room,
DNS is intercepted, redirected to signup page
ipv6 users can’t get beyond first page
hotel instructions say to disable v6
erroneus DNS redirection system and stub-resolver
redirection system always returns specific A record
when getting non-A queries
client’s stub resolver queries AAAA for any address,
blindly accepts A return response.

case 2: DNS server slowdown
Japanse ISP
ISP upgraded a DNS cache to BIND9, recieved complaints
about slowdown.
recompiling BIND9 with --disable-ipv6, fixed problem,
reported to JANOG
Caused by older BIND9 w/o IPv6 connectivity
server w/o v6 connectivity always tries to talk over v6,
ends up failing back to v4 after timeouts
fixed in BIND9.2.5 and 9.3.1

Common factors
1 problems appear only with specific combinatorial conditions
2 implementors and operators didn’t notice until reported
3 even for professionals, not easy to track down problems.

Kenjiro Cho, Tools:
v6 tools and measurement results
Goal: to understand the macro-level v6 healthiness
current methodologies
wide area meaasuremetn of behaviours of 2nd/3rd level
DNS servers
dual stack issues

DNS server measurements of .jp domain
AAAA responses: 0.13% DNS servers can’t deal with
AAAA requests
Most are lame delegation type errors.
ignore AAAA queries
respond with RCODE 3 (“name error”) NXDOMAIN

dual-stack path analysis
measurement techniques specifically designed for
take measurements for v4 and v6 at same time
compare v6 results with v4 results
extract problems that exist in v6 only
dual-stack node discovery
create dual-stack node list by monitoring DNS AAAA replies.
dual stack ping
run ping/ping6 to target dual-stack nodes
select a few representative nodes per site (/48) by RTT
dual-stack traceroute
trace/trac6 to selected nodes
visula v6 MTU to look at issues
visualize path issues

distribution of v6/v4 RTTs
4000 ping targets v4 on x-axis, v6 on y axis
individual nodes far above unity line–leaf issues

paths and PMTU visualization
from NYSERNET to ARIN sites

Many of ARIN paths via jp! (lack of peering)

From ISC to ARIN sites–paths look much better, but
lots of blue == lots of tunnels

Abilene case: well known problem.
Abilene trying to encourage v6 adoption
no AUP, tunnel services for v6
but ended up with horrible v6 paths, mostly with tunnels
ISPs are reluctant to move to paid v6 connectivity
Abilene thinking about suspending its relaxed AUP for v6
tool tries to illustrate such issues, convince users to
move to native v6

dual stack traceroute to ABILENE from WIDE (v4 upper,
v6 lower)
similar RTTs/hops for v4/v6; native dual-stack paths

dual-stack trace to ABILENE from IIJ
similar RTTs, but different paths: currently more common

dualstack traceroue to ABILINE fro ES
v6 RTTs much larger than v4: roundabout tunnels

Conclusion: faulty behaviours are only 1% and often
combinatorial, but can be fatal to acceptance of v6
slow fallback to v4 on v6 errors

knowledge sharing
need to realize the dangers of harmful of adoption of v6
cooperation among researchers, implementers, and ops
need to act now, or will bring negative impact to v6

v6fix members, etc.
overview, documents, and fact database.

contact at
reports of issues are welcome