Automatic IPv6 due to broadcast

Mailman List Mirror (Read Only)
1

Hello everyone

Just got a awfully crazy issue. I heard from our support team about failure
of whois during domain registration. Initially I thought of port 43 TCP
block or something but found it was all ok. Later when ran whois manually
on server via terminal it failed. Found problem that server was connecting
to whois server - whois.verisign-grs.com. I was stunned! Server got IPv6
and not just that one - almost all. This was scary - partial IPv6 setup and
it was breaking things.

In routing tables, routes were all going to a router which I recently setup
for testing. That router and other servers are under same switch but by no
means I ever configured that router as default gateway for IPv6. I found
option of "broadcast" was enabled on router for local fe80... address and I
guess router broadcasted IPv6 and somehow (??) all servers found that they
have a IPv6 router on LAN and started using it - automated DHCP IPv6?

I wonder if anyone else also had similar issues? Also, if my guesses are
correct then how can we disable Red Hat distro oriented servers from taking
such automated configuration - simple DHCP in IPv6 disable?

Thanks

2

To completely disable ipv6 in Redhat:

1) Modify /etc/modprobe.conf (add)
  
  alias ipv6 off
  alias net-pf-10 off
  options ipv6 disable=1

2) Modify /etc/sysconfig/network (add)

   NETWORKING_IPV6=no

   I usually also add

   NOZEROCONF=yes

That should completely disable ipv6 in Redhat 5.x

3

Hi Anurag,

Hello everyone

<snip>

I wonder if anyone else also had similar issues? Also, if my guesses are
correct then how can we disable Red Hat distro oriented servers from taking
such automated configuration - simple DHCP in IPv6 disable?

Instead of disabling IPv6 on all the nodes in question you might be better off switching off Router Advertisements and investing a little bit of your time into what this IPv6 thing is.

Something on your network is advertising itself as the router, I suggest looking at it now, it won't magically fix itself.

If what is advertising itself as a IPv6 router is not the right device, disable. And while you are at it, setup one of which you know it is supposed to be one. It's really not all that hard anymore.

http://lists.pfsense.org/pipermail/list/2012-April/001942.html

And as I discovered a few days ago, I can't access github.com from my IPv6 only connection, which is a problem for software development. And for those wondering, the circuit is brand new, and no, it really doesn't have any IPv4.

Good thing that Google works though, and the website of my local Grocery store does too.

Regards,

Seth

4

More a host config issue than a NANOG issue, but what the heck...

I wonder if anyone else also had similar issues? Also, if my guesses are
correct then how can we disable Red Hat distro oriented servers from taking
such automated configuration - simple DHCP in IPv6 disable?

The *right* answer is, of course, to hurry up and deploy proper IPv6.

It sounds like the distro is shipping some apps that don't do happy-eyeballs yet - you
probably want to file bug reports about that.

(It's easy enough to disable IPv6 if you haven't deployed it - in
/etc/sysconfig/network-scripts/ifcfg-<interface> add:

IPV6INIT=no

then ifdown/ifup the interface and you should be good to go. Make sure you
remember to take that line out when you get around to deploying IPv6.

(The difference between this and Matthew Huff's suggestion is that this way,
it disables IPv6 on the interface, and leaves the ::1 loopback address in place.)

5

Anurag,

  You have a rogue RA in your network. Now is just an annoying DoS, but it can easily be turned in a real security concern.

  I suggest to either deploy properly IPv6 or disable it. I am more on the former, but it is your choice.

Regards
-as

6

I know you mentioned RedHat, but not if it was the router or other
servers. Were you playing with Microsoft's Direct Access and turn on
the dns entry (isatap.domain.com) internally?
At my current place of employment, we had a security student (at the
direction of our security analyst) turn up a DA test server. When they
enabled the DNS entry, just about every Windows 7 and 2008 server setup
a v6 tunnel back to this little tiny VM. This also included the DNS
entries in AD, so all of the sudden, servers have v6 addresses.
Needless to say, everything was horribly slow, and some things even
flat out broke. Sadly this event left a really sour taste for IPv6 with
Networking department (whom I was occasionally bugging about v6).

If you weren't testing this, did you possibly setup something similar
where it would automatically generate a tunnel?

  Brandon Penglase

7

Talking point: "If you guys had deployed a proper IPv6 infrastructure, those
tunnels wouldn't have happened and there would have been no problem".

The best defense against a user accidentally deploying some infrastructure
incorrectly is to do a pre-emptive correct deployment.

8

direction of our security analyst) turn up a DA test server.

<snip>

Needless to say, everything was horribly slow, and some things even
flat out broke.

To be expected when DNS is given the rôle of routing packets munged by tunneling in several layers of indirection.

Sadly this event left a really sour taste for IPv6 with
Networking department (whom I was occasionally bugging about v6).

The Notworking department should be driving the v6 deploy, if they want to network in the future. If they don't, replace them with a working Networking department.

9

IMO it's much easier to disable one rogue than to disable IPv6 on the
whole network. That is if you can find it, but with some proper
tcpdumping and/or CLI commands (depending on the switches that you have)
it should be relatively easy.

Not to mention that, as pointed by others, this provides a wonderful
opportunity to look into this new (*grin*) protocol.

Cheers!

~Carlos

10

I don't understand why a problem with a tunnel 'leaves a bad taste with
IPv6'. Since when a badly configured DNS zone left people with a 'bad
taste for DNS', or a badly configured switch left people with 'a bad
taste for spanning tree' or 'a bad taste for vlan trunking' ?

It seems to me that what are perceived as operational mistakes and/or
plain lack of knowledge for some technologies is perceived as a fault of
the protocol itself in the case of IPv6.

People need to get their acts together.

~Carlos

11

Op 17-4-2012 10:33, Carlos Martinez-Cagnazzo schreef:

IMO it's much easier to disable one rogue than to disable IPv6 on the
whole network. That is if you can find it, but with some proper
tcpdumping and/or CLI commands (depending on the switches that you have)
it should be relatively easy.

Even better, the IPv6 gateway you got assigned is based on the MAC address. That means you can also find what brand of device is advertising.

http://standards.ieee.org/develop/regauth/oui/public.html

You can most likely find which IPv4 address the MAC corresponds too as well. Was that so hard?

Not to mention that, as pointed by others, this provides a wonderful
opportunity to look into this new (*grin*) protocol.

Indeed!

Cheers!

~Carlos

Cheers,

Seth

12

You have a rogue IPv6 router on your network. It's not a host problem.
It's along the lines of having a rogue DHCP server on your network but
faster propagation.

It needs to be tracked down and disabled.

You can use tcpdump (as root) to capture IPv6 RA and see who's doing it,
and what's being advertised:

tcpdump -ni eth0 'ip6 dst ff02::1'

06:48:48.044409 IP6 fe80::2d0:1ff:fedf:8400 > ff02::1: ICMP6, router
advertisement, length 64

Then look at your IPv6 neighbor table for the MAC of that host:

ip -6 neigh show

fe80::2d0:1ff:fedf:8400 dev eth0 lladdr 00:d0:01:df:84:00 router REACHABLE

Once you have the MAC, track it down and disable it.

On a Cisco device "show mac address-table" (or "show mac-address-table" on
older hardware).

13

tcpdump -e will show source and dest mac address.

14

RA guard is useful if your tcam capacity and or switching platform allows -
http://tools.ietf.org/html/draft-ietf-v6ops-ra-guard-implementation-01

An older yet still a good read from Cisco on some IPv6 first hop security:
http://www.cisco.com/en/US/docs/ios/ipv6/configuration/guide/ip6-first_hop_security.html#wp1054246

Mick

15

Thanks for useful reply everyone!

As I mentioned - I applied quick temporary fix by stop broadcast from
router and clearing of routing table on servers. Will apply disabling of
autoconfig now.

16

I don't understand why a problem with a tunnel 'leaves a bad taste with
IPv6'. Since when a badly configured DNS zone left people with a 'bad
taste for DNS', or a badly configured switch left people with 'a bad
taste for spanning tree' or 'a bad taste for vlan trunking' ?

It seems to me that what are perceived as operational mistakes and/or
plain lack of knowledge for some technologies is perceived as a fault of
the protocol itself in the case of IPv6.

rogue dhcp servers are sufficiently common that tools had to be
developed to address their existence and they're still a nuisance after
all that.

People need to get their acts together.

indeed.

17

Most switches nowadays have dhcpv4 detection that can be enabled for port
ranges. Not sure about v6.

-Grant

18

Most switches nowadays have dhcpv4 detection that can be enabled for port

Yes. Many L2 switches have DHCPv4 "Snooping", where some port(s) can
be so designated as trusted DHCP server ports, for certain Virtual
LANs; and dhcp messages can be detected and suppressed from
unauthorized edge ports. Particularly good L2 switches also have
DAI or "IP Source guard" IPv4 functions, which when properly
enabled, can foil certain L2 ARP and IPv4 source address spoofing
attacks, respectively.

e.g. Source IP address of packet does not match one of the DHCP leases
issued to that port -- then drop the packet.

As for IPv6; rfc6105; you have
ipv6 nd raguard

and IOS NDP inspection.

However, there are caveats that should be noted. RA guard
implementations can be trivially fooled by the use of crafted packets.

These are potentially good protections against accidental
configuration errors, but not malicious attack from a general purpose
computer.

Currently, IPv4 seems to win at L2 easily in regards to the level of
hardware security features commonly available on L2 switches that
pertain to IP.

19

Most switches nowadays have dhcpv4 detection that can be enabled for port

Yes. Many L2 switches have DHCPv4 "Snooping", where some port(s) can
be so designated as trusted DHCP server ports, for certain Virtual
LANs; and dhcp messages can be detected and suppressed from
unauthorized edge ports.

Sounds suspiciously like IPv6 RA Guard, no?

  Particularly good L2 switches also have
DAI or "IP Source guard" IPv4 functions, which when properly
enabled, can foil certain L2 ARP and IPv4 source address spoofing
attacks, respectively.

e.g. Source IP address of packet does not match one of the DHCP leases
issued to that port -- then drop the packet.

Meh... I can see many cases where that might be more of a bug than feature.

Especially in environments where loops may be possible and the DHCP lease might
have come over a different path than the port in question during some network event.

As for IPv6; rfc6105; you have
ipv6 nd raguard

and IOS NDP inspection.

However, there are caveats that should be noted. RA guard
implementations can be trivially fooled by the use of crafted packets.

Frankly, I suggest dropping any RA that doesn't fit in a single packet as a simple solution to the crafted-packet issue. (The crafted packet attacks by and large depend on crafting it so that there are enough extension headers to put the RA header in the second or later fragment).

These are potentially good protections against accidental
configuration errors, but not malicious attack from a general purpose
computer.

If you have a malicious attack from a general purpose computer on your own LAN, you've already lost on multiple levels to some extent or other.

The most effective solution at that point is to identify, locate, and excise said attacker.

Currently, IPv4 seems to win at L2 easily in regards to the level of
hardware security features commonly available on L2 switches that
pertain to IP.

There was a time when one probably could have argued that Novell beat IP on that basis alone.

IPv4 loses when you consider that there are more than 3.2 billion people in the world. That people likely will need a minimum of 5 IP addresses each. That we also need to number infrastructure, routers, servers, sensor grids, etc.

IPv4 also loses when you consider the pervasiveness of debilitated IPv4 internetworking in favor of address conservation over the last 20 years.

Owen

20

You're only supposed to use those features on the port directly
connected to the end-system, or to a few end-systems via an unmanaged
office switch that doesn't have redundant uplinks. I.e. edge ports.