is your host or dhcp server sending dns dynamic updates for rfc1918?

according to http://root-servers.org/, dns transactions concerning rfc1918
address space are now being served by an anycast device near you (no matter
who you might be, or where.) there will eventually be official statistics,
but i thought i'd give everybody a chance to clean up their houses first.

on the instance ISC runs, i noticed that the log files were turning over
really fast. that is to say, really, really, really fast. see below.

-rw-r--r-- 1 root sys 10485795 Apr 18 12:11 update-1.log.33
-rw-r--r-- 1 root sys 10485850 Apr 18 12:19 update-1.log.32
-rw-r--r-- 1 root sys 10485794 Apr 18 12:26 update-1.log.31
-rw-r--r-- 1 root sys 10485846 Apr 18 12:33 update-1.log.30
-rw-r--r-- 1 root sys 10485787 Apr 18 12:41 update-1.log.29
-rw-r--r-- 1 root sys 10485830 Apr 18 12:48 update-1.log.28
-rw-r--r-- 1 root sys 10485776 Apr 18 12:55 update-1.log.27
-rw-r--r-- 1 root sys 10485873 Apr 18 13:02 update-1.log.26
-rw-r--r-- 1 root sys 10485847 Apr 18 13:09 update-1.log.25
-rw-r--r-- 1 root sys 10485830 Apr 18 13:17 update-1.log.24
-rw-r--r-- 1 root sys 10485783 Apr 18 13:24 update-1.log.23
-rw-r--r-- 1 root sys 10485871 Apr 18 13:31 update-1.log.22
-rw-r--r-- 1 root sys 10485794 Apr 18 13:39 update-1.log.21
-rw-r--r-- 1 root sys 10485866 Apr 18 13:46 update-1.log.20
-rw-r--r-- 1 root sys 10485821 Apr 18 13:54 update-1.log.19
-rw-r--r-- 1 root sys 10485843 Apr 18 14:01 update-1.log.18
-rw-r--r-- 1 root sys 10485831 Apr 18 14:08 update-1.log.17
-rw-r--r-- 1 root sys 10485809 Apr 18 14:16 update-1.log.16
-rw-r--r-- 1 root sys 10485765 Apr 18 14:23 update-1.log.15
-rw-r--r-- 1 root sys 10485802 Apr 18 14:31 update-1.log.14
-rw-r--r-- 1 root sys 10485853 Apr 18 14:39 update-1.log.13
-rw-r--r-- 1 root sys 10485779 Apr 18 14:46 update-1.log.12
-rw-r--r-- 1 root sys 10485822 Apr 18 14:54 update-1.log.11
-rw-r--r-- 1 root sys 10485864 Apr 18 14:59 update-1.log.10
-rw-r--r-- 1 root sys 10485770 Apr 18 15:03 update-1.log.9
-rw-r--r-- 1 root sys 10485801 Apr 18 15:07 update-1.log.8
-rw-r--r-- 1 root sys 10485795 Apr 18 15:14 update-1.log.7
-rw-r--r-- 1 root sys 10485810 Apr 18 15:22 update-1.log.6
-rw-r--r-- 1 root sys 10485762 Apr 18 15:29 update-1.log.5
-rw-r--r-- 1 root sys 10485844 Apr 18 15:37 update-1.log.4
-rw-r--r-- 1 root sys 10485813 Apr 18 15:45 update-1.log.3
-rw-r--r-- 1 root sys 10485806 Apr 18 15:53 update-1.log.2
-rw-r--r-- 1 root sys 10485769 Apr 18 16:00 update-1.log.1
-rw-r--r-- 1 root sys 10485853 Apr 18 16:07 update-1.log.0
-rw-r--r-- 1 root sys 8108342 Apr 18 16:13 update-1.log

what these files are is a whole lot of lines that look like (broken by me):

18-Apr-2002 16:16:05.491 security: notice: \
  denied update from [63.198.141.30].2323 for "168.192.in-addr.arpa" IN

by "a whole lot" i mean we've logged 3.3M of these in the last four hours.

so who are these people and why are they sending dynamic updates for rfc1918
address space PTR's? second answer first: it's probably Windows' fault.
after a successful DHCP transaction, the corresponding A RR and PTR RR have
to be updated. if rfc1918 is in use, dns transactions about these PTR's
ought to be caught and directed toward some local server, who can do something
useful with them. this local capture often does not occur, and so these
dns transactions end up coming to us.

now as to who's responsible, first off you have to understand that we block
rfc1918-sourced packets at our AS boundary. (otherwise these numbers would
be Much Higher, but tracking them back to their source is Too Hard.) the
list of broken hosts (or hosts inside broken firewalls, depending on how you
look at it and depending on how DHCP is configured), can be seen here. if
histogrammed as /32's, the more-than-5000-times club has the following members:

  /32 31049 65.185.183.173
  /32 21293 66.36.29.99
  /32 16498 206.13.58.232
  /32 11983 67.80.208.88
  /32 11679 210.76.208.42
  /32 11188 64.131.161.29
  /32 10878 212.247.56.33
  /32 10650 194.209.225.5
  /32 9455 24.153.174.135
  /32 8140 64.94.107.2
  /32 8110 64.180.126.207
  /32 7992 65.66.33.249
  /32 7959 211.74.245.10
  /32 7864 64.221.109.114
  /32 7740 214.1.35.254
  /32 6842 156.1.60.60
  /32 6593 202.103.200.224
  /32 6232 212.219.116.67
  /32 5901 66.105.121.2
  /32 5113 130.67.113.72

viewing the results through a /24 shaped lense, the winners are:

  /24 32180 65.185.183
  /24 22018 66.36.29
  /24 16986 206.13.58
  /24 12361 67.80.208
  /24 12065 210.76.208
  /24 11641 64.131.161
  /24 11319 212.247.56
  /24 11057 194.209.225
  /24 10056 24.153.174
  /24 8661 64.180.126
  /24 8365 64.94.107
  /24 8262 64.221.109
  /24 8254 211.74.245
  /24 8124 65.66.33
  /24 7975 214.1.35
  /24 6957 156.1.60
  /24 6657 202.103.200
  /24 6473 212.219.116
  /24 6107 66.105.121
  /24 5850 65.69.149
  /24 5595 151.196.98
  /24 5299 130.67.113
  /24 5101 218.5.65
  /24 5034 139.130.213

these aren't cidr blocks, just a cheesy perl script. better detail will come
later, when the statistics are officialized and centralized amongst the various
anycasted instances of these "servers". anyone who'd like to think about ways
to not appear in those later stats should check out the full list of /32's at:

  ftp://ftp.isc.org/isc/slash32.txt

these are just the addresses who route toward ISC's servers; if your nets are
"closer to" (in routing terms) VeriSign, WIDE, or Autonomica, you won't be in
this list, so, your public ridicule can come later on instead of today.

Paul,

now as to who's responsible, ...

I hate to say it, but "Microsoft". This is the default for w2k and the like. The interesting thing is that it's got a very short timer for retries and hence why your logs are so big. I found this...

http://www.isc.org/ml-archives/bind-users/2001/02/msg01806.html

http://www.domainregistry.ie/tech/dynamic-dns.html

       Windows 2000
       The option can be found from:
       Click Start -> Settings -> Network and Dialup
       View the Properties of Local Area
       Select Adapter -> Protocols -> TCP/IP -> Advanced -> DNS
       The "Register this name" option box should be clear.

...the later would have to be done on millions of boxes around the world.

I wanted to add a flag to bind to "silently ignore" these requests, but alas this is not a good solution for reverse-dns private space.

I also thought that w2k and the like should not do a dynamic dns update if it's on private IP space, but that's not a valid test either, as the "enterprise" may well only exist in private IP space. (Yes... they should run their own zone for the reverse dns).

Martin

now as to who's responsible, first off you have to understand that we block
rfc1918-sourced packets at our AS boundary. (otherwise these numbers would
be Much Higher

are you sure? i suspect they are windows 2000 systems behind NATs. so
the dynamic update is for the 1918 address, but the packet source address
has been natted into real space.

randy

Paul,

> now as to who's responsible, ...

I hate to say it, but "Microsoft". This is the default for w2k and the like. The interesting thing is that it's got a very short timer for retries and hence why your logs are so big. I found this...

http://www.isc.org/ml-archives/bind-users/2001/02/msg01806.html

http://www.domainregistry.ie/tech/dynamic-dns.html

.. time for a BCP, perhaps?

I also thought that w2k and the like should not do a dynamic dns update if it's on private IP space, but that's not a valid test either, as the "enterprise" may well only exist in private IP space. (Yes... they should run their own zone for the reverse dns).

What _should_ happen IMHO is that this becomes an option thats off
by default, rather than on by default. The amount of time saved by admins
having this turned on is probably negated by the load placed on
bind servers all over the planet - perhaps someone should send M$ an
invoice.. :stuck_out_tongue:

Adrian

And right you are. However, pray tell, why doesn't bind feature a simple way
to not log these spurious updates? As far as I can tell lots of people want
to just ignore these messages but can only do so by turning off all security
logging.

Please note that PowerDNS is just as silly in this respect up to 1.99.9. The
next version features --log-failed-updates which defaults to off.

Regards,

bert

Maybe I'm stupid (it wouldn't be the first time).

Why do we bother having "public" nameservers answering for this space at all?

Why don't we have "blackhole-[12].iana.org" have A records of "127.0.0.1"? Then, if the local resolver doesn't have authority for that network, it'll loopback to itself looking for the answer (failing just as miserably as it would by beating up on the IANA.ORG servers, but without wasting anyone's bandwidth).

I'm sure there's a reason why we don't already do this (or something similar), but can someone educate me as to why that is?

D

If people set up their Win2K networks right, it wouldn't be a problem.
Simply install the MS DNS server, point their clients at that, then all the
updates go there. And if that DNS server has connectivity to the 'Net at
large, it will resolve all their other requests too by chasing the chain
from the root down.

Best of both worlds, or at least the best you can do in the situation ...

[snip]

what these files are is a whole lot of lines that look like (broken by me):

18-Apr-2002 16:16:05.491 security: notice: \
  denied update from [63.198.141.30].2323 for "168.192.in-addr.arpa" IN

by "a whole lot" i mean we've logged 3.3M of these in the last four hours.

so who are these people and why are they sending dynamic updates for rfc1918
address space PTR's? second answer first: it's probably Windows' fault.
after a successful DHCP transaction, the corresponding A RR and PTR RR have
to be updated. if rfc1918 is in use, dns transactions about these PTR's
ought to be caught and directed toward some local server, who can do something
useful with them. this local capture often does not occur, and so these
dns transactions end up coming to us.

[snip]

Does anyone already have a SNORT signature to match on these updates to
aid in tracking down which hosts behind a NAT are guilty for generating
this garbage?

The problem is that the sites that are the big offenders are probably not
the sort of sites that would run Snort.

Now, think about it - one /32 popped of *30K* of these in 4 hours -
and a 'dig -x' shows it to apparently be a DSL line. So we're seeing
2 or 3 DCHP events *PER SECOND* behind that NAT. Either they've got
a bunch of machines doing the Reboot Shuffle and have bigger problems,
or they're big enough that 2-3 DHCP per second is reasonable (at which
point you have to wonder how they're THAT big, and depending on a DSL
line.. :wink:

A) not all failed update attempts should be ignored
B) putting your head in the sand typically does not make a problem go away.

Rgds,
-drc

Why do we bother having "public" nameservers answering for this space at all?

Why don't we have "blackhole-[12].iana.org" have A records of "127.0.0.1"?

127.0.0.1 is a convention, not a standard. and to the extent that it is ever
upgraded to a standard, i don't think putting A RR's pointing at other folks'
hosts into the IANA.ORG domain would be a polite thing to do.

Besides, there's already an authoritative A record for 127.0.0.1:
warez.bofh.net. Or, um, at the moment it doesn't seem to be configured,
but that's historically been the correct answer.

                                -Bill

bert hubert wrote:

>
> according to http://root-servers.org/, dns transactions concerning

rfc1918

> address space are now being served by an anycast device near you (no

matter

> who you might be, or where.) there will eventually be official

statistics,

> but i thought i'd give everybody a chance to clean up their houses

first.

And right you are. However, pray tell, why doesn't bind
feature a simple way to not log these spurious updates? As far as I

can tell lots

of people want to just ignore these messages but can only do so by

turning

off all security logging.

Both bind8 & 9 can do seperate update logging:
http://www.crt.se/dnssec/bind9/Bv9ARM.ch06.html#AEN1548 "Logging
Statement Grammar"
The category 'update' is meant for Dynamic updates, one could either log
that to /dev/null or to a seperate file.
Which one could then filter for fails etc. If one chooses to log to
/dev/null succeeded updates won't be logged either.
But I'd rather see failed updates in my own reverse zones than all the
other crap, which makes the filter the best way.
Which is also most probably what Bert meant. It would be nice to be able
to ignore certain failures though and actually
have a more fine grained control of it all. One could always opt to use
the source and modify it ofcourse :wink:

As for the Win2k/XP dyndns updates; it's a great thing when one uses it,
if you don't simply either ignore all updates
from these boxes, fix them with that simple clickety click option, some
nice registry script on user-login and never forget the
power of policies.

Also those 10.in-addr.arpa. DNS servers should simply be 'deployed' by
the ISP's to block them off.
But then again I also think it's a good idea to block port 25 outgoing
on ISP's and force clueless users to use the ISP's relay
which would save us of many spams from around this world.

Btw... 'a' way to catch these things :wink:
I have these in place, but this host does apparently pop-up 48 times in
the /32 list, so Paul if you can pass me the entries in the logs for my
host (purgatory.unfix.org / 195.64.92.136).
I will be figuring it out as there are only 2 win2k here and both have
the dynamic dns stuff disabled, so they really shouldn't be poppin up in
yours...

Thou named.conf:
8<-------
   // RFC1918 Catches
   // 10.0.0.0/8
   zone "10.in-addr.arpa" { type master; file
"/etc/namedb/pz/rev/catch/10"; };
    // 192.168.0.0/16
   zone "168.192.in-addr.arpa" { type master; file
"/etc/namedb/pz/rev/catch/empty"; };
   // 172.16.0.0/12
   zone "16.172.in-addr.arpa" { type master; file
"/etc/namedb/pz/rev/catch/empty"; };
   zone "17.172.in-addr.arpa" { type master; file
"/etc/namedb/pz/rev/catch/empty"; };
   zone "18.172.in-addr.arpa" { type master; file
"/etc/namedb/pz/rev/catch/empty"; };
   zone "19.172.in-addr.arpa" { type master; file
"/etc/namedb/pz/rev/catch/empty"; };
   zone "20.172.in-addr.arpa" { type master; file
"/etc/namedb/pz/rev/catch/empty"; };
   zone "21.172.in-addr.arpa" { type master; file
"/etc/namedb/pz/rev/catch/empty"; };
   zone "22.172.in-addr.arpa" { type master; file
"/etc/namedb/pz/rev/catch/empty"; };
   zone "23.172.in-addr.arpa" { type master; file
"/etc/namedb/pz/rev/catch/empty"; };
   zone "24.172.in-addr.arpa" { type master; file
"/etc/namedb/pz/rev/catch/empty"; };
   zone "25.172.in-addr.arpa" { type master; file
"/etc/namedb/pz/rev/catch/empty"; };
   zone "26.172.in-addr.arpa" { type master; file
"/etc/namedb/pz/rev/catch/empty"; };
   zone "27.172.in-addr.arpa" { type master; file
"/etc/namedb/pz/rev/catch/empty"; };
   zone "28.172.in-addr.arpa" { type master; file
"/etc/namedb/pz/rev/catch/empty"; };
   zone "29.172.in-addr.arpa" { type master; file
"/etc/namedb/pz/rev/catch/empty"; };
   zone "30.172.in-addr.arpa" { type master; file
"/etc/namedb/pz/rev/catch/empty"; };
   zone "31.172.in-addr.arpa" { type master; file
"/etc/namedb/pz/rev/catch/empty"; };
   // 169.254.0.0/16 - Autoconfig zone Catch
   zone "254.169.in-addr.arpa" { type master; file
"/etc/namedb/pz/rev/catch/empty"; };
--------->8
As you see I defined the 10.0.0.0/24 seperatly as I have some lines with
'downstream' (read my local) NS's in there...

8<----------------
; BIND reverse data file for *.in-addr.arpa
; Reverse-Lookup zones for subnets:
; Empty - Catch
;
$TTL 604800
@ IN SOA purgatory.unfix.org.
dnsadmin.purgatory.unfix.org. (
           2000122901 ; Serial
                         604800 ; Refresh
                          86400 ; Retry
                        2419200 ; Expire
                         604800 ) ; Default TTL
                NS purgatory.unfix.org.
-------------->8
And use something like this for the empty, the $ORIGIN can be
irritating, otherwise generate seperate ones.
At least one can't lookup 'outside' dns's anymore and send updates there
unless they are specifically instructed too.

8<-------
jeroen@heaven:~$ host -t ns 13.100.10.in-addr.arpa.
13.100.10.in-addr.arpa NS purgatory.unfix.org
jeroen@heaven:~$ host -t ns 100.10.in-addr.arpa.
100.10.in-addr.arpa does not exist, try again
jeroen@heaven:~$ host -t ns 10.in-addr.arpa.
10.in-addr.arpa NS purgatory.unfix.org
jeroen@heaven:~$ host -t ns 18.172.in-addr.arpa.
18.172.in-addr.arpa NS purgatory.unfix.org
------->8
And ofcourse that box doesn't allow updates but does nicely deny them,
so no delay.

Someone prolly has a cleaner solution for filtering it...

On another note..... IPv6 has the fec0::/10 addresses and we also got
two reverses for that: 0.c.e.f.ip6.int & 0.c.e.f.ip6.arpa.
And we will probably be updating those soon too, so people better start
implementing their filters there and at some other addresses too :wink:
Fortunatly one can easily catch the f.ip6.int/arpa and is done then,
nibble reverses hooray.

Greets,
Jeroen

PS: for snoopy people, the above config is only the inside view, not the
outside, which doesn't recurse.

<snip>

what these files are is a whole lot of lines that look like (broken by me):

18-Apr-2002 16:16:05.491 security: notice: \
  denied update from [63.198.141.30].2323 for "168.192.in-addr.arpa" IN

by "a whole lot" i mean we've logged 3.3M of these in the last four hours.

I saw similar behavior on my little box (ns.bl.org) about a year or so ago,
logs have long since rotated out, so I don't recall exactly when, but there
was an IP somewhere in S. America trying to do a dyn update, something like
one attempt every two seconds.

I emailed the ISP, didn't get anything back, so I set up a black-hole
in BIND and stuck that /24 in it. A few days later, it was back,
from a different /24, but in the same /16, so I blackholed the /16.
Then again, from another /16, but the same ISP, so I blackholed it.

Haven't seen anything in a long time.

Explain how to fix everyone else's machines in the world. I host the domains owl.com and jove.com, among others, for clients. Apparently many people around the world would LIKE to own one or the other of these, and so program owl.com or jove.com into their Win2K machine setups. Those machines then bash the crap out of my name servers with dynamic updates.

We changed the MNAME in the SOA for those two domains to something that resolves to 127.0.0.1, and that took care of our load issue.

PLEASE understand the problem is NOT something people running the name servers have control over!

This is a totally irresponsible implementation on the part of Microsoft. It really reminds me of SNMP Management stations which would do network discovery by walking the entire IPv4 address space. Implementers of the products did not think through their designs, and many other people suffer for it.

I had a dynamic-dns client on my home ADSL system that was generating
requests at that rate a few months ago - I read logs and fixed it, don't
remember how... so this DOES happen ( and to people who do not read logs.. )

Bruce Williams
Benchmarks: Engineering wants to see how fast they can get the wheels to
spin on a car. Operations wants to know how fast the car will go. These
are different.

yeah, but windows isn't really targeted at the types of people who
do things "right" .. its targeted at the types of people who want
to do things "now".

I mean, if people set up their hosts/networks right - MTU path discovery
would actually still _work_ these days, spam wouldn't be a problem ..
DDoS just Wouldn't Happen(tm) ..

adrian