ISC BIND 9 breakage?

Did anyone else on CentOS 6 just have some DNS resolvers totally fall over?

I noticed that this command: dnssec-lookaside auto; was causing the issue. The issue occurred right at about 1PM EST.

I see this note in the ISC key file…

ISC DLV: See https://www.isc.org/solutions/dlv for details.

The fix is either to remove "dnssec-lookaside auto;" from the config or else set "dnssec-lookaside no;" and then reload named.

Nick

Oh, yes. I am aware.

I am asking if anyone has any info as to why it just randomly stopped running perfectly normally at exactly 1PM EST?

Thanks,
-Drew

a message of 97 lines which said:

Did anyone else on CentOS 6 just have some DNS resolvers totally fall over?

dlv.isc.org signatures just expired.

        # NOTE: The ISC DLV zone is being phased out as of February
        2017;

And yet some people still use it, it seems.

We just left the dnssec-lookaside auto; configuration in there. Probably because it specifically says in the documentation from ISC that it won't hurt anything to leave it in there...

# Configuring "dnssec-lookaside auto;" to activate this key is
        # harmless

Guess not?

Thanks,
-Drew

On the BIND Users list:

https://lists.isc.org/pipermail/bind-users/2020-March/102820.html

Yeah, looks like that comment should have been updated to “harmless until…”

Owen

Normally when there is an impending doom moment with BIND or another software release there is at least some amount of coverage of it.

Was this not announced or known in advance?

Thanks,
-Drew

It was accidental breakage of the RRSIGs on the dlv.isc.org zone.

More detail to follow tomorrow once I've had some sleep...

Ray Bellis
Director of DNS Operations, ISC.

It was a glitch with the re-signing of the zone. There should be a official
report sometime tomorrow. That said "dnssec-lookaside auto;" has been a no-op
in BIND since BIND 9.9.12, BIND 9.10.7, BIND 9.11.3 and a fatal configuration
error as of BIND 9.12.0. We didn’t want the DLV lookup traffic and provides no
benefit as the zone has been empty since 2017.

If you have dnssec-lookaside configured in named.conf please remove it otherwise
the DLV code in the validator has to cryptographically prove that DLV records don’t
exist before returning that the response is insecure. That requires talking to the
servers for dlv.isc.org. It does this every hour for a active validating resolver
that is still running DNSSEC lookaside validation.

Mark

Was it a "glitch" or someone just plain old forgot to do it?

forgot to re-sign the zone on dlv.isc.org or forgot to remove dnssec-lookaside from the config?

Not kidding here. People need to take responsibility for their configurations.

Nick

Nick Hilliard wrote:

forgot to re-sign the zone on dlv.isc.org or forgot to remove
dnssec-lookaside from the config?

Not kidding here. People need to take responsibility for their
configurations.

Anyone running BIND provided with CentOS 6 has a release from ~2012 (bind 9.8.2) and it is understandable why their documentation is out-of-date (like OP).

To get more recent bugs and fixes from ISC directly, install from ISC's copr:

https://copr.fedorainfracloud.org/coprs/isc/bind-esv/

On CentOS 7 I needed to install dnf and yum-plugin-copr first. I don't see these in the usual places for CentOS 6, so getting copr sources enabled is the first challenge.

ISC sources for other distros:

Mike

Our report is at:

<https://lists.isc.org/pipermail/bind-users/2020-March/102828.html>

Ray Bellis
Director of DNS Operations, ISC.

We had issues with that feature back in 2018. We disabled it since then
as a matter of course:

//dnssec\-lookaside auto;

Mark.