DNSSEC broken for login.microsoftonline.com

FYI our DNS requests to resolve login.microsoftonline.com are failing because of a DNSSEC error.

http://dnssec-debugger.verisignlabs.com/login.microsoftonline.com

http://dnsviz.net/d/login.microsoftonline.com/dnssec/

ns1 domain]$ drill -DT login.microsoftonline.com
Warning: No trusted keys were given. Will not be able to verify authenticity!
;; Domain: .
;; Signature ok but no chain to a trusted key or ds record
[S] . 172800 IN DNSKEY 257 3 8 ;{id = 19036 (ksk), size = 2048b}
. 172800 IN DNSKEY 256 3 8 ;{id = 62530 (zsk), size = 1024b}
Checking if signing key is trusted:
New key: . 172800 IN DNSKEY 256 3 8 AwEAAbgVvZmZibtBpha3AIykU0OY4gcCXTcskYJUxGsdmV/awfmKcHlSrjNMioSgy4sByj+HpcbsyrZVGPp+JBXzYwwuEF/6w1k7vKYTK6vMSqgVcgooNkfb5MaRF2y7MEpPxfStnfwu8knE24ExB0hYE1URxJ9CqB3zMSl/vicXYXXl ;{id = 62530 (zsk), size = 1024b}
[S] com. 86400 IN DS 30909 8 2 e2d3c916f6deeac73294e8268fb5885044a833fc5459588f4a9184cfc41a5766
;; Domain: com.
;; Signature ok but no chain to a trusted key or ds record
[S] com. 86400 IN DNSKEY 256 3 8 ;{id = 51797 (zsk), size = 1024b}
com. 86400 IN DNSKEY 257 3 8 ;{id = 30909 (ksk), size = 2048b}
[S] Existence denied: microsoftonline.com. DS
;; No ds record for delegation
;; Domain: microsoftonline.com.
;; No DNSKEY record found for microsoftonline.com.
;; No DS for login.microsoftonline.com.;; No ds record for delegation
;; Domain: login.microsoftonline.com.
;; No DNSKEY record found for login.microsoftonline.com.
[U] No data found for: login.microsoftonline.com. type A
;;[S] self sig OK; [B] bogus; [T] trusted
[ns1 domain]$

[ns1 domain]$ drill -DT medicare.gov
Warning: No trusted keys were given. Will not be able to verify authenticity!
;; Domain: .
;; Signature ok but no chain to a trusted key or ds record
[S] . 172800 IN DNSKEY 256 3 8 ;{id = 62530 (zsk), size = 1024b}
. 172800 IN DNSKEY 257 3 8 ;{id = 19036 (ksk), size = 2048b}
Checking if signing key is trusted:
New key: . 172800 IN DNSKEY 256 3 8 AwEAAbgVvZmZibtBpha3AIykU0OY4gcCXTcskYJUxGsdmV/awfmKcHlSrjNMioSgy4sByj+HpcbsyrZVGPp+JBXzYwwuEF/6w1k7vKYTK6vMSqgVcgooNkfb5MaRF2y7MEpPxfStnfwu8knE24ExB0hYE1URxJ9CqB3zMSl/vicXYXXl ;{id = 62530 (zsk), size = 1024b}
[S] gov. 86400 IN DS 7698 8 1 6f109b46a80cea9613dc86d5a3e065520505aafe
gov. 86400 IN DS 7698 8 2 6bc949e638442ead0bdaf0935763c8d003760384ff15ebbd5ce86bb5559561f0
;; Domain: gov.
;; Signature ok but no chain to a trusted key or ds record
[S] gov. 86400 IN DNSKEY 256 3 8 ;{id = 13175 (zsk), size = 1024b}
gov. 86400 IN DNSKEY 257 3 8 ;{id = 7698 (ksk), size = 2048b}
Checking if signing key is trusted:
New key: gov. 86400 IN DNSKEY 256 3 8 AQPCY4NZARQ0HDzGismy6sZdJ17o2+yzmZSkw6d9PeeJ8NCnw9atj4PGHO50LX1Hy0n4YimUcDEXHu+sI4MBaeTkHY3ilsC2kpWGGOFW2fkXn6XNvvPVRjwk04hDsEFphOXPPdoXWjXtQiTVYkFpgUbxJYo24/JxM5JuC4v0+qDmLQ== ;{id = 13175 (zsk), size = 1024b}
[S] medicare.gov. 3600 IN DS 16500 7 1 ea88786ecaa04e66322e4405b1c1a55e31485281
medicare.gov. 3600 IN DS 16500 7 2 43a0e12df89bb342c15229495cd2bc18dddce0d9fb315aeb5b06b0d849b9a3ee
;; Domain: medicare.gov.
;; Signature ok but no chain to a trusted key or ds record
[S] medicare.gov. 7200 IN DNSKEY 256 3 7 ;{id = 58988 (zsk), size = 1024b}
medicare.gov. 7200 IN DNSKEY 256 3 7 ;{id = 41714 (zsk), size = 1024b}
medicare.gov. 7200 IN DNSKEY 257 3 7 ;{id = 16500 (ksk), size = 2048b}
[S] medicare.gov. 20 IN A 23.213.71.152
;;[S] self sig OK; [B] bogus; [T] trusted

FYI our DNS requests to resolve login.microsoftonline.com are failing
because of a DNSSEC error.

There's no DS record for microsoftonline.com so you shouldn't have any
DNSSEC problems with it - my servers can resolve it OK. DNSvis doesn't
show any problems. The only thing which might cause trouble is the
SERVFAIL responses to DNSKEY queries flagged by the Verisign DNSSEC
debugger.

DNSSEC Debugger - login.microsoftonline.com

login.microsoftonline.com | DNSViz

Tony.

Also, ns2.bdm.microsoftonline.com is offline for about 12 hours

Also, ns2.bdm.microsoftonline.com is offline for about 12 hours

  The problems started yesterday, more than 12 hours ago.

Thanks.

FYI our DNS requests to resolve login.microsoftonline.com are failing
because of a DNSSEC error.

There's no DS record for microsoftonline.com so you shouldn't have any
DNSSEC problems with it - my servers can resolve it OK. DNSvis doesn't
show any problems. The only thing which might cause trouble is the
SERVFAIL responses to DNSKEY queries flagged by the Verisign DNSSEC
debugger.

  DNSvis did list 4 errors earlier.

  4 recursive DNS servers here still fail to resolve login.microsoftonline.com.

  I turned DNSSEC validation off on one and it then resolved correctly.

  dnssec-validation no;

  Thanks for the info. Our customers have reported that it does resolve at the Google public DNS servers also.

FYI our DNS requests to resolve login.microsoftonline.com are failing
because of a DNSSEC error.

There's no DS record for microsoftonline.com so you shouldn't have any
DNSSEC problems with it - my servers can resolve it OK. DNSvis doesn't
show any problems. The only thing which might cause trouble is the
SERVFAIL responses to DNSKEY queries flagged by the Verisign DNSSEC
debugger.

DNSvis did list 4 errors earlier.

4 recursive DNS servers here still fail to resolve login.microsoftonline.com.

I turned DNSSEC validation off on one and it then resolved correctly.

  dnssec-validation no;

Thanks for the info. Our customers have reported that it does resolve at the Google public DNS servers also.

  Drill run on one of our name servers shows that the error is

  Existence denied: microsoftonline.com

[ns1 domain]$ drill -k /tmp/rootkey -DT login.microsoftonline.com
;; Number of trusted keys: 2
;; Domain: .
[T] . 172800 IN DNSKEY 256 3 8 ;{id = 62530 (zsk), size = 1024b}
. 172800 IN DNSKEY 257 3 8 ;{id = 19036 (ksk), size = 2048b}
Checking if signing key is trusted:
New key: . 172800 IN DNSKEY 256 3 8 AwEAAbgVvZmZibtBpha3AIykU0OY4gcCXTcskYJUxGsdmV/awfmKcHlSrjNMioSgy4sByj+HpcbsyrZVGPp+JBXzYwwuEF/6w1k7vKYTK6vMSqgVcgooNkfb5MaRF2y7MEpPxfStnfwu8knE24ExB0hYE1URxJ9CqB3zMSl/vicXYXXl ;{id = 62530 (zsk), size = 1024b}
  Trusted key: . 143619 IN DNSKEY 256 3 8 AwEAAbgVvZmZibtBpha3AIykU0OY4gcCXTcskYJUxGsdmV/awfmKcHlSrjNMioSgy4sByj+HpcbsyrZVGPp+JBXzYwwuEF/6w1k7vKYTK6vMSqgVcgooNkfb5MaRF2y7MEpPxfStnfwu8knE24ExB0hYE1URxJ9CqB3zMSl/vicXYXXl ;{id = 62530 (zsk), size = 1024b}
Key is now trusted!
  Trusted key: . 143619 IN DNSKEY 257 3 8 AwEAAagAIKlVZrpC6Ia7gEzahOR+9W29euxhJhVVLOyQbSEW0O8gcCjFFVQUTf6v58fLjwBd0YI0EzrAcQqBGCzh/RStIoO8g0NfnfL2MTJRkxoXbfDaUeVPQuYEhg37NZWAJQ9VnMVDxP/VHL496M/QZxkjf5/Efucp2gaDX6RS6CXpoY68LsvPVjR0ZSwzz1apAzvN9dlzEheX7ICJBBtuA6G3LQpzW5hOA2hzCTMjJPJ8LbqF6dsV6DoBQzgul0sGIcGOYl7OyQdXfZ57relSQageu+ipAdTTJ25AsRTAoub8ONGcLmqrAmRLKBP1dfwhYB4N7knNnulqQxA+Uk1ihz0= ;{id = 19036 (ksk), size = 2048b}
  Trusted key: . 172800 IN DNSKEY 256 3 8 AwEAAbgVvZmZibtBpha3AIykU0OY4gcCXTcskYJUxGsdmV/awfmKcHlSrjNMioSgy4sByj+HpcbsyrZVGPp+JBXzYwwuEF/6w1k7vKYTK6vMSqgVcgooNkfb5MaRF2y7MEpPxfStnfwu8knE24ExB0hYE1URxJ9CqB3zMSl/vicXYXXl ;{id = 62530 (zsk), size = 1024b}
Key is now trusted!
  Trusted key: . 172800 IN DNSKEY 257 3 8 AwEAAagAIKlVZrpC6Ia7gEzahOR+9W29euxhJhVVLOyQbSEW0O8gcCjFFVQUTf6v58fLjwBd0YI0EzrAcQqBGCzh/RStIoO8g0NfnfL2MTJRkxoXbfDaUeVPQuYEhg37NZWAJQ9VnMVDxP/VHL496M/QZxkjf5/Efucp2gaDX6RS6CXpoY68LsvPVjR0ZSwzz1apAzvN9dlzEheX7ICJBBtuA6G3LQpzW5hOA2hzCTMjJPJ8LbqF6dsV6DoBQzgul0sGIcGOYl7OyQdXfZ57relSQageu+ipAdTTJ25AsRTAoub8ONGcLmqrAmRLKBP1dfwhYB4N7knNnulqQxA+Uk1ihz0= ;{id = 19036 (ksk), size = 2048b}
[T] com. 86400 IN DS 30909 8 2 e2d3c916f6deeac73294e8268fb5885044a833fc5459588f4a9184cfc41a5766
;; Domain: com.
[T] com. 86400 IN DNSKEY 256 3 8 ;{id = 51797 (zsk), size = 1024b}
com. 86400 IN DNSKEY 257 3 8 ;{id = 30909 (ksk), size = 2048b}
[T] Existence denied: microsoftonline.com. DS
;; No ds record for delegation
;; Domain: microsoftonline.com.
;; No DNSKEY record found for microsoftonline.com.
;; No DS for login.microsoftonline.com.;; No ds record for delegation
;; Domain: login.microsoftonline.com.
;; No DNSKEY record found for login.microsoftonline.com.
[U] No data found for: login.microsoftonline.com. type A
;;[S] self sig OK; [B] bogus; [T] trusted

Actually login.microsoftonline.com is resolving but the CNAME it points to, login.microsoftonline.com.nsatc.net is not resolving because of a DNSSEC issue.

[ns1 ~]$ drill -k /tmp/rootkey -DT login.microsoftonline.com.nsatc.net CNAME
;; Number of trusted keys: 2
;; Domain: .
[T] . 172800 IN DNSKEY 257 3 8 ;{id = 19036 (ksk), size = 2048b}
. 172800 IN DNSKEY 256 3 8 ;{id = 62530 (zsk), size = 1024b}
Checking if signing key is trusted:
New key: . 172800 IN DNSKEY 256 3 8 AwEAAbgVvZmZibtBpha3AIykU0OY4gcCXTcskYJUxGsdmV/awfmKcHlSrjNMioSgy4sByj+HpcbsyrZVGPp+JBXzYwwuEF/6w1k7vKYTK6vMSqgVcgooNkfb5MaRF2y7MEpPxfStnfwu8knE24ExB0hYE1URxJ9CqB3zMSl/vicXYXXl ;{id = 62530 (zsk), size = 1024b}
  Trusted key: . 143619 IN DNSKEY 256 3 8 AwEAAbgVvZmZibtBpha3AIykU0OY4gcCXTcskYJUxGsdmV/awfmKcHlSrjNMioSgy4sByj+HpcbsyrZVGPp+JBXzYwwuEF/6w1k7vKYTK6vMSqgVcgooNkfb5MaRF2y7MEpPxfStnfwu8knE24ExB0hYE1URxJ9CqB3zMSl/vicXYXXl ;{id = 62530 (zsk), size = 1024b}
Key is now trusted!
  Trusted key: . 143619 IN DNSKEY 257 3 8 AwEAAagAIKlVZrpC6Ia7gEzahOR+9W29euxhJhVVLOyQbSEW0O8gcCjFFVQUTf6v58fLjwBd0YI0EzrAcQqBGCzh/RStIoO8g0NfnfL2MTJRkxoXbfDaUeVPQuYEhg37NZWAJQ9VnMVDxP/VHL496M/QZxkjf5/Efucp2gaDX6RS6CXpoY68LsvPVjR0ZSwzz1apAzvN9dlzEheX7ICJBBtuA6G3LQpzW5hOA2hzCTMjJPJ8LbqF6dsV6DoBQzgul0sGIcGOYl7OyQdXfZ57relSQageu+ipAdTTJ25AsRTAoub8ONGcLmqrAmRLKBP1dfwhYB4N7knNnulqQxA+Uk1ihz0= ;{id = 19036 (ksk), size = 2048b}
  Trusted key: . 172800 IN DNSKEY 257 3 8 AwEAAagAIKlVZrpC6Ia7gEzahOR+9W29euxhJhVVLOyQbSEW0O8gcCjFFVQUTf6v58fLjwBd0YI0EzrAcQqBGCzh/RStIoO8g0NfnfL2MTJRkxoXbfDaUeVPQuYEhg37NZWAJQ9VnMVDxP/VHL496M/QZxkjf5/Efucp2gaDX6RS6CXpoY68LsvPVjR0ZSwzz1apAzvN9dlzEheX7ICJBBtuA6G3LQpzW5hOA2hzCTMjJPJ8LbqF6dsV6DoBQzgul0sGIcGOYl7OyQdXfZ57relSQageu+ipAdTTJ25AsRTAoub8ONGcLmqrAmRLKBP1dfwhYB4N7knNnulqQxA+Uk1ihz0= ;{id = 19036 (ksk), size = 2048b}
  Trusted key: . 172800 IN DNSKEY 256 3 8 AwEAAbgVvZmZibtBpha3AIykU0OY4gcCXTcskYJUxGsdmV/awfmKcHlSrjNMioSgy4sByj+HpcbsyrZVGPp+JBXzYwwuEF/6w1k7vKYTK6vMSqgVcgooNkfb5MaRF2y7MEpPxfStnfwu8knE24ExB0hYE1URxJ9CqB3zMSl/vicXYXXl ;{id = 62530 (zsk), size = 1024b}
Key is now trusted!
[T] net. 86400 IN DS 35886 8 2 7862b27f5f516ebe19680444d4ce5e762981931842c465f00236401d8bd973ee
;; Domain: net.
[T] net. 86400 IN DNSKEY 257 3 8 ;{id = 35886 (ksk), size = 2048b}
net. 86400 IN DNSKEY 256 3 8 ;{id = 37703 (zsk), size = 1024b}
;; No DS for nsatc.net.;; No ds record for delegation
[B] ;; Error verifying denial of existence for name nsatc.net.NS: No DNSSEC signature(s)

cemacmini:~ curtis$ drill -k /tmp/rootkey -DT login.microsoftonline.com.nsatc.net CNAME
;; Number of trusted keys: 2
;; Domain: .
[T] . 172800 IN DNSKEY 256 3 8 ;{id = 62530 (zsk), size = 1024b}
. 172800 IN DNSKEY 257 3 8 ;{id = 19036 (ksk), size = 2048b}
Checking if signing key is trusted:
New key: . 172800 IN DNSKEY 256 3 8 AwEAAbgVvZmZibtBpha3AIykU0OY4gcCXTcskYJUxGsdmV/awfmKcHlSrjNMioSgy4sByj+HpcbsyrZVGPp+JBXzYwwuEF/6w1k7vKYTK6vMSqgVcgooNkfb5MaRF2y7MEpPxfStnfwu8knE24ExB0hYE1URxJ9CqB3zMSl/vicXYXXl ;{id = 62530 (zsk), size = 1024b}
  Trusted key: . 29585 IN DNSKEY 256 3 8 AwEAAbgVvZmZibtBpha3AIykU0OY4gcCXTcskYJUxGsdmV/awfmKcHlSrjNMioSgy4sByj+HpcbsyrZVGPp+JBXzYwwuEF/6w1k7vKYTK6vMSqgVcgooNkfb5MaRF2y7MEpPxfStnfwu8knE24ExB0hYE1URxJ9CqB3zMSl/vicXYXXl ;{id = 62530 (zsk), size = 1024b}
Key is now trusted!
  Trusted key: . 29585 IN DNSKEY 257 3 8 AwEAAagAIKlVZrpC6Ia7gEzahOR+9W29euxhJhVVLOyQbSEW0O8gcCjFFVQUTf6v58fLjwBd0YI0EzrAcQqBGCzh/RStIoO8g0NfnfL2MTJRkxoXbfDaUeVPQuYEhg37NZWAJQ9VnMVDxP/VHL496M/QZxkjf5/Efucp2gaDX6RS6CXpoY68LsvPVjR0ZSwzz1apAzvN9dlzEheX7ICJBBtuA6G3LQpzW5hOA2hzCTMjJPJ8LbqF6dsV6DoBQzgul0sGIcGOYl7OyQdXfZ57relSQageu+ipAdTTJ25AsRTAoub8ONGcLmqrAmRLKBP1dfwhYB4N7knNnulqQxA+Uk1ihz0= ;{id = 19036 (ksk), size = 2048b}
  Trusted key: . 172800 IN DNSKEY 256 3 8 AwEAAbgVvZmZibtBpha3AIykU0OY4gcCXTcskYJUxGsdmV/awfmKcHlSrjNMioSgy4sByj+HpcbsyrZVGPp+JBXzYwwuEF/6w1k7vKYTK6vMSqgVcgooNkfb5MaRF2y7MEpPxfStnfwu8knE24ExB0hYE1URxJ9CqB3zMSl/vicXYXXl ;{id = 62530 (zsk), size = 1024b}
Key is now trusted!
  Trusted key: . 172800 IN DNSKEY 257 3 8 AwEAAagAIKlVZrpC6Ia7gEzahOR+9W29euxhJhVVLOyQbSEW0O8gcCjFFVQUTf6v58fLjwBd0YI0EzrAcQqBGCzh/RStIoO8g0NfnfL2MTJRkxoXbfDaUeVPQuYEhg37NZWAJQ9VnMVDxP/VHL496M/QZxkjf5/Efucp2gaDX6RS6CXpoY68LsvPVjR0ZSwzz1apAzvN9dlzEheX7ICJBBtuA6G3LQpzW5hOA2hzCTMjJPJ8LbqF6dsV6DoBQzgul0sGIcGOYl7OyQdXfZ57relSQageu+ipAdTTJ25AsRTAoub8ONGcLmqrAmRLKBP1dfwhYB4N7knNnulqQxA+Uk1ihz0= ;{id = 19036 (ksk), size = 2048b}
[T] net. 86400 IN DS 35886 8 2 7862b27f5f516ebe19680444d4ce5e762981931842c465f00236401d8bd973ee
;; Domain: net.
[T] net. 86400 IN DNSKEY 257 3 8 ;{id = 35886 (ksk), size = 2048b}
net. 86400 IN DNSKEY 256 3 8 ;{id = 37703 (zsk), size = 1024b}
[B] Error verifying denial of existence for nsatc.net. DS: General LDNS error
;; No ds record for delegation
;; Domain: nsatc.net.
;; No DNSKEY record found for nsatc.net.
;; No DS for com.nsatc.net.;; No ds record for delegation
[B] ;; Error verifying denial of existence for name com.nsatc.net.NS: No DNSSEC signature(s)

  Drill run on one of our name servers shows that the error is

  Existence denied: microsoftonline.com

No, drill just says there are no DS records which means the domain is
insecure so any problems with it should be unrelated to DNSSEC.

[T] Existence denied: microsoftonline.com. DS
;; No ds record for delegation
;; Domain: microsoftonline.com.
;; No DNSKEY record found for microsoftonline.com.
;; No DS for login.microsoftonline.com.;; No ds record for delegation
;; Domain: login.microsoftonline.com.
;; No DNSKEY record found for login.microsoftonline.com.

Tony.