couldn't get address for 'ns1.arin.net': not found
couldn't get address for 'ns2.arin.net': not found
couldn't get address for 'u.arin.net': not found
couldn't get address for 'ns3.arin.net': not found
dig: couldn't get address for 'ns1.arin.net': no more
net. 172800 IN NS g.gtld-servers.net.
net. 172800 IN NS c.gtld-servers.net.
net. 172800 IN NS j.gtld-servers.net.
net. 172800 IN NS e.gtld-servers.net.
net. 172800 IN NS h.gtld-servers.net.
net. 172800 IN NS k.gtld-servers.net.
net. 172800 IN NS m.gtld-servers.net.
net. 172800 IN NS i.gtld-servers.net.
net. 172800 IN NS f.gtld-servers.net.
net. 172800 IN NS b.gtld-servers.net.
net. 172800 IN NS a.gtld-servers.net.
net. 172800 IN NS d.gtld-servers.net.
net. 172800 IN NS l.gtld-servers.net.
net. 86400 IN DS 35886 8 2 7862B27F5F516EBE19680444D4CE5E762981931842C465F00236401D 8BD973EE
net. 86400 IN RRSIG DS 8 1 86400 20190124130000 20190111120000 16749 . uahpltN27UkKaFJRaAU1on+IpC2lpgZo84XEM7Pk7dQysKfSnqUkaVLY PXQf9kvgW5eOx/+BttQB2OWFLckJs8vv5ScOpz7dDhs8zR2FPLm93HTD 4F/XEKDNOQbFGSA3g4pZq3fatY7kFEkV9sFTH90WqJt0sXe64LYFcwr2 FtrJaS/yhEV4XDbsN3RLkBP58bf526LPpvonwSZsMUTDZcnXtUnc57ZI dlTHg2snNhVWu4qJfHDsEQPwOZagRXJhjlRT8Ox/7HwXvplmRfmeuhZb Vj5kdiY+3j0RTxpLRCG/SZRDIRcvdFKh9umdwQvAzuTS0xzO8OyPw9q8 8QCCYg==
;; Received 1171 bytes from 192.112.36.4#53(g.root-servers.net) in 207 ms
arin.net. 172800 IN NS ns1.arin.net. arin.net. 172800 IN NS ns2.arin.net. arin.net. 172800 IN NS u.arin.net. arin.net. 172800 IN NS ns3.arin.net. arin.net. 86400 IN DS 48281 5 2 6EB0CCF325A8101A768C93D10CE084303D3714D4E92FEE53D6E683D2 22291017 arin.net. 86400 IN DS 48281 5 1 FCBF93357C8FE3247CECB2CD277F45EB955EE4CE arin.net. 86400 IN RRSIG DS 8 2 86400 20190117062448 20190110051448 6140 net. stuWyfC0PDuk2hNF/Bnz0lnypk+bA/slTa2KYznjmoLXDtq7v1obJq41 ZfloQKXuC7MnzpCQj70GU9ZESZq1/XU+u6wDmCqmEUbJ3kyrILxkVrln bTEySJWPmurpwUVzDVfvqFpXEOhWxOjDu6drZMcC3wG9EdPqBuFC6wlf FIQ=
couldn't get address for 'ns1.arin.net': not found
couldn't get address for 'ns2.arin.net': not found
couldn't get address for 'u.arin.net': not found
couldn't get address for 'ns3.arin.net': not found
dig: couldn't get address for 'ns1.arin.net': no more
We’re aware and working the problem. It looks to me like expired RRSIG/DNSKEY’s for the zone,
so if you’re using a DNSSEC validating resolver (e.g. Google, Cloudflare, Cogent) then ARIN.NET
is unreachable. ARIN’s engineering team is working on resolution now.
/John
John Curran
President and CEO
American Registry for Internet Numbers
Is this the right time to ask whether everyone who operates DNSSEC validating resolvers was required to click somewhere on the ARIN website that they agree to be bound by the Relying Party Agreement before their resolver can make DNSSEC lookups against the ARIN nameservers?
Or does that logic only apply for access to the RPKI TAL?
Thanks for the update that dnssec STILL causes more real world problems
than it solves.
Do you feel the same way about RPKI?
Misorgination is a real threat we see all the time (threat on uptime, if not more)
That said, i think history has shown we get more kilometers out of good BGP policy control hygiene and IRR data than RPKI. I don’t think that will change in the future. I do wish irr data was better, for many values of better.
My routes are rpki signed. But, my router kit and ops procedure don’t make me enforcing near-term achievable.
The ARIN.NET zone on our public signed DNS servers are populated via an internal DNS server and associated workflow. As part of system maintenance near the end of 2018, the zone file used by the master internal DNS server was updated incorrectly, resulting in an invalid zone file. Since the zone file was invalid, the zone did not reload on our internal master, and the associated workflow to DNSSEC sign and push this zone to the public servers did not execute. Our monitoring systems reported being green until the signatures expired as they presently check that the SOA’s match on the internal and external nameservers.
At approximately 8:30AM eastern time today (11 January 2019), ARIN operations started seeing issues within its monitoring. Initial review suggested the problem was DNSSEC-related due to expired signatures. We pulled the DS record from the zone so that DNSSEC validation would not be performed by those validating resolvers that had not already cached our DS records. Upon further investigation we determined that it was the result of human error in editing a zone file that went undetected and resulted in interruption of our routine zone publication process. The issue was fixed and signed zones where then pushed out at 10:25 AM ET. The DS record was reinstated in the parent at 10:30AM ET.
As a result of this incident, we will add additional alerting to the zone loading process for any errors and perform monitoring of zone signature lifetimes, with appropriate alerting for any potential expiration of DNSSEC signatures.
My apologies for this incident – while ARIN does have some fragility in our older systems (which we have been working aggressively to phase out via system refresh and replacements), it is not acceptable to have this situation with key infrastructure such as our DNS zones. We will prioritize the necessary alert and monitor changes and I will report back to the community once that has been completed.
Thank you for your patience in this regard.
/John
John Curran
President and CEO
American Registry for Internet Numbers
Our monitoring systems reported being green until the signatures
expired as they presently check that the SOA's match on the internal
and external nameservers.
I indicated that we would report back once appropriate DNSSEC monitoring is in place - this has now been completed (ref: attached announcement of same)
Thanks again for your patience in this matter,
/John
John Curran
President and CEO
American Registry for Internet Numbers
