We have a report on outages that he.net has been placed in ICANN client hold, and people’s DNS service is falling over on this Independence day. If you work in DNS for HE, you might want to look into this.
I have double checked the report, and I am seeing the status as well.
Hurricane serves lots of dns, I would classify this as a P1 ticket.
Cheers,
– jra
I called their support when that outage thread came in, they’re already aware and taking a look now.
Ryan Hamel
Cool, thanks. We had a couple of other reports of people making support calls and being asked to reboot their modems, so I wanted to make sure tier 3 had gotten it.
And I figured tier 3 would be here. 
Cheers,
– jra
Our he.net dns appears to be fine at this time:
$ nslookup
server ns1.he.net
Default server: ns1.he.net
Address: 2001:470:100::2#53
Default server: ns1.he.net
Address: 216.218.130.2#53
set type=A
jet.net.
Server: ns1.he.net
Address: 216.218.130.2#53
Name: jet.net
Address: 206.83.0.42
-mel beckman
I’ve been informed that the CEO of HE is on this as of 1512EDT.
I approve of the scale of this response.
Cheers,
– jra
Mel,
Your local caching resolver knows the IPs for ns[1-5].he.net, which skips over the need for querying the root DNS resolvers, and gtld-servers (glue records). If the TTL (2 days) expires on your resolver before HE fixes their issue, you will not be able to resolve anything for that domain.
At the moment, a simple DNS trace (dig he.net +trace) cannot complete fully.
Ryan,
Right you are. The dig still fails. hopefully the ICANN issue gets fixed, and a pox on any bureaucrat who arranged for this to happen over a holiday weekend!
-mel
Network Solutions has decided to put our domain name on Client Hold due to a single phishing complaint about a web page, which happens to just be a page of information about another domain from bgp.he.net. Network Solutions has been contacted, and refuses to handle this issue in ANY expedited manner. Executives from Hurricane have been calling and emailing Network Solutions for HOURS trying to have this addressed. If anyone has an escalation contact at Network Solutions, please email it to me at redhead@lightning.net, or rfishler@he.net. Thanks.
Reid Fishler
Sr Director
Hurricane Electric
Aha. Just as I suspected, bureaucrats at Network Solutions are to blame. I have had many run-ins with NS and their inscrutable policies and odd viewpoints. I was once suspended for running a web cache that NS incorrectly claimed was stealing domain content. No engineer on the NS side seemed to know what a web cache does.
-mel via cell
We have a report on outages that he.net has been placed in ICANN
client hold, and people's DNS service is falling over on this
Independence day.
Seems to have had hold removed 20:20 zulu, according to whois.
Domain back in .net and working again.
On the other side of this, we all may be learning the value of not having all of you NS records in a single zone with a domain under a single registrar.
(From someone who has personal domains hosted on HE DNS.)
Yup; I blew that one too.
I've been told it was cleared around 2020Z, and whois reflects that,
though my dig +trace doesn't seem to be behaving as expected.
Cheers,
-- jra
It appears that Reid Fishler via NANOG <redhead@lightning.net> said:
-=-=-=-=-=-
Network Solutions has decided to put our domain name on Client Hold due to
a single phishing complaint about a web page, which happens to just be a
page of information about another domain from bgp.he.net. Network Solutions
has been contacted, and refuses to handle this issue in ANY expedited
manner. Executives from Hurricane have been calling and emailing Network
Solutions for HOURS trying to have this addressed. If anyone has an
escalation contact at Network Solutions, please email it to me at
redhead@lightning.net, or rfishler@he.net. Thanks.
Glad to see that they fixed it, but this would be a good time to remember
why nobody who cares about their domain names would use Netsol.
The usual choices for high value domains are Markmonitor and CSC. They
cost more but sometimes it's worth it. (Not that Netsol is
particularly cheap.)
R's,
John
On the other side of this, we all may be learning the value of not
having all of you NS records in a single zone with a domain under a
single registrar.
From some trainings I did on how to be sure your DNS was robust:
- don't have all your business critical domains under the same
registrar (unless it's of the CSC/markmonitor class)
- don't have all your auth NS for your domain in bailiwick (within the
domain being served)
- don't have all your auth NS in the same routing domain (anycast can
be an exception to this if robust enough)
- don't have the account registrar credential emails all within the
domain, nor with personal emails like gmail. do have them all under
control of your IT
- protect all account credentials with strong passwords, MFA
- have MX for your domain either with a very large provider or across
multiple domain names
It's painfully easy to fall off the internet and be unreachable if
you're not thinking about all this for business critical domains. You
don't ever want to be hoping that some customer kept your NOC phone
number in their phone. 
After a metric ton of screaming, we did get the issue solved. Thanks everyone, and we WILL be following up with the powers that be.
Reid
not to distract from everyone diagnosing someone else's problem, but ...
what foss dns monitoring tools do folk use to alert of
- iminent delegation expiry
- inconsistent service (lame, soa mismatches, ...)
- dnssec signing and timer issues
- etc.
randy
The majority of real large DNS hosting providers have their authoritative under multiple TLDs, bar a couple of exceptions (Google Domains / Cloudflare if memory serves me well).
I’d really curious to hear from them how they think about resilience and whether some sort of special protection has been added to their domains.
Or, the value of not using a free DNS service with (likely) no SLA for seemingly “critical” services. Good DNS services are relatively cheap in the grand scheme of things.
http://www.rfc-editor.org/rfc/rfc2182.txt
Is what you should use as guidance