So have *.app.linktechs.net that I have been trying to get to work, we have DNSSEC on this, and its failing, but cannot for the life of me understand why. I think it may have something to do with proving it exists as a wildcard, but any DNSSEC experts want to take a stab at it ?
Dennis Burgess
* nanog@nanog.org (Dennis Burgess via NANOG) [Fri 15 Mar 2024, 16:26 CET]:
So have *.app.linktechs.net that I have been trying to get to work, we have DNSSEC on this, and its failing, but cannot for the life of me understand why. I think it may have something to do with proving it exists as a wildcard, but any DNSSEC experts want to take a stab at it ?
There are better mailing lists to ask this question (like dns-operations at dns-oarc.net) but have you checked www.app.linktechs.net | DNSViz ?
-- Niels.
It appears that Niels Bakker <niels=nanog@bakker.net> said:
* nanog@nanog.org (Dennis Burgess via NANOG) [Fri 15 Mar 2024, 16:26 CET]:
So have *.app.linktechs.net that I have been trying to get to work,
we have DNSSEC on this, and its failing, but cannot for the life of
me understand why. I think it may have something to do with proving
it exists as a wildcard, but any DNSSEC experts want to take a stab
at it ?There are better mailing lists to ask this question (like
dns-operations at dns-oarc.net) but have you checked
www.app.linktechs.net | DNSViz ?
I agree there are better places to ask, but here's a quick
diagnosis: your nameserver is returning the wrong answer.
What kind of server is it? Any modern nameserver should automatically
return the correct DNSSEC stuff for wildcard responses.
R's,
John
Dennis Burgess via NANOG <nanog@nanog.org> writes:
So have *.app.linktechs.net that I have been trying to get to work, we
have DNSSEC on this, and its failing, but cannot for the life of me
understand why. I think it may have something to do with proving it
exists as a wildcard, but any DNSSEC experts want to take a stab at it
?
Not an expert, but Google knows the answer (as always 
bjorn@miraculix:~$ dig a www.app.linktechs.net. +dnssec +multiline @8.8.8.8
; <<>> DiG 9.18.24-1-Debian <<>> a www.app.linktechs.net. +dnssec +multiline @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 65184
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 512
; EDE: 10 (RRSIGs Missing): (For _acme-challenge.app.linktechs.net/nsec)
;; QUESTION SECTION:
;www.app.linktechs.net. IN A
;; Query time: 156 msec
;; SERVER: 8.8.8.8#53(8.8.8.8) (UDP)
;; WHEN: Fri Mar 15 18:26:16 CET 2024
;; MSG SIZE rcvd: 98
And they're right. Your server includes the epected NSEC record, but
leaves out the associated RRSIG. Compare this to the
RFC 7129 - Authenticated Denial of Existence in the DNS example:
bjorn@miraculix:~$ dig a www.app.linktechs.net. +dnssec +multiline @139.60.210.20
; <<>> DiG 9.18.24-1-Debian <<>> a www.app.linktechs.net. +dnssec +multiline @139.60.210.20
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11828
;; flags: qr aa rd; QUERY: 1, ANSWER: 3, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 1280
;; QUESTION SECTION:
;www.app.linktechs.net. IN A
;; ANSWER SECTION:
www.app.linktechs.net. 3600 IN A 139.60.210.81
www.app.linktechs.net. 3600 IN RRSIG A 8 3 3600 (
20240427232616 20240313222616 37041 linktechs.net.
NYC/4H2VZg12vj+tiWVkEROhXwm7JkBna6RQg6LO8kXr
oosDUpGnxrgOtJYsWYbYfM58opiC1OeAbcaCB9+nctIU
grrwcpuhmvlXYLZi1n/oAmelPldnQ6Hf93HuHi4ULFsS
Qfsoo8sdfjt/YSJ4WxjmsM9LMbZ2CZPMU44a3MdftGW1
fNKmZ1fLtVreP41KmvP6b01lyUMvjrvT26Yq57DgUDTo
iqU5skT+OHzx6ERJkt3tzzwm2pBMvBWFDXC668NtouIW
s3mrhJRBuNW3xSCsroaLQ0vmdml2BqNNh7MZNc38FNMJ
eh+ts3mbMnOOkzlI1Q8gKMMCWv+VRmv2DA== )
www.app.linktechs.net. 3600 IN RRSIG A 8 3 3600 (
20240427232616 20240313222616 11340 linktechs.net.
Th3OcZwOMNUb1zMdipnTnFdgFEaOGJ/VofQOTyxmnNCg
wl+1Q7eiQ89KHAWEDBisxd0S+EHu6/YBWY2srNx5q58P
XIZJ9oQXCqDLzSE884DTQNDEVrSMoKJ9slRU4N4Lj5tT
9LzbODmCM9ytRavOKXJHIddQa0MZT4p9cV8K2HI7XSFX
0rjieKFa7wDRJqhKyqrT3Rh/S93pavhKWUgN3GVO6hkI
H5F67UFpZK7o7nRlyqvM42ep5XaRZS/WJtLuXcTk/QM3
MBPTDWgJ0Bh8qpNuHDOb2XFH2I5dwjeKxuYCzeQzN1hL
gsmw3d1J2pNsYbC40jmi1bZr0bz2fDurIA== )
;; AUTHORITY SECTION:
_acme-challenge.app.linktechs.net. 1200 IN NSEC auto.linktechs.net. TXT RRSIG NSEC
;; Query time: 120 msec
;; SERVER: 139.60.210.20#53(139.60.210.20) (UDP)
;; WHEN: Fri Mar 15 18:27:13 CET 2024
;; MSG SIZE rcvd: 724
Lots of resolvers will probably just look up that NSEC, get the RRSIG
and be happy with that. But your problem is that there always will be a
number of resolvers not doing that. So you get arbitrary failures.
I believe there are so many examples of really bad fallout due to
wildcards combined with DNSSEC that it's advisable to stay away. Do one
or the other. Not both.
Bjørn
So have *.app.linktechs.net that I have been trying to get to work, we have DNSSEC on this, and its failing, but cannot for the life of me understand why. I think it may have something to do with proving it exists as a wildcard, but any DNSSEC experts want to take a stab at it ?
As others have mentioned, the DNS-operations list would be a better place to get help: <https://lists.dns-oarc.net/mailman/listinfo/dns-operations>
But, right off the top I can see that your name server is returning the NSEC record in the wrong section of the response.
Matthew Pounsett <matt@conundrum.com> writes:
But, right off the top I can see that your name server is returning the
NSEC record in the wrong section of the response.
No, the Authority section is correct here. See:
But the RRSIG is missing.
Bjørn
The authority section is the correct section for the NSEC.
Ask the question using TCP. I suspect that the server isn’t truncating the UDP response correctly. If I’m right you will get RRSIGs for the NSEC added to the additional section. If not the zone needs to be resigned as they are missing. I’m answering from my phone or else I would look it up myself.
Looks like your DNS server correctly queues up the RRs, but erronously
believes it can drop data from the Authority section without setting the
TC bit.
Reducing the bufsize so the answer doesn't fit makes trucation work:
bjorn@miraculix:~$ dig a www.app.linktechs.net. +dnssec +multiline +norecur @139.60.210.20 +bufsize=512
;; Truncated, retrying in TCP mode.
; <<>> DiG 9.18.24-1-Debian <<>> a www.app.linktechs.net. +dnssec +multiline +norecur @139.60.210.20 +bufsize=512
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5946
;; flags: qr aa; QUERY: 1, ANSWER: 3, AUTHORITY: 3, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 1280
;; QUESTION SECTION:
;www.app.linktechs.net. IN A
;; ANSWER SECTION:
www.app.linktechs.net. 3600 IN A 139.60.210.81
www.app.linktechs.net. 3600 IN RRSIG A 8 3 3600 (
20240427232616 20240313222616 37041 linktechs.net.
NYC/4H2VZg12vj+tiWVkEROhXwm7JkBna6RQg6LO8kXr
oosDUpGnxrgOtJYsWYbYfM58opiC1OeAbcaCB9+nctIU
grrwcpuhmvlXYLZi1n/oAmelPldnQ6Hf93HuHi4ULFsS
Qfsoo8sdfjt/YSJ4WxjmsM9LMbZ2CZPMU44a3MdftGW1
fNKmZ1fLtVreP41KmvP6b01lyUMvjrvT26Yq57DgUDTo
iqU5skT+OHzx6ERJkt3tzzwm2pBMvBWFDXC668NtouIW
s3mrhJRBuNW3xSCsroaLQ0vmdml2BqNNh7MZNc38FNMJ
eh+ts3mbMnOOkzlI1Q8gKMMCWv+VRmv2DA== )
www.app.linktechs.net. 3600 IN RRSIG A 8 3 3600 (
20240427232616 20240313222616 11340 linktechs.net.
Th3OcZwOMNUb1zMdipnTnFdgFEaOGJ/VofQOTyxmnNCg
wl+1Q7eiQ89KHAWEDBisxd0S+EHu6/YBWY2srNx5q58P
XIZJ9oQXCqDLzSE884DTQNDEVrSMoKJ9slRU4N4Lj5tT
9LzbODmCM9ytRavOKXJHIddQa0MZT4p9cV8K2HI7XSFX
0rjieKFa7wDRJqhKyqrT3Rh/S93pavhKWUgN3GVO6hkI
H5F67UFpZK7o7nRlyqvM42ep5XaRZS/WJtLuXcTk/QM3
MBPTDWgJ0Bh8qpNuHDOb2XFH2I5dwjeKxuYCzeQzN1hL
gsmw3d1J2pNsYbC40jmi1bZr0bz2fDurIA== )
;; AUTHORITY SECTION:
_acme-challenge.app.linktechs.net. 1200 IN NSEC auto.linktechs.net. TXT RRSIG NSEC
_acme-challenge.app.linktechs.net. 1200 IN RRSIG NSEC 8 4 1200 (
20240427232616 20240313222616 11340 linktechs.net.
grjacRLmt+h5UMJkWMgrxeeY4m8kzNCokMsEFAi/10ld
2zcx7IZnB5oljSoZo2ZoqN0DEWVOrORGaU0kAcXDIwmD
89JG728W78+gikb8D+rpcSejfpAO8tRFO9saPSDY72uk
oP0Wle87oMcKmP9EXGcgsTZhd6Dld9qcAlUByGAZC/bi
SL5SDeALjpdqzXPXivP597VyJGakeEEjW0y2SmUOIDcg
6lOcSGX1QdmbaiHyAxHSjBsg4VV2Qpo2Br75xyfw3o1Z
oHMeacsAhhz5HQhtzv9DzULzmtmoA5sQn2VyBm2kcS+S
ZKpKioFnHj9BtOv3dn/F5hrQFhEInNPROw== )
_acme-challenge.app.linktechs.net. 1200 IN RRSIG NSEC 8 4 1200 (
20240427232616 20240313222616 37041 linktechs.net.
bt6W5P4VDC5fs2r/lxwSnI8bhqS2MH7n67Gd2EK6+DDx
HYy9MAmSZEy2OYGg7QHamrWr2I+Bq2Og8A0bRRA5TitQ
VcWyq3b+VpXUPukg7bmXl4KRNGxdAB8NysoOT75yvPTe
Jy1baNzYv9/in6rf8VKXUrKSPUqcAsK3Sz5QHkuzzaIP
d+u5m59DAlobNi17QbRGKIQaXTtgkSHpj4rt61MMEzpB
JDXE5FRLCJ4pqQPm+DcF0ZrKoYqKv/1rYZSVbW3rY0XB
VEBDVy5MJg0YenhbVPcDM9OYh2dfvh5ZvYS6xsXZulv8
mKnjdJo7v6qAzPNvIhymghM+0Tp8INxAjw== )
;; Query time: 120 msec
;; SERVER: 139.60.210.20#53(139.60.210.20) (TCP)
;; WHEN: Fri Mar 15 18:57:20 CET 2024
;; MSG SIZE rcvd: 1326
And directly using tcp also works:
bjorn@miraculix:~$ dig a www.app.linktechs.net. +dnssec +multiline +norecur @139.60.210.20 +vc
; <<>> DiG 9.18.24-1-Debian <<>> a www.app.linktechs.net. +dnssec +multiline +norecur @139.60.210.20 +vc
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 29513
;; flags: qr aa; QUERY: 1, ANSWER: 3, AUTHORITY: 3, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 1280
;; QUESTION SECTION:
;www.app.linktechs.net. IN A
;; ANSWER SECTION:
www.app.linktechs.net. 3600 IN A 139.60.210.81
www.app.linktechs.net. 3600 IN RRSIG A 8 3 3600 (
20240427232616 20240313222616 37041 linktechs.net.
NYC/4H2VZg12vj+tiWVkEROhXwm7JkBna6RQg6LO8kXr
oosDUpGnxrgOtJYsWYbYfM58opiC1OeAbcaCB9+nctIU
grrwcpuhmvlXYLZi1n/oAmelPldnQ6Hf93HuHi4ULFsS
Qfsoo8sdfjt/YSJ4WxjmsM9LMbZ2CZPMU44a3MdftGW1
fNKmZ1fLtVreP41KmvP6b01lyUMvjrvT26Yq57DgUDTo
iqU5skT+OHzx6ERJkt3tzzwm2pBMvBWFDXC668NtouIW
s3mrhJRBuNW3xSCsroaLQ0vmdml2BqNNh7MZNc38FNMJ
eh+ts3mbMnOOkzlI1Q8gKMMCWv+VRmv2DA== )
www.app.linktechs.net. 3600 IN RRSIG A 8 3 3600 (
20240427232616 20240313222616 11340 linktechs.net.
Th3OcZwOMNUb1zMdipnTnFdgFEaOGJ/VofQOTyxmnNCg
wl+1Q7eiQ89KHAWEDBisxd0S+EHu6/YBWY2srNx5q58P
XIZJ9oQXCqDLzSE884DTQNDEVrSMoKJ9slRU4N4Lj5tT
9LzbODmCM9ytRavOKXJHIddQa0MZT4p9cV8K2HI7XSFX
0rjieKFa7wDRJqhKyqrT3Rh/S93pavhKWUgN3GVO6hkI
H5F67UFpZK7o7nRlyqvM42ep5XaRZS/WJtLuXcTk/QM3
MBPTDWgJ0Bh8qpNuHDOb2XFH2I5dwjeKxuYCzeQzN1hL
gsmw3d1J2pNsYbC40jmi1bZr0bz2fDurIA== )
;; AUTHORITY SECTION:
_acme-challenge.app.linktechs.net. 1200 IN NSEC auto.linktechs.net. TXT RRSIG NSEC
_acme-challenge.app.linktechs.net. 1200 IN RRSIG NSEC 8 4 1200 (
20240427232616 20240313222616 11340 linktechs.net.
grjacRLmt+h5UMJkWMgrxeeY4m8kzNCokMsEFAi/10ld
2zcx7IZnB5oljSoZo2ZoqN0DEWVOrORGaU0kAcXDIwmD
89JG728W78+gikb8D+rpcSejfpAO8tRFO9saPSDY72uk
oP0Wle87oMcKmP9EXGcgsTZhd6Dld9qcAlUByGAZC/bi
SL5SDeALjpdqzXPXivP597VyJGakeEEjW0y2SmUOIDcg
6lOcSGX1QdmbaiHyAxHSjBsg4VV2Qpo2Br75xyfw3o1Z
oHMeacsAhhz5HQhtzv9DzULzmtmoA5sQn2VyBm2kcS+S
ZKpKioFnHj9BtOv3dn/F5hrQFhEInNPROw== )
_acme-challenge.app.linktechs.net. 1200 IN RRSIG NSEC 8 4 1200 (
20240427232616 20240313222616 37041 linktechs.net.
bt6W5P4VDC5fs2r/lxwSnI8bhqS2MH7n67Gd2EK6+DDx
HYy9MAmSZEy2OYGg7QHamrWr2I+Bq2Og8A0bRRA5TitQ
VcWyq3b+VpXUPukg7bmXl4KRNGxdAB8NysoOT75yvPTe
Jy1baNzYv9/in6rf8VKXUrKSPUqcAsK3Sz5QHkuzzaIP
d+u5m59DAlobNi17QbRGKIQaXTtgkSHpj4rt61MMEzpB
JDXE5FRLCJ4pqQPm+DcF0ZrKoYqKv/1rYZSVbW3rY0XB
VEBDVy5MJg0YenhbVPcDM9OYh2dfvh5ZvYS6xsXZulv8
mKnjdJo7v6qAzPNvIhymghM+0Tp8INxAjw== )
;; Query time: 120 msec
;; SERVER: 139.60.210.20#53(139.60.210.20) (TCP)
;; WHEN: Fri Mar 15 18:57:56 CET 2024
;; MSG SIZE rcvd: 1326
So you might be able to configure yourself oout of that by simply
dropping one of the ZSKs, reducing the number of signatures. And/or
using an algorithm with shorter signatures.
But it will always be fragile.
Bjørn
Wildcards and DNSSEC work fine as long as the nameserver vendor has not stuffed up. Too many vendors play fast and loose with the DNS protocol. Getting this correct is not hard but you do need to test before shipping. Additionally OS vendors tend to be way behind current releases from the nameserver vendors.
Looks like Bjorn was correct, one two many signatures 
Removed one and its all fixed! Thanks too all that replied!!
Dennis Burgess <dmburgess@linktechs.net> writes:
Looks like Bjorn was correct, one two many signatures
and its all fixed! Thanks too all that replied!!
Glad to hear that. But do note that Mark is right, of course. The real
problem is a bug in your name server. What you have now is a workaround
as solid as a house made of straw.
Bjørn
Yep. Look for an upgrade then file a bug report if not fixed by the upgrade. It should be < 10 minutes work to fix + tests etc.
is there anysi network simulator for carrier networks ?
well, from 2023 to 2024 there happes so many carrier network outage caused by network operation.
to my limited knowledge , SP guru from riverbed could simulate carrier network. but I just checked
riverbed website, SP guru stop updating in 2014.
joe
There is a bunch of others, but that should get your started down the rabbit hole. Mark.