Insanely weird issue with DNS resolution to www.cdc.gov
-
@johnpoz said in Insanely weird issue with DNS resolution to www.cdc.gov:
The cdc really should fix up their shit ;)
I’m experiencing this problem, also. When I disable DNSSEC the problem goes away and CDC.GOV loads.
Can anything else be done as a workaround, which wouldn’t have as broad an scope as toggling DNSSEC?
Thank you —
-
@timtrace said in Insanely weird issue with DNS resolution to www.cdc.gov:
Can anything else be done as a workaround
One way would be to do a domain override to say 9.9.9.10, which is quad9 that doesn't do dnssec.. So that shouldn't fail.. You do a domain override for cdc.gov to any NS that doesn't do dnssec..
Another option should be to set unbound not to do dnssec for that domain.. In the options box
server:
domain-insecure: "cdc.gov"You would think they would have fixed their shit by now to be honest.. You might actually have to do it for domains the cnames point to if you don't do the domain override forwarding to a non dnssec ns..
But looks like they just have the 1 cname currently www.akam.cdc.gov, so cdc.gov as the unsecure domain should work.
Worse case is you add the other domains as unsecure as well
www.akam.cdc.gov. 3378 IN CNAME www.cdc.gov.edgekey.net. www.cdc.gov.edgekey.net. 20544 IN CNAME e9313.dscb.akamaiedge.net.
Who ever is in charge of their dns should really be fired..
-
@johnpoz said in Insanely weird issue with DNS resolution to www.cdc.gov:
server:
domain-insecure: "cdc.gov"Thanks, man! That worked perfectly.
-
@johnpoz Thanks! This option fixed the issue immediately.
- about a month later
-
@johnpoz said in Insanely weird issue with DNS resolution to www.cdc.gov:
Another option should be to set unbound not to do dnssec for that domain.. In the options box
server:
domain-insecure: "cdc.gov"Thank you! Worked for me, too.
-
johnpoz LAYER 8 Global Moderatorlast edited by johnpoz Feb 24, 2021, 11:16 PM Feb 24, 2021, 11:14 PM
I have been checking on this now and then... They are still freaking hosed.. They clearly made some sort of changes - but its still a freaking mess..
If you don't understand how to dnssec - just don't do it.. A borked dnssec is worse than not having it that is for damn sure..
edit:
Just some of the errors still being seenRRSIG cdc.gov/A alg 7, id 42473: DNSSEC specification recommends not signing with DNSSEC algorithm 7 (RSASHA1NSEC3SHA1). RRSIG cdc.gov/DNSKEY alg 7, id 42473: DNSSEC specification recommends not signing with DNSSEC algorithm 7 (RSASHA1NSEC3SHA1). RRSIG cdc.gov/DNSKEY alg 7, id 65139: DNSSEC specification recommends not signing with DNSSEC algorithm 7 (RSASHA1NSEC3SHA1).
- 7 days later
-
I had been experiencing this issue myself for the last several months and it wasn't until my wife complained about it last week that I decided to investigate and came across this thread. I actually went ahead and submitted a message via their contact form and linked to this exact forum thread describing the issue. It's hard to say whether that was the impetus for their finally updating their DNS records, but it does at least seem to be working correctly now without using the 'domain-insecure: "cdc.gov"' workaround. Big thanks to everybody who actually took the time to discover the underlying issue. Now hopefully they don't break it again.
-
This DNS issue for www.cdc.gov site should have been resolved. If anyone is still seeing similar issues, please let CDC know at imtech@cdc.gov. Thank you!
-
This was driving me nuts also. Needed to access the CDC for Covid info and check in after vaccination. Yes "lucky" enough to get one, unless you count having a compromised immune system as not lucky :)
At first I blamed my pfsense and Ngblocker setup as I make ALOT of mistakes as I am just learning. But I found NO errors....I then found this topic and adding the server: domain-insecure: "cdc.gov" solved the issue.
Thanks I was getting ready to pull my DNS setup apart to figure out where I screwed up..........
-
Not sure I would call it "fixed" while its better - they still have stuff wrong if you ask me.. They are using algos they shouldn't be.
If your not going to follow best practices for dnssec - why even try and use it to be honest..
From RFC 8624
+--------+--------------------+-----------------+-------------------+ | Number | Mnemonics | DNSSEC Signing | DNSSEC Validation | +--------+--------------------+-----------------+-------------------+ | 1 | RSAMD5 | MUST NOT | MUST NOT | | 3 | DSA | MUST NOT | MUST NOT | | 5 | RSASHA1 | NOT RECOMMENDED | MUST | | 6 | DSA-NSEC3-SHA1 | MUST NOT | MUST NOT | | 7 | RSASHA1-NSEC3-SHA1 | NOT RECOMMENDED | MUST | | 8 | RSASHA256 | MUST | MUST | | 10 | RSASHA512 | NOT RECOMMENDED | MUST | | 12 | ECC-GOST | MUST NOT | MAY | | 13 | ECDSAP256SHA256 | MUST | MUST | | 14 | ECDSAP384SHA384 | MAY | RECOMMENDED | | 15 | ED25519 | RECOMMENDED | RECOMMENDED | | 16 | ED448 | MAY | RECOMMENDED | +--------+--------------------+-----------------+-------------------+
They are using 7.. plus their NS listed have issues.
gov to cdc.gov: The following NS name(s) were found in the authoritative NS RRset, but not in the delegation NS RRset (i.e., in the gov zone): icdc-us-ns1.cdc.gov, icdc-us-ns3.cdc.gov, icdc-us-ns2.cdc.gov gov to cdc.gov: The following NS name(s) were found in the delegation NS RRset (i.e., in the gov zone), but not in the authoritative NS RRset: auth00.ns.uu.net, auth100.ns.uu.net
But hey if its actually working, better than it was ;)
-
@johnpoz said in Insanely weird issue with DNS resolution to www.cdc.gov:
I would contact the cdc webmaster and show him that above dnsviz link.. Tell him to fix his shit..
Or : he was fired after all, and the new one wiped everything.
https://dnsviz.net/d/www.cdc.org/dnssec/ is clean for me. No more issues.
That is : www.cdc.org becames like the other 85 % (?) of the net : no DNSSEC.
( so, all is well, www.cdc.org is DNS spoofable again ).https://dnsviz.net/d/www.cdc.org/dnssec/
@espalmer said in Insanely weird issue with DNS resolution to www.cdc.gov:
domain-insecure: "cdc.gov" solved the issue.
shouldn't be needed any more as they cleaned up (very recently ?).
Btw : Compare with this site : https://dnsviz.net/d/forum.netgate.com/dnssec/
No DNSSEC neither, and no need to inform unbound about it.Maybe Netgate should activate DNSSEC, just to show that DNSSECworks without any settings on the 'viewers' side (neither our pfSense). After all, its about 'network' security and so.
- 2 months later
-
@johnpoz
I know this is very late, but CDC still hasn't fixed everything.But thank you for the server directive tip. That resolved the issue for me.
server:domain-insecure: "cdc.gov"
-
Yeah I still show it as a big mess..
But at least they are only warnings, and not full on failures, etc.
- 12 days later
-
I'm also having this exact issue now on a Turris Omnia 2020 router with DNSSEC enabled or disabled and forwarding to OpenDNS turned on. If I bypass the router and use direct OpenDNS numbers on my computer I can get to cdc.gov but when I use the router's DNS which is supposed to send my requests up stream to OpenDNS it won't let it through. I don't even get the Block page from OpenDNS it just acts like it won't resolve in the router at all. Other sites seem to work just fine.
I've manually added cdc.gov numbers to the router's /etc/hosts file and it works for now, but this is too hacky and not sustainable, really. Also would like to know how many other sites on the internet aren't working unbeknownst to me.
-
I stumbled on this thread because I was having the same issue right now. I use openDNS on DNS resolver (unbound) on Pfsense (Netgate device).
I used JohnPoz domain override suggestion as a band aid, and it works. For other non-sophisticated home users like myself, here's an example of how to do it on the WebGUI.
Services / DNS Resolver / General Settings
"8.8.8.8" is Google's DNS and for some reason tolerates the incorrect DNS setup at cdc.gov (as described in this thread by people who understand such stuff).
-
@drphil :
https://cachecheck.opendns.com/
cdc.gov
and
www.cdc.govdid resolve for me.
Maybe your own 'opendns' settings are to strict ?
-
@gertjan My openDNS settings are set correctly since directly using them from my computer works fine. Only when the router uses forwarding does it not work.
-
@gregarios said in Insanely weird issue with DNS resolution to www.cdc.gov:
@gertjan My openDNS settings are set correctly since directly using them from my computer works fine. Only when the router uses forwarding does it not work.
The 'DNS' settings might be ok.
But OpenDNS - chekc your OpenDNS account - can use 'filters' like : nothing, no adds - no spam - no sex - no bitcoin - no whatever.Again, check the (non filtered) OpenDNS resolver manually : https://cachecheck.opendns.com/
It resolves.Depending on what you use as for a filtering
If your not uses any filtering on the OpenDNS side, then why use OpenDNS ? Use the real, build in stuff, : the official 13 root servers == use the resolver as your Resolver. Easy to set up, easy to maintain. Plain works.Btw : Finally, the admins of cdc.org finally ditched DNSSEC support.
https://dnsviz.net/d/www.cdc.org/dnssec/ -
@gertjan As I said... my setting on the router and my computer are correct. I have tested it by connecting successfully to the CDC using OpenDNS numbers on my computer. If I use the router's DNS that forwards to the same OpenDNS IP addresses, it does not work. If my filters were too strict it would not work when I connected to it from my computer.
I'll have to retest this though since now CDC has dropped their DNSSEC. Yesterday it was active and in error mode.
-
@gregarios said in Insanely weird issue with DNS resolution to www.cdc.gov:
I'll have to retest this though since now CDC has dropped their DNSSEC. Yesterday it was active and in error mode.
I saw the same thing.
First time it look clean like that : just a non DNSSEC domain.No DNSSEC errors.
OpenDNS is a Resolver, you have to check with their policies, but I can image that if a domain announces "I have DNSSEC" that it should be implemented correctly. If not, well, it should fail.
If you were using the Resolver as a resolver, you could use an option in the custom box that excludes cdc.gov from any DNSSEC issues.
Maybe OpenDNS offers also such a possibility.edit : Oh sh*t. OpenDNS has become less 'open' : it's bought by Cisco .... well. What to say : great ! (?)
-
I have not seen that - they still have a shit ton of errors.. If you do not know how to do dnssec - then you shouldn't even attempt to do it.. Which is what I suggest they should do.. Since clearly they have no idea how to do it correctly.
Its still a horrible mess as of test just did
2021-05-17 09:56:53 UTC -
@johnpoz :
Strange, asked a new "Analysis" this morning, and it looked some what ok.
Re tested again just now, and it's pure BS again !
Now, I'm just a European guy, and I know one has to go to "cdc" when things go bad ( learned this from the Walking Dead ? ).
It looks like an important governmental agency to me.edit : oh, wait : www.irs.gov isn't any better. And no one is complaining .....
-
Just more proof the currnt U.S. government doesn't react in the slightest to any "warnings" — only flat-out catastrophic failure.
-
@gregarios
It's just an admin ** messing up.
I don't think using or adding DNSSEC is a governmental decision.
And give them some time, this issue started more then a year ago, not everybody (admins) have been replaced yet ;)** On both sides : pfSense, using default DNS settings, works just fine.
-
To answer your question
If your not uses any filtering on the OpenDNS side, then why use OpenDNS ?
I am using their free family filter DNS. 208.67.222.123.
(I am a home user and need the filtering).Use the real, build in stuff, : the official 13 root servers == use the resolver as your Resolver. Easy to set up, easy to maintain. Plain works.
I would love to. Is there an easy way to setup a filter that I don't have to maintain on an ongoing basis? I am willing to put in the time / effort upfront. I tried pfBlockerNG, but didn't like the experience and gave up.
Btw : Finally, the admins of cdc.org finally ditched DNSSEC support.
https://dnsviz.net/d/www.cdc.org/dnssec/Whatever they did, isn't working for me yet. And I have a feeling it's not OpenDNS filtering that's causing the issue but an incorrect setup on CDC's side. (could it be that you're looking at cdc.org instead of cdc.gov?)
oh, wait : www.irs.gov isn't any better. And no one is complaining
irs.gov is working fine for me (through my openDNS setup). Whatever is making cdc.gov fail, it does not share with irs.gov
-
@drphil said in Insanely weird issue with DNS resolution to www.cdc.gov:
irs.gov
I don't show any issues with irs.gov
Now I have not personally seen any issues with accessing cdc.gov - not something I go to every hour or even every day, or week even, etc. But their dnssec is not optimal that is for sure - could it cause issues, possible for sure depending on the resolver, and what specific NS they end up talking to, etc.
It is disappointing to see such a mess that is for sure.. But I am not doing anything specific to not do dnssec for that domain, nor anything that would forward requests for that domain to some other resolver.
Issues in accessing it from where your at on the globe could have nothing to do with their mess that is their dnssec deployment.. But there was mention of cloudflare trying to mitigate their problems on cloudflares end, etc.
If for whatever reason your having issues resolving - its easy enough to set unbound to not do dnssec for that domain, or another option is a domain override for the domain to something like cloudflare or googledns, etc.
Its quite possible that their have been issues with ddos attacks or other congestion problems accessing their site.. I am quite sure since covid, there are enough crazies in the world that would like to attack resources in any manner possible, etc.
As with all problems when trying to access it - you need to troubleshoot it with the specifics. Disable all of dnssec is not as solution - could it be a troubleshooting aid, ok.. I would look more into the specifics if I thought dnssec was a problem vs just system wide disable.. But for some with less understanding, that is a valid option. I personally would not suggest they leave it like that for the sake of 1 site.. And just set that domain to not do dnssec, until such time that have fully corrected what is clearly an not optimal configuration.. From what I see they are signing stuff with alg 7 which the spec clearly says not to do. And they are also doing sha-1 also no longer valid. And they clearly seem to be missing some NS entries that are delegation but not in the authority.. I would say their whole setup is just borked ;)
-
@johnpoz said in Insanely weird issue with DNS resolution to www.cdc.gov:
@drphil said in Insanely weird issue with DNS resolution to www.cdc.gov:
irs.gov
I don't show any issues with irs.gov
I knew it.
The "last update" is stored on a 'per user' history list.I saw your last 'udate scan' , and I checked again without updating :
This is what I saw :So, my last update is not your last update : they are stored with our IP's ( ?) so we all have our own individual history of events.
These are actually really usefull, and can't get disrupted by other who scan the same domain :
Updating it shows the same thing for me : warnings and insecure :
Btw : dnsviz.net must have some huge database behind the screens. Sometimes they do announce that they have database == user history, issues.
.....is just borked ;)
Same thing for me.
If I need to access these sites, and DNSSEC stops me from doing so, and I know that these sites really can't be forged **, I just put them on the "DNSSEC exclusion list", and, because they are listed now, check ones in a while if I can remove them from my list.To @all ; when I say "list" I mean : I've read the manual](https://www.nlnetlabs.nl/documentation/unbound/howto-turnoff-dnssec/).
So, example ::Sites like https://dnsviz.net should be part of the box with basic tools that any (DNS) (pfSense) admin uses.
** well, just hope ...
-
@gertjan Not sure about all that.. I just looked and I am seeing warnings with www.irs.gov, but not with just irs.gov
- 11 months later
-
Hello, we just ran into this today.
Unbound as a forwarder using OpenDNS (Cisco Umbrella DNS Essentials subscription).
I don't see any red errors on the https://dnsviz.net/d/www.cdc.gov/dnssec/ site.
Thanks for all the resources here for figuring out how to bypass the errors. The 'domain-insecure: "cdc.gov"' bypass seems to work for us also.
I'll send a note to the CDC IT contact that popped up in this thread.
Josh
-
@stompro said in Insanely weird issue with DNS resolution to www.cdc.gov:
www.cdc.gov
I still show that domain a mess from dnssec point of view..
Unbound as a forwarder using OpenDNS
If your forwarding with unbound, you should not have dnssec checked. Where you forward to does dnssec or it doesn't.. That checkmark telling unbound to do dnssec isn't going to do anything other than problems...
-
I wasn't sure if warnings were a problem or not. Some of the other screenshots were filled with red errors... which is what I though the real problem was.
@johnpoz said in Insanely weird issue with DNS resolution to www.cdc.gov:
If your forwarding with unbound, you should not have dnssec checked. Where you forward to does dnssec or it doesn't.. That checkmark telling unbound to do dnssec isn't going to do anything other than problems...
Thank you for saying this again so it would get through to me.
So if the servers that I'm using to forward my request to do not support dnssec... then unbound doesn't do any extra checks on the returned information if dnssec is checked? There is no possible benefit there?
And if the upstream dns server does do dnssec, then it has already performed the extra checks, so the unbound setting is redundant?
Am I understanding that correctly?
Josh -
@stompro asking who your forwarding for dnssec info gets you nothing.. They could send you whatever they have cached, etc.
For dnssec to be valid you need to directly talk to the authoritative name servers and validate the info, etc.
Its just going to be extra queries and bandwidth for no actual real benefit.. And could cause problems..
There is zero reason to have your dns software ask for dnssec info if its not actually resolving..
-
@johnpoz said in Insanely weird issue with DNS resolution to www.cdc.gov:
For dnssec to be valid you need to directly talk to the authoritative name servers and validate the info, etc.
Its just going to be extra queries and bandwidth for no actual real benefit.. And could cause problems..
There is zero reason to have your dns software ask for dnssec info if its not actually resolving..Thank you for the extra explanation. I'll fix all my installs to uncheck the "DNSSEC" unbound config option.
-
Keep in mind that you are forwarding to ... a resolver.
That resolver probably does DNSSEC checks, and the cdc announced it supports DNSSEC.
If there was more then a warning, this an error, and the errors concerns a DNS record you were askinf for, protection kicks in : no answer or NXDOMAIN.
You think : my DNS is bad.
The reality is : the zone cdc is bad.
You have just one choice : use a resolver without DNSSEC. So now cdc records as any other DNS can be spoofed etc. I hope this "cdc" site isn't important for you.can you imagine what happens when facebook start to implement DNSSEC, and they f*ck up on their side ?
-
johnpoz LAYER 8 Global Moderatorlast edited by johnpoz Apr 22, 2022, 11:02 AM Apr 22, 2022, 11:01 AM
@gertjan said in Insanely weird issue with DNS resolution to www.cdc.gov:
Keep in mind that you are forwarding to ... a resolver.
Might not actually be true - could be another forwarder ;) But yeah at some point in the chain there has to be a resolver... So your point is valid.
You know for sure its not actually say 8.8.8.8 doing the actual resolving.. This is why for example when you setup 8.8.8.8 as your dns, and you do one of those dns leak tests, it shows other IPs as your dns that are not 8.8.8.8, same when you point to 1.1.1.1
I would be pretty sure in saying that the IP you point to when using any of these services, isn't actually doing the "resolving" itself. But just forwarding to some other box in their network. DNS wouldn't work at all if at some point a resolver doesn't get asked.
-
@gertjan Thank you for the extra info.
I don't quite understand how the unbound DNSSEC option and the work around of setting 'server:domain-insecure: "cdc.gov"' relate to each other, if at all. But I haven't looked up the domain-insecure option yet to see what it actually does. You don't need to google it for me. :-)
I'll try and do more testing when I have a chance.
- Cisco Umbrella DNS Servers
- With DNSSEC Enabled - cdc.gov doesn't work for clients.
- With DNSSEC enabled in unbound and 'server:domain-insecure: "cdc.gov"' set, cdc.gov resolves fine for clients.
I can test with cloudflare and google dns, with dnssec on and off.
The pfsense dns lookup gui seems to always work... but I don't know if that is DNSSEC aware.
Maybe the end clients are doing some sort of dnssec validation... but I think @johnpoz explained that that isn't how it works. The resolvers do the validation.
I kind of hate not knowing why something isn't working... but who knows if this whole DNSSEC thing is really going to stick around,,, seems like a fad, like fidget spinners. :-)
- Cisco Umbrella DNS Servers
-
@stompro said in Insanely weird issue with DNS resolution to www.cdc.gov:
seems like a fad, like fidget spinners
Not sure if I would say that - but the overall adoption is disappointing to be sure..
Here is the thing that site is all kinds of messed up when it comes to dnssec... I don't have any problem resolving it, using dnssec - but with some of the errors I see, it could for sure be hit or miss.
If your forwarding, and also have dnssec enabled that can cause issues. So are you saying when you uncheck dnssec in unbound, and forward to cisco it fails? Is that something you have to enable do disable in your subscription.. Cisco Umbrella is a subscription service is in not?
$ dig @192.168.9.253 www.cdc.gov ; <<>> DiG 9.16.27 <<>> @192.168.9.253 www.cdc.gov ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 15485 ;; flags: qr rd ra ad; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;www.cdc.gov. IN A ;; ANSWER SECTION: www.cdc.gov. 3600 IN CNAME www.akam.cdc.gov. www.akam.cdc.gov. 3600 IN A 104.98.82.250 ;; Query time: 185 msec ;; SERVER: 192.168.9.253#53(192.168.9.253) ;; WHEN: Fri Apr 22 09:45:15 Central Daylight Time 2022 ;; MSG SIZE rcvd: 79