ISP blocking or interfering with traffic? Cant resolve certain domains all of a sudden
-
@johnpoz I assume these DNS issues are probably not related to misconfiguration(s) since most of the time things just work well... If it was a misconfiguration (and a serious one), either things would not work at all or rarely work well... Am I wrong to assume so?
This DNS resolution issue is intermittent. I've seen it a handful of times in the last year or so....
I have a hard time to understand your 1st sentence. Are you saying that because I am FW, the "place" I am FW is the culprit? Yes apparently I FW to TLS based on the custom settings of unbound. Not sure where I got this in the past, maybe from a Quad9 tutorial or setup blog?
DNSSEC has always been OFF
pfsense DNS settings are
@johnpoz said in ISP blocking or interfering with traffic? Cant resolve certain domains all of a sudden:
also per quad9, did you setup the fqdn for forwarding to their dot ?
I believe so (from the General setup)...
Some of unbound's setup
-
-
@pftdm007 that is not the way to forward.. So does that work sometimes not sure...
https://docs.netgate.com/pfsense/en/latest/recipes/dns-over-tls.html#configuring-dns-over-tls
But again if you forward and where you forward sends you back a NX... Its on them, not on the client..
If you ask billy hey you have freds phone number, and he tells you no.. Who at fault? Did you ask for it wrong, did you really mean to ask for freddys number? And hey only knows him by freddy?
I agree if there was some blatent wrong you would think you would have either just horrible failure rate, or nothing would work.. But how your doing it is not the recommended way, and where in there can you validate quad9.. using dns.quad9.net
If you were just forwarding I could tell multiple methods to see if they are messing with your dns, but if your going to encrypt it and they send you back NX, and you trust them, but you didn't validate them - how do we know exactly was going on..
Host reddit.com not found: 3(NXDOMAIN)
NX would come who you forwarded too.. Or he didn't know how to forward?? And telling you hey I don't have that record in my local data..
pfsense config page and everything seems OK to me...
You went over that - but your not doing it that way are you.. Where on that page does it say anything about putting stuff in the custom option box?
An intelligent man is sometimes forced to be drunk to spend time with his fools
If you get confused: Listen to the Music Play
Please don't Chat/PM me for help, unless mod related
SG-4860 24.11 | Lab VMs 2.8, 24.11 -
@johnpoz said in ISP blocking or interfering with traffic? Cant resolve certain domains all of a sudden:
and where in there can you validate quad9.. using dns.quad9.net
Well, first of all in the DNS resolver logs I see a lot of
Mar 4 22:03:28 unbound 27738 [27738:2] info: reply from <.> 149.112.112.112#853 Mar 4 22:03:28 unbound 27738 [27738:2] debug: sending to target: <.> 149.112.112.112#853 Mar 4 22:03:28 unbound 27738 [27738:2] info: reply from <.> 9.9.9.9#853 Mar 4 22:03:28 unbound 27738 [27738:2] debug: sending to target: <.> 9.9.9.9#853 Mar 4 22:03:28 unbound 27738 [27738:2] info: reply from <.> 149.112.112.112#853 Mar 4 22:03:28 unbound 27738 [27738:2] debug: sending to target: <.> 149.112.112.112#853 Mar 4 22:03:28 unbound 27738 [27738:2] info: reply from <.> 149.112.112.112#853 Mar 4 22:03:28 unbound 27738 [27738:2] debug: sending to target: <.> 149.112.112.112#853 Mar 4 22:03:28 unbound 27738 [27738:2] info: reply from <.> 9.9.9.9#853 Mar 4 22:03:28 unbound 27738 [27738:2] debug: sending to target: <.> 9.9.9.9#853 Mar 4 22:03:28 unbound 27738 [27738:2] info: reply from <.> 9.9.9.9#853 Mar 4 22:03:28 unbound 27738 [27738:2] debug: sending to target: <.> 9.9.9.9#853 Mar 4 22:03:28 unbound 27738 [27738:2] info: reply from <.> 149.112.112.112#853 Mar 4 22:03:28 unbound 27738 [27738:2] debug: sending to target: <.> 149.112.112.112#853But I also see some of those:
Mar 4 22:01:40 unbound 27738 [27738:2] debug: tcp error for address 9.9.9.9 port 853Secondly, in the states (as instructed on pfsense's config page), I see a lot of connections to Quad9's servers over port 853:
WAN tcp XXX.XXX.XXX.XXX:62219 -> 149.112.112.112:853 SYN_SENT:CLOSED 1 / 0 60 B / 0 B WAN tcp XXX.XXX.XXX.XXX:49236 -> 9.9.9.9:853 SYN_SENT:CLOSED 1 / 0 60 B / 0 B WAN tcp XXX.XXX.XXX.XXX:29203 -> 149.112.112.112:853 SYN_SENT:CLOSED 1 / 0 60 B / 0 B WAN tcp XXX.XXX.XXX.XXX:21627 -> 9.9.9.9:853 SYN_SENT:CLOSED 1 / 0 60 B / 0 B WAN tcp XXX.XXX.XXX.XXX:63379 -> 9.9.9.9:853 SYN_SENT:CLOSED 1 / 0 60 B / 0 B WAN tcp XXX.XXX.XXX.XXX:48545 -> 9.9.9.9:853 SYN_SENT:CLOSED 1 / 0 60 B / 0 B WAN tcp XXX.XXX.XXX.XXX:16872 -> 149.112.112.112:853 SYN_SENT:CLOSED 1 / 0 60 B / 0 BSince a picture is worth a 1000's words...
-
@johnpoz said in ISP blocking or interfering with traffic? Cant resolve certain domains all of a sudden:
@pftdm007 that is not the way to forward.. So does that work sometimes not sure...
https://docs.netgate.com/pfsense/en/latest/recipes/dns-over-tls.html#configuring-dns-over-tls
Not the way ? I'm curious, as the settings I saw do look fine to me.
Like "configuring-dns-over-tls" explained.Btw : adding this in the custom block :
isn't needed anymore for many years now.
This :
@pftdm007 said in ISP blocking or interfering with traffic? Cant resolve certain domains all of a sudden:
Secondly, in the states (as instructed on pfsense's config page), I see a lot of connections to Quad9's servers over port 853:
could be important. IIRC : there was an identical forum thread some time ago .... 9.9.9.9 (actually : nobody) doesn't like to get chain gunned with 'very expensive' TLS connections. If they apply throttling then that would explain what happens. Normally, unbound shuld keep exiting TLS connections open for future usage .... again IIRC.
And another thing : if I had to push the number of DNS connections up for whatever reason, I would not 'risk' my own WAN "IP" connection, I would use a VPN ISP ^^
Now, if things go bad, the IP would get 'blacklisted" and I don't care : when I'm done, I throw it away ^^
And no you drop in, using the same VPN ISP, and you get my previous IP ...... you get the point ? ^^
9.9.9.9 knows of course you use a "VPN ISP IP", so maybe they throttle you right from the beginning.So : why forwarding ? True : a possible gain : DNS requests will get TLS protected.
On the down side :
Just one DNS end point.
TLS is xxxxx% times more resource expensive.
'They' know where you ....
One last thing :
@pftdm007 said in ISP blocking or interfering with traffic? Cant resolve certain domains all of a sudden:
Without connecting to VPN, the error I get in Chrome/Brave is :
Always double check your browsers for DNS usage !
A browser doesn't have to use the DNS of the OS of 'host device'. For example : it doesn't have to use the DNS IP the OS obtained by using the DHCP-client - normally this is 192.168.1.1, the pfSense LAN IP.
A browser could very well using 8.8.8.8 (over TLS, why not) using not port '53' but some other port.
This means that all browser DNS requests will completely bypass "pfSense", the resolver.
Most browser put in place such a scheme these days. This is because they "want to protect you".
The real reason is : they want to know where you go, as this info is worth $$. -
@Gertjan said in ISP blocking or interfering with traffic? Cant resolve certain domains all of a sudden:
So : why forwarding ? True : a possible gain : DNS requests will get TLS protected.
On the down side :
Just one DNS end point.
TLS is xxxxx% times more resource expensive.
'They' know where you ....You're saying no forwarding with TLS?
@Gertjan said in ISP blocking or interfering with traffic? Cant resolve certain domains all of a sudden:
Always double check your browsers for DNS usage !
Browsers are fine. They all throw pretty much the same error...
@Gertjan said in ISP blocking or interfering with traffic? Cant resolve certain domains all of a sudden:
The real reason is : they want to know where you go, as this info is worth $$.
Most technologies are nefarious to a certain extent nowadays...
-
Quad9 is totally fu***ing up with me. I tried a few other DNS's (should have tried this to begin with, d'oh) and using google's dns (which I refrain due to my absolute and unconditional lack of trust for google) everything is working normally.
So.... I believe that @Gertjan may be right with rate limiting of some sort. Question is why would they rate limit me when I am only a home user? Its not like I'm running servers and websites with TB's of traffic and billions of requests per day....
-
@pftdm007 said in ISP blocking or interfering with traffic? Cant resolve certain domains all of a sudden:
You're saying no forwarding with TLS?
Exact.
Really : try for yourself. Its like magic. Using the DNS as it was meant to be used: getting the info 'from the source' without any intermediates.So, no I do not need to forward to anyone.
-
@pftdm007 said in ISP blocking or interfering with traffic? Cant resolve certain domains all of a sudden:
Question is why would they rate limit me
You as a person with your device : maybe not.
But .... there was (still is ?) an issue with unbound in the was recycling 'expensive' TLS connection very fast .... and forwarders don't like this at all.
Compare with another one 1.1.1.1 or 8.8.8.8 etc.
Or, as I said above : your IP is (was ?) 'suspect' as it is and stays a VPN ISP IP.Btw : when I was testing 9.9.9.9 I saw the same thing ....
-
@pftdm007 said in ISP blocking or interfering with traffic? Cant resolve certain domains all of a sudden:
Quad9 is totally fu***ing up with me. I tried a few other DNS's (should have tried this to begin with, d'oh) and using google's dns (which I refrain due to my absolute and unconditional lack of trust for google) everything is working normally.
So.... I believe that @Gertjan may be right with rate limiting of some sort. Question is why would they rate limit me when I am only a home user? Its not like I'm running servers and websites with TB's of traffic and billions of requests per day....
I have the same issue today with Quad9 which I have been using for a long time without issues.
So either they have changed something today, or UNBOUND has reached a limit or something in its code today that makes it act differently. -
@keyser said in ISP blocking or interfering with traffic? Cant resolve certain domains all of a sudden:
I have the same issue today with Quad9 which I have been using for a long time without issues.
So either they have changed something today, or UNBOUND has reached a limit or something in its code today that makes it act differently.Wonder if this might somehow be related to the massive Meta/Facebook/Google auth outage being reported worldwide (started around 10:00 AM EST this morning). Meta/Facebook/Instagram are massively impacted, but lots of reports of Google auth issues as well.
As of around 12:30 PM EST things are reportedly improving on the Meta/Facebook end of things.
-
@bmeeks Could be in my case. But the OP started his thread 17 hours ago, so that seems unlikely for him. I’ll do some additional testing.
-
@bmeeks Probably unrelated, but I feel compelled to mention since this started for me around the same time (about 5 or 6 PM Eastern time yesterday in the U.S.). I've used unbound with forwarding mode disabled for many years with the outgoing interfaces set my VPN provider client interfaces. I've never had a problem. Yesterday, at the time mentioned, DNS resolution began failing on two independent pfSense machines that I administer that are configured in this way.
I found that if I switched the outgoing interface to WAN (i.e. don't route the queries via the VPN tunnels) it worked again. But since I didn't want this, I enabled forwarding mode and configured system DNS servers, routed via the VPN interfaces, and that's working for me. But I have no idea what changed. From outward appearances, it's as if all the root servers stopped accepting queries from my VPN provider.
-
@TheNarc Is it a more general Internet issue (routing?) that is causing the issue - including meta's challenges?
-
@keyser I wish I knew. I'm not really sure how to diagnose my issue further. Right now I plan to just try switching back to non-forwarding mode in a day or two and see whether it's still broken. I feel like I'd be able to find some evidence of others reporting similar issues if the root servers have just blocked my VPN provider. But I also think whatever is going on in my case must be external to my configuration, both because it did not coincide with any changes I made, and because it began occurring simultaneously on two separate pfSense machines I administer in two separate physical locations.
-
@TheNarc I would tend to agree, but for me coincided with a ISP change (but no pfSense changes - WAN = DHCP).
I'm still unable to poperly resolve names using DOT from pfSense to Quad9. So it's probably something related to TLS sessions/throtteling or whatever.
EDIT: I have no issues using Quad9 as a regular DNS (Not as DOT from pfsense)
-
@TheNarc Hmm, it seems it's related to quad9´s regular DNS name records not resolving correct. Depending on where I resolve, dns.quad9.net does not resolve.
That is needed to resolve for DOT to work (used for certificate verification in TLS setup)Love the no fuss of using the official appliances :-)
-
@keyser Interesting . . . it certainly feels like these things should be related somehow, but I certainly can't definitively tie them together. I'll report back when I try turning off forwarding mode again in a day or two (maybe later tonight if I get impatient, although I tried earlier today and it was still broken).
-
@keyser said in ISP blocking or interfering with traffic? Cant resolve certain domains all of a sudden:
@TheNarc Hmm, it seems it's related to quad9´s regular DNS name records not resolving correct. Depending on where I resolve, dns.quad9.net does not resolve.
That is needed to resolve for DOT to work (used for certificate verification in TLS setup)I think my issue is releated to a bug I have previously experienced pfSense make. Even though SYSTEM -> GENERAL is set for using Remote only (ignore local DNS), it happens pfSense still uses the local service. I have blocked all DOH/DOT server names with pfBlockerNG DNSBL. That seems to cause my own pfsense no to be able to resolve dns.quad9.net at times (thus killing DOT forwarding from UNBOUND).
Today was the first time in a LOONG time I had WAN down, so it might be happening when WAN is gone, and UNBOUND then continues to remember the NXDOMAIN for dns.quad9.net it got from itself when pfSense tried to use the local DNS service instead of the remote DNS
EDIT: But it's quite hard to troubleshoote because pfBlockerNG does not log blocks of DOT/DOH servers like it does blocks from various block lists.
-
@keyser When I reported this issue here, the problem had already been ongoing for about a day or so...
Since yesterday afternoon I added 2 more DNS servers in General Setup (76.76.2.0 & 76.76.10.0 and "p0.freedns.controld.com" for DOT) and everything is back to normal for me..... These new DNS servers are inserted BEFORE Quad9' DNS servers. Everything else is as per the screenshots I posted above....
This morning I got a notice from pfsense that unbound was available to update.
unbound: 1.18.0_1 -> 1.19.1 [pfSense]
