DNS Headaches Since Switching to PFSense
-
@gertjan Thank you so much for the helpful reply. I have removed the external resolvers as they were unnecessary.
The challenge that I'm having is that everything works perfectly with other firewalls. It's only when I have PFSense running (again, as opposed to EdgeRouter or Sonicwall) that I randomly have the issue. I never had the issue with those other platforms.
I assumed that it was DNS related because it almost looks like the app can't resolve or isn't connecting to the internet, BUT I have a constant ping (to Google) running on a server that has no gaps. There's no internet outage here.
Does anyone have any other leads for me to chase down?
-
@ashkaan And what app is this?
So what IP does your phone get.. Why don't you sniff (diag menu packet capture) and actually see what is going on.
So this title should be changed - because you really have no clue to what the issue is.. You don't even know where your app is trying to go? Or what its trying to do..
If you were having dns problems - that should present itself as lots of stuff not working, or atleast this one thing you know not working on every device using pfsense for dns, etc. etc.
Out of the box pfsense does no filtering outbound, and does not block any dns.. Have you changed this default? Are you using pfblocker? Are you running IPS/IDS?
Turning wifi on and off on the phone would do ZERO to pfsense, sure and the hell wouldn't fix a dns issue on pfsense, etc. So if turning on wifi on your phone and back on fixes your issues - that screams something wrong with your phone or your wifi..
So you have no wired devices? Do they have problems?
-
@johnpoz It's many apps from games to crypto apps to Nest. It's not a single app, but the phones and iPads just stop working for a moment. I switch to cell service (off the network) and they work perfectly. The server never misses a ping. This doesn't happen on other firewalls
It would be impossible for me to guess when it will happen to run a packet capture at that exact time. It also only happens a few times a week, sometimes once a day.
Feel free to change the title. From my experience, this looks like a DNS issue and I don't have evidence to the contrary at the moment.
This is a VERY default setup. I don't like complicating things so I have not messed with anything.
Yes, I understand that turning off WiFi would not DO anything to PFSense, but it proves that there's something wrong with the network. Again, the WiFi works great with other firewalls, so we know the PFSense is the common denominator here.
My only wired device is my server and I don't browse the net often enough on it to tell. I can tell you that it never misses a ping. I suspect (only guessing) that because all other devices that I regularly use have the issue that it also has the issue since I ruled out the WiFi being the issue.
-
@ashkaan said in DNS Headaches Since Switching to PFSense:
It also only happens a few times a week, sometimes once a day.
Well that points to the dns restarting issue we have brought up a couple of times already..
unbound goes off, you go to switch your wifi off and on and by time you come back its restarted and you think turning off wifi and back on fixed it.
Turn off registration of your dhcp clients.. Watch how often unbound is restarting - and the next time you have the issue, look in your logs - did unbound just happen to restart?
Next time it happens - before you go flipping yoru wifi on and off - do the directed query I gave as example to something, www.google.com - or something you haven't gone to in a while so your sure its not cached, etc.
If you want to get to the root of the problem your going to have to do something more than flipping yoru wifi on and off..
-
@johnpoz Yes, thank you for reminding me. Ok, I'll test that now. Just to confirm, it should look like this to prevent further DNS reboots?
-
@ashkaan yes no registering dhcp clients should keep unbound from restarting every time a dhcp client renews or gets an IP.. You can check your unbound uptime with this.
[22.01-RELEASE][admin@sg4860.local.lan]/: unbound-control -c /var/unbound/unbound.conf status version: 1.13.2 verbosity: 1 threads: 4 modules: 2 [ validator iterator ] uptime: 334795 seconds options: control(ssl) unbound (pid 32247) is running... [22.01-RELEASE][admin@sg4860.local.lan]/:So mine has been up for 334795 seconds or 93 hours.. I have seen uptimes much much longer than that - but I tend to restart mine quite a bit testing stuff for users and their threads.
You can register static if you want, these would be like your host overrides..
-
@johnpoz Ok, thank you so much! I'll keep everyone posted how this goes.
-
@ashkaan
We know already how it will go ;)Just check your resolver log file in the GUI, and look for the 'stop', where unbound logs that is about to stop - to be restarted right away.
The frequency will be a lot lesser, maybe one a day or even once a week.
Btw : saving DNS or DNS related settings will restart unbound also.If this was your issue, it will be gone.
unbound could still be restarted, as there are many more reasons why it would restart, but these are (should be) less frequent.
And remember : even if unbound restart, DNS will be stopped for just a couple of seconds.
That should be a non-event for devices and users.Things start to be noticeable when the network changes often :
The admin rips out LAN or WAN cables : this creates a network OS event, and 'attached' processes that listen on these interfaces will restart.
A NIC or cable could be bad : it loses the carrier every x seconds, then it works fine for hours, the result will be the same. Random network access.
Or : a switch is hidden in the attic has a bad power supply : it resets very often, all ports go down - and this will create the same event as mentioned above.And the most important one :
As people can't smell, see or hear radio signals, they are considered to be "working perfectly well".
Or, in reality, the contrary is most often the case.
Just last weekend a showed a friend why it's iPhone was horribly slow, or, he wanted to show me his latest unifi-ap-6-lite (long range !) that did not from well a 30 meters (we use meters here, as our feet do not have a fixed size).
I showed him that his iPhone negotiated a less then 2Mbits/sec connection. Now my friiend is member of the club : "Ok for the high speed, but transmit/receive range size really became very small"
The radio frequency spectrum is often a mess. radio communication is a mess. People just don't stop adding 'connected' devices, and because they can't "see" what these devices are doing they are presumed to "will work just fine". What happens to constructive pessimism ?
Depending on what you said when you were at school, they would have gave you a a spectrum analyser (instead of a adjustable wrench), and you would have seen the invisible : lets look at the classic FM band, the band where the TV was - here in Europe it was called the VHF & UHF band. VHF and the top of UHF has been recycled now. TV is now compressed the digital way, 5 or 6 channels a carrier, and the space that was freed up is now called : 5G (even more mess). Go a bit higher and you enter the micro wave frequency. And the Boeing 737-x00 alti meter, a device used to stabilize the approach by measuring the distance between the plane and ground.
I saw what was there in the end of the '70. No wifi bleutooh and only 3 national TV channels back then.Be assured, I'm not against the usage of Wifi (I'm not Swedish
) but it is an important "does not work well" factor.Same thing for our ISP upstream connection : people tend to think, "I pay, so it works".
But noop, ISPs are not all identical but they do all the same thing : they give you what they have. They all sell their bandwidth "many times". So, many clients have the impression that they "can't get it all". Or worse, (small) DNS packets get dropped.
Or, and this should happen a lot : for the 8.8.8.8 lovers : what happens when 8.8.8.8 gets over run ? Well ? people drop in here, and say there are issues with pfSense. The ones who know a little bit more would say "DNS error"
The even smarter ones would say ... nothing. They'll suddenly ask themselves : why should my resolver forward to a resolver ? resolvers like 8.8.8.8 use the 13 main root internet servers. So pfSense could do the same ?!!![ ranting mode = stop]
-
@gertjan Haha! Well, thank you very much for the insights. I’ll keep a look out and let you know how it goes.
-
@ashkaan Still happening. It happened at 8a yesterday and again at 832a this morning.
Unfortunately, there's nothing in the DNS logs about this around those times. I'm totally baffled. Where should I look next?
Again, the symptom is that apps and websites in Safari will just hang. As soon as I switch to Cell, they load. My server hasn't dropped a ping and I've since unplugged my desktop and put on WiFi and started a ping and that's solid too.
-
@ashkaan said in DNS Headaches Since Switching to PFSense:
there's nothing in the DNS logs about this around those times
So your unbound has been up for how long, once you had the issue. Did you try a directed dns query?
What specific website did not load - you need to have a specific thing that is not working if you want to troubleshoot..
What exactly are you pinging? By IP or by name? Are you pinging the specific site that is not working?
Are you forwarding for dns, or did you change to resolving? Is your client using doh for dns? Are you using IPv6 or just IPv4.. Without specific it is impossible to try and figure out what exactly is going on that your seeing the issue your seeing.
-
@johnpoz I was in a work emergency so I didn't have time to try a direct query, but it was google.com of all sites! Google wouldn't load in Safari. On cell, it loaded immediately. The WiFi desktop didn't lose a ping to 8.8.8.8.
It's been up since midnight of the prior day, so it wasn't stopped at the issue times.
I was using Resolving, then I tested Forwarding for a couple of weeks, then I went back to Resolving about a week ago. No DoH. Only IPv4.
-
@johnpoz, with https://redmine.pfsense.org/issues/12613 still not being fixed I would honestly be more surprised if it did work in a typical home setup then when it did not ... and while we're at it - please fix it properly!
-
@thiasaef said in DNS Headaches Since Switching to PFSense:
https://redmine.pfsense.org/issues/12613
He would be able to tell if that was what he was running into - by simple query for something. And he is never restarting dns - so how would that be the issue. Just flipping a client wifi on off wouldn't restart unbound, etc.
I don't see how there is enough info to say what is going on one way or the other.
What I can say is I have not run into any issues, nor that specific issue at all.. I don't use a vpn for vpn traffic, nor have unbound bound to the test vpn connection I leave up. I don't have multiple interfaces that could be going up or down.
And I bind unbound only to loopback anyway, and just let it nat outbound, etc. So even if my wan went offline I should be fine, but my wan connection is pretty rock solid as well.
What needs to happen to move forward here is some actual specifics of what is not working, and when. I get it you don't always have time to troubleshoot when something goes wrong, and you just want it back up now.. But if want to get to the bottom of the actual problem - going to need to actually get some details when it happens..
-
@johnpoz Ok, so I'll try to resolve the site I'm trying to resolve on mobile on my server? Or, where should I attempt to resolve?
-
@ashkaan yes you should do a directed query to unbound on pfsense (this is what is providing you dns?).. From sure your wifi device, and also from something else on your network.
Does unbound on pfsense provide a response - if so what, the IP, some sort of failure or refused, etc.
For the actual fqdn that is failing, and as well something else. Do nothing non local resolve - just this one site, or sites that are failing. What about something local like pfsense fqdn, etc.
-
@johnpoz said in DNS Headaches Since Switching to PFSense:
I don't see how there is enough info to say what is going on one way or the other.
Neither do I, but I also did not say that the above problem was necessarily his problem.
@johnpoz said in DNS Headaches Since Switching to PFSense:
What I can say is I have not run into any issues, nor that specific issue at all.. I don't use a vpn for vpn traffic, nor have unbound bound to the test vpn connection I leave up. I don't have multiple interfaces that could be going up or down.
And I bind unbound only to loopback anyway, and just let it nat outbound, etc. So even if my wan went offline I should be fine, but my wan connection is pretty rock solid as well.
This mentality is what is driving me crazy right now. Answers like this feel like being slapped in the face.
-
@thiasaef said in DNS Headaches Since Switching to PFSense:
. Answers like this feel like being slapped in the face.
huh? What does an issue with an interface going down and no longer bound to a service have to do with this issue?
You said a typical home setup right - well I am in a home, and what am I doing that is not typical for someone using pfsense in their home? From your comment - you make it sound like that specific issue would be causing everyone with a home setup issue. I don't see how that could be the case really..
I have not seen that issue, I have no input to them fixing it how ever they are going to fix it - be it "properly" enough for you or not, etc..
Not even sure why you brought it up here - since I don't see how it could have anything to do with the current threads issue.. Unless I am missing something? Where has he stated that when his problem starts after an interface outage, and he has to restart it to get anything working?
-
@johnpoz said in DNS Headaches Since Switching to PFSense:
What does an issue with an interface going down and no longer bound to a service have to do with this issue?
I agree that we do not know the root cause of the Unbound restarts @Ashkaan is seeing yet.
From your comment - you make it sound like that specific issue would be causing everyone with a home setup issue.
That's not what I said, sorry.
Not even sure why you brought it up here
Because it is possible that his problems are related to the bug and because other people with other Unbound issues will likely stumble across this thread and probably be happy if they find a solution.
@johnpoz said in DNS Headaches Since Switching to PFSense:
be it "properly" enough for you or not, etc..
It did work perfectly fine on 2.4.5-p1, so all I am asking for is a regression bug being fixed instead of being swept under the carpet.
since I don't see how it could have anything to do with the current threads issue..
The symptoms do match and there are Unbound restarts in his screenshots that are not caused by the DHCP registration bug.
Where has he stated that when his problem starts after an interface outage
Never, but how likely is it, that someone will see a connection between the two things without knowing that this bug exists in the first place?
-
@thiasaef said in DNS Headaches Since Switching to PFSense:
I agree that we do not know the root cause of the Unbound restarts @Ashkaan is seeing yet.
What we do know is that the "DNS Registration", ones disabled, will not restart unbound any more.
Btw : unbound will still restart. Here is mine does.
The restart are : Me, as I'm messing around with pfSense, It could be reboots, it could be pfBlockerNG-devel after feeds gets updates (ones a day or ones a week).
But the graphs show that this happens ones a day, or less.
earlier in the thread we saw a lot of "unbound stop". These should come with a 'start' :
grep 'start\|stop' /var/log/resolver.logSome one, or something, another process, is restarting unbound. This could be the pfSEnse package pfBlockerNG-devel, I know because I'm using that pfSense package.
It could also be a hardware 'link' event. I just ripped out the WAN cable of pfSense, and put it back in : unbound was restarted. For the LAN interface the behaviour would be identical (I didn't test) These events should be very rare.unbound does not restart itself, and I did not saw it crashing.
Btw : @Ashkaan :
This :
You've treated the "hostnames" column as some sort of comment field.
That's wrong.
A host name should be a host name. And not only that, it should be 'PTR' of the IP :
1.1.1.1 == one.one.one.one.
8.8.8.8 == dns.google.
