Changes in DNS?
-
Have there been changes in the DNS code recently?
This weekend I started getting severe connectino problems. After some investigations I found out the problem isn't related to a specific internet link (I use 4 different providers right now). After some more investigations it looked like some dns requests don't get answered. I think restarting the dns service sometimes helped.
I tried different snapshots and always had the same problems. even snapshots.pfsense.org wasn't found once.
Manually setting a different DNS (not pfsense) helped…
Now I went back to the oldest snapshot online (17th) and it looks as if the problem weren't there in that version. -
Hm, I found something now
(going back to an older snapshot didn't really help, I just forgot to delete the manually configured dns from my system…)I had the option 'Allow DNS server list to be overridden by DHCP/PPP on WAN' active and at least one of the DNS in that list wasn't working. So probably that's what caused the problems. But then, the forwarder code doesn't seem to be very robust, when one non working server out of 8 causes severe problems…
-
Would depend on how that 1 wasn't working.. And were you sending to all, or had you enabled sequential?
Query DNS servers sequentially
If this option is set, pfSense DNS Forwarder (dnsmasq) will query the DNS servers sequentially in the order specified (System - General Setup - DNS Servers), rather than all at once in parallel.If your asking all of them, and this 1 bad one answers with nxdomain - and it comes in first, guess what.. Your out of luck, or if it answers first with wrong info, etc. But if it does not answer then NO it would not have any effect on anything other then pfsense sending out extra packets it doesn't need too.
Now if you were using sequential queries and it was first on the list, then sure your clients on the other side of pfsense might get some timeouts and not resolve stuff because pfsense was wasting time asking something that is either really slow or does not answer (you said it was bad)
Your lack of understanding of how dns works, or the forwarder works does not mean its not robust ;)
A better option vs bouncing around snapshots when you thought there might be something wrong with dns, would of been to actually troubleshoot why you were getting dns issues. " looked like some dns requests don't get answered." Why were they not getting answered - did you think to see if pfsense was sending out the queries, and where they getting answered via a simple sniff? Or just manually doing the queries via say nslookup or dig with either debug or +trace set to see what is happening in the query, etc.
-
I agree with the above. You can't blame all outages on pfsense or even the ISP.
As a for instance, I was getting outages last week and some quick diagnosing revealed that pfsense wasn't the issue and neither was my internet provider. My DNS was failing because about 5 hops down the path from my computer to the dyndns servers there is one particular link that is suffering on average 95% packet loss. This condition has existed between me and all their servers for a week or so. I'm on the East coast. Logging into another of my computers more central USA and using the same DNS servers, there is no issue. Different route, so no packet loss.In short, its not pfsense's fault, not my ISP's fault, not the fauilt of my DNS provider - Its no one's fault I can directly motivate. So, here, I just changed my DNS servers in the short term and I do an MTR about once a day to check and see when/if that weak link is repaired ever.
-
^ exactly.. Troubleshoot a bit is much better course of action vs just trying different snaps.. And blaming something without RCA is just pointless.
-
I set up my first BINDs back in the 90ties, thank you. I think some of them are still running. So I think I have some very basic knowledge at least.
The DNSs that weren't responding weren't answering at all, not NX DOMAIN.
I'm glad your protecting your providers so much. So you think if they provide you with non working DNS servers via DHCP that's clearly not them to blame… How nice of you, but I don't see it that way. One of the ISPs answered I should use 2 different ones or even better 8.8.8.8 und 8.8.4.4. anyway. LOL.
Of course I'm happy to troubleshoot, but first one has to find out where to shoot at.
I was quite surprised to find out that pfsense used the DNSses sent by DHCP, I was quite confident it wasn't that way until at least some weeks ago. Seems I was wrong. Just never game me problems in the past. In fact that's nice, but I just wasn't aware of that fact. Of course I first checked the ones I entered manually and those all worked.I didn't mean to insult anybody but if 2 not responding DNS server out of 8 give that kind of problems that's not my understanding of redundancy. Then it's clearly better to only use 1 DNS. In my opinion not answering DNS should not break lookup as long as there are alternate DNS configured. I also think that's the way it was intended...
(Query secuentally is not active)My intention wasn't to 'blame' anybody (I don't believe that does anybody any good). Could just have been that somebody says: Of course, I changed something in the code recently, and now I see that could be the problem.
-
I didn't mean to insult anybody but if 2 not responding DNS server out of 8 give that kind of problems that's not my understanding of redundancy.
You configured 8 DNS servers? Well, that is pretty amazing, considering the GUI allows just 4. In case you did not and you got those 8 DNS servers assigned via DHCP, I'd like to remind you that some platforms (such as Linux/glibc) allow for only 3 nameservers. Might be something for your ISP to think about. Since if their first 3 DNS servers are useless, then no others will ever get used.
-
I didn't mean to insult anybody but if 2 not responding DNS server out of 8 give that kind of problems that's not my understanding of redundancy.
You configured 8 DNS servers? Well, that is pretty amazing, considering the GUI allows just 4. In case you did not and you got those 8 DNS servers assigned via DHCP, I'd like to remind you that some platforms (such as Linux/glibc) allow for only 3 nameservers. Might be something for your ISP to think about. Since if their first 3 DNS servers are useless, then no others will ever get used.
Yes I'm talking about the ones assigned by DHCP. And as I said I have 4 providers, so every provider just gives me 2…
-
Best course of action would be to stop using those DHCP-assigned DNS servers at all. If 25% of them fails at best, clearly those are useless. Either set up your own or use the public ones, such as Google public DNS, OpenDNS or whatever.
-
I'd agree with you in most cases, but just when you think you have gone and made something like DNS idiot proof, they go and invent a better idiot. I wanted to see how badly I could shoot myself in the foot, so just to be stupid, I loaded 21 DNS servers on a VM. (It won't be staying that way)
-
Best course of action would be to stop using those DHCP-assigned DNS servers at all. If 25% of them fails at best, clearly those are useless. Either set up your own or use the public ones, such as Google public DNS, OpenDNS or whatever.
Yes that's what I did. Like I said I wasn't even aware they ware used in first place.
Maybe we could learn one thing from the whole story: Maybe it's not really clear what happenes when there are DNS servers configured in 'General setup' and provided by DHCP as well. Are the manually set used at all? is only the assigned gateway used if the same server is provided via DHCP etc…
PS: Another thing. The gui says:
When using multiple WAN connections there should be at least one unique DNS server per gateway.Now, you can only enter 4 nameservers in 'general setup'. Maybe that's why I subconsiously used the DHCP provided ones? I used to have 5 WAN links, so I wasn't able to provide one DNS per gateway….
-
Yes you would normally want to have atleast 1 dns server per wan connection.. In case your other connection goes down, etc.. If that name server is only available via that connection.
Here is the thing with ISP dns - they are normally only able to be queried from their NETWORK!! So if you have multiple wan connections, which path are you taking to the name servers IP? Since its unlikely the name server is on the same segment the connection is on. You could be taking any of your other connections paths to try and get to a specific IP - what is your default route, do you have specific routes setup for those dns IPs?
So if your having issues doing queries to ISP based dns – its quite possible your trying to hit them from a source IP that is not their network. And then yeah they most likely will not answer you.
Again - your lack of understanding does not mean a system is not robust ;)
-
Again - your lack of understanding does not mean a system is not robust ;)
Please, could you stop making a fool of yourself? I've set up RIP, OSPF, EIGRP, static and last, but not least BGP4 routing in the 90ies, I've built an ISP we sold in the year 2000 so you can guess I know some things about routing. I'm even capable of distinguishing between 'not reachable' and 'no dns service running'.
Anyway, even if my routing would be screwed up, having 2 DNS servers that are not reachable (never mind the reason) breaking pfsense couldn't be called robust, could it?No, don't answer, I already know the answer… My lack of understanding is responsible for every bug that ever had been in pfsense…
-
Oh, I see… My DNS servers are unreachable -> pfsense suxxxx, it does not resolve. Makes a lot of sense. facepalm
-
I tend to prefer public servers. I've been testing the OpenNIC servers for a while to see how reliable they are.
I usually give pfsense 4 geographically separated DNS servers not too far away and then point all the clients at pfsense only.
I think we should all have about 3 double espressos and chat this some more ;D
Maybe during a traffic jam on the way home… -
If I were OP I would turn off the DNS forwarder in pfSense and set up a couple or three local, caching name servers (with no forwarders configured) and point my local clients at them.
They would do recursion on behalf of the clients using whatever WAN links happen to be available at the time. They would only be seeking answers from authoritative servers so the "local queries only" problem with multiple WANs would not exist.
I would completely disregard the name servers the WAN links set.
-
Oh, I see… My DNS servers are unreachable -> pfsense suxxxx, it does not resolve. Makes a lot of sense. facepalm
Probably you had too many facepalms.
What do you have several DNS for? Redundancy? So, if 2 out of 8 don't work, of course it's normal that name resolution doesn't work anymore? -
I tend to prefer public servers. I've been testing the OpenNIC servers for a while to see how reliable they are.
I usually give pfsense 4 geographically separated DNS servers not too far away and then point all the clients at pfsense only.
I think we should all have about 3 double espressos and chat this some more ;D
Maybe during a traffic jam on the way home…My clients are pointing to pfsense, too (caching…). I still like to use the ISP nameservers when ever possible? Why? My internet connections aren't the fastest ones and no DNS can be nearer than the one of the ISP - possibly one with an overloaded upstream…
-
I am swilling coffee as we speak and also taking isoproterenol (an adrenaline antagonist).
I'll be ready to share my feelings on DNS forwarder function in pfsense momentarily.As far as "fast", I agree that the local ones ping faster but once the local ones have proven unreliable, fast doesn't matter.
I'd prefer reasonable ping time + reliability over speed. Especially once I realized that when one of my WAN links drop that DNS server is just going to become a big speed bump in my internet. -
If I were OP I would turn off the DNS forwarder in pfSense and set up a couple or three local, caching name servers (with no forwarders configured) and point my local clients at them.
They would do recursion on behalf of the clients using whatever WAN links happen to be available at the time. They would only be seeking answers from authoritative servers so the "local queries only" problem with multiple WANs would not exist.
I would completely disregard the name servers the WAN links set.
I do disregard them now. But don't you think your setup is somewhat an overkill for a private household? 3 additional nameservers? Disabling the DHCP provided DNS already solved my problems, I think that's good enough for me. By the way, WAN links weren't the problem, there the failover works. And there's no 'local queries only' problem, the routes are correct. Of course, I don't know wether pfsense is smart enough not to query over a gateway that is marked down… But I guess so.
Well I have one BIND running in my network already, of course I could use that one. On the other hand I have to reboot that machine from time to time…
