I Think This is a DNS Issue...
-
I'm having trouble with my internet connection and have already posted and received help on another thread. I was pretty sure, at that point, that the issue was our internet router for several reasons, but the deeper in I go, the more I think I have something like a DNS issue.
I'm now using a graph in Status/Monitoring to keep track of my internet connection, using the 8.8.8.8 Google name server for my Monitor IP and since I started using that, I've had no lost packets. I have had trouble pinging sites, but since there's packet loss for pfSense monitoring the connection, now I'm thinking that when I couldn't ping a site before, it was more likely the server not responding right away than it was that my internet router was down.
Here's what we're seeing:
- Sometimes we try to go to a site in Chrome and get "No internet connection." Or it might be, "Server Not Responding," or similar messages.
- Streaming sites show no thumbnails. I know streaming sites like Disney or Netflix often serve thumbnails and graphics from different domains than they use for their content.
- Sometimes we can pull up the page for a streaming site, but not stream any content.
- It often takes notably longer, when we type something in a search bar, to get a response from a search engine and even when typing in a domain directly, using a bookmark, or clicking on a link, it takes longer to connect than it should. (Sometimes it's even slower than when we were on satellite internet.)
- All issues are intermittent. Things can work well for a while and suddenly it's like there's no internet connection at all. The delays in establishing a connection can persist for long times, but actual "outages" are more intermittent.
I can post screenshots of my configurations for any help with straightening this out, but at this point, I'm not sure where to begin and I'm hoping that I'm right in completely ruling out my ISPs router as the culprit.
I have posted DHCP and DNS Resolver logs in the other thread I linked to above. There's a lot of continual activity for both services - as in several log entries each second. Is that normal for a LAN covering two houses with a number of Sonos speakers and various devices like tablets, phones, Blu-Ray players, Apple TVs and so on?
-
Well, without knowing more about your setup, it's hard to say, but based on "my ISPs router as the culprit" it sounds like you are double natted, PFSense behind another router? Is the PFSense router in the DMZ of the ISP router (getting an internet route-able IP from the ISP router)? Or is your "ISP router" only a cable/network box and not a router? If you think your issue is DNS, there are a few tests in PFSense that you can do. Diagnostics tab, DNS Lookup- do a lookup or three of some internet domains that have been problematic. Do all the DNS servers listed in SYSTEM/GENERAL respond? Do any timeout? From a command prompt, if you ping a domain that you are having issues getting to, does the IP resolve or do you get "ping request could not find host XYZ"? If it resolves, does it ping? Are any pings failing? Remember that DNS only resolves the IP from the name, once you get an IP, its work is done. If pings drop, that is not a DNS issue. If IFRAMES in a website (parts like advertisements and other subframes) fail to load, the DNS lookup for them may be failing. Do you have any packages installed (Snort...PFBLOCKER...) possibly blocking them?
If your ISP box is a router, can you connect one computer directly to that and test going to sites that can't resolve correctly in PFSense. Try a ping -t of a problematic site from there for a bit, to check for packet drops with PFSense out of the picture.
Remember, ping packet drops are not a DNS issue. You can see the resolved IP.
What kind of NICs are in your PFSense box. I hate to sound like a broken record parroting what everyone here says, but don't rely on Realdrek NICs. -
I've been working on some experimenting today and was just finishing up. I'll go into that at the end of the post. (Along with some other background that might help.)
@riften said in I Think This is a DNS Issue...:
Well, without knowing more about your setup, it's hard to say, but based on "my ISPs router as the culprit" it sounds like you are double natted, PFSense behind another router?
We have a wireless broadband connection. (Basically it means our ISP resells cell data for home internet use. Considering it's that or Viasat, which is horrid, it's pretty good.) The wireless router offers wifi (which I don't use) and one RJ45 for the LAN. I have an ethernet connection running from the wireless router to the WAN side of my Netgate SG-1100. Then I have an ethernet connection on the LAN side of the SG-1100 to an ethernet switch. Connected to the switch is a wifi hub, but I only use wifi for mobile devices. Desktop computers, Apple TVs, blu-ray players, Sonos speakers - all that kind of stuff is hardwired with ethernet connections to the switch.
The wireless router, on the LAN side provides a DHCP server. It uses the 192.168.xxx.xxx address space and it is at 192.168.0.1. The Netgate's WAN, connected directly to the wireless router's LAN connection, is usually 192.168.0.180.
Is the PFSense router in the DMZ of the ISP router (getting an internet route-able IP from the ISP router)? Or is your "ISP router" only a cable/network box and not a router?
Definitely a LAN address, not one for the open internet.
If you think your issue is DNS, there are a few tests in PFSense that you can do. Diagnostics tab, DNS Lookup- do a lookup or three of some internet domains that have been problematic. Do all the DNS servers listed in SYSTEM/GENERAL respond? Do any timeout? From a command prompt, if you ping a domain that you are having issues getting to, does the IP resolve or do you get "ping request could not find host XYZ"? If it resolves, does it ping? Are any pings failing? Remember that DNS only resolves the IP from the name, once you get an IP, its work is done. If pings drop, that is not a DNS issue. If IFRAMES in a website (parts like advertisements and other subframes) fail to load, the DNS lookup for them may be failing. Do you have any packages installed (Snort...PFBLOCKER...) possibly blocking them?
If your ISP box is a router, can you connect one computer directly to that and test going to sites that can't resolve correctly in PFSense. Try a ping -t of a problematic site from there for a bit, to check for packet drops with PFSense out of the picture.I did connect directly to the router via wifi with my iPad and used Blink, which gives me a command line shell on it. First, when my wife was having issues with being able to load any sites on her computer, I connected to cell data and could easily ping the external address on the ISP's router. I got good results. Then I connected to the ISP router with wifi and could ping any site I tried. So while my wife was having issues, the router connection was good and a connection going directly to the router, then to the internet was working fine.
Remember, ping packet drops are not a DNS issue. You can see the resolved IP.
What kind of NICs are in your PFSense box. I hate to sound like a broken record parroting what everyone here says, but don't rely on Realdrek NICs.Yeah, Realtek has sucked for many years!
There's more background. I had been using an old version of pfSense for years, on an old Soekris Net5501, bought back when they were a newer model. I had been updating my version regularly, but then there was an issue at some point and I stopped upgrading it. So the version of pfSense I was using is now over 10 years old. I didn't count on a configuration from that long ago working in a new version, since I had no idea how much had changed since then. I took screenshots of all the settings from the old system, then plugged in the new one, set up the DHCP leases and tried to do things as close to possible to the old system.
My one change was using the DNS Resolver instead of the DNS forwarder. It seems like that's been my trouble.
For troubleshooting, today, I saved the full configuration from the Netgate and saved just the DHCP configuration, with the new leases I'm using, in a separate file. Then I removed the Netgate and replaced it with the old Soekris box, with the more-than-a-decade-old pfSense on it. I took time to test the setup. Almost immediately, I lost my DVR (which is software running on a computer) and a DLNA server because they're both on a computer that did not have a lease on the old DHCP server. But the other problems were gone.
I had not been able to stream Disney+ at all and saw no thumbnails when I tried using it on Apple TV. With the old firewall in place, it worked perfectly. So did Netflix in Chrome. Left it running for a few hours and my wife had no trouble with anything internet related. Then I tested it by loading the DHCP configuration from the SG-1100 and the DVR and DLNA server both returned. Then I saved the configuration.
I swapped in the SG-1100 and turned off the DNS Resolver and tested. Waited and tested again. All the problems were gone for both tests. Left it that way for hours. Wife had no problems and when I came back to it later, everything was behaving.
I think that narrows it down to something I did wrong with the DNS Resolver. (Remember, on the old system, I had used the DNS Forwarder and thought it would be better to try the Resolver on the new system.)
I went over the pfSense docs on the Netgate site and tried making sure I set up the DNS Resolver according to their instructions. I saved, then applied the changes. The problem is I could get results on LAN systems with NSLOOKUP, but not with PING. PING can't find anything on my LAN. Even after a few hours (in case of propagation delays).
I deactivated the Resolver and tried the Forwarder. Same thing. NSLOOKUP and no PING.
I've stopped the internet connectivity problems, but it seems like I can't get get either the Resolver or Forwarder to do the DNS work for my LAN domain.
-
I got it working, but don't understand why. I'd appreciate help in understanding why one change made it work:
I had been trying DNS Resolver and DNS Forwarder (never at the same time). Each time I made sure I was following the directions in the Netgate documentation to set them up. By default, on DNS Resolver, the "DNS Query Forwarding" is not checked. The Netgate docs for this feature say:
Enable Forwarding Mode: Controls whether Unbound will query root servers directly (unchecked, disabled) or if queries will be forwarded to the upstream DNS servers defined under System > General or those obtained by DHCP/PPPoE/etc (checked, enabled). Forwarding mode may be enabled if the upstream DNS servers are trusted and also provide DNSSEC support. Forwarding mode is necessary for Multi-WAN configurations unless default gateway switching is enabled.
I had left this unchecked, since it was unchecked by default when I reset to factory defaults and started over. But, after several attempts to get the Resolver working, I checked it, saved, and applied. No joy. So I went to Settings/General and added 8.8.8.8 for DNS, saved and applied and it worked.
Why is it that this wouldn't work unless I specifically added the name server? And why is this unchecked by default?
-
@TangoOversway said in I Think This is a DNS Issue...:
And why is this unchecked by default?
Because, by default, The Internet works just fine without Google's DNS servers like 8.8.8.8.
The default Resolver settings work out of the box.
Your DNS not working 'out of the box' indicate a networking problem.8.8.8.8 is needed if you have to communicate all your private DNS searches over to Google.
-
@Gertjan said in I Think This is a DNS Issue...:
@TangoOversway said in I Think This is a DNS Issue...:
And why is this unchecked by default?
Because, by default, The Internet works just fine without Google's DNS servers like 8.8.8.8.
The default Resolver settings work out of the box.
Your DNS not working 'out of the box' indicate a networking problem.Can you give me ideas for what to look for? Could it be that the ISP router is reading requests from the DNS Resolver to the internet? My Netgate SG-1100 is the only thing I have on my LAN that is supposed to be handling stuff like that. I don't use the wifi on the internet router, other than for testing, so would turning off all DNS functions on that router be a good idea?
8.8.8.8 is needed if you have to communicate all your private DNS searches over to Google.
So if have the domain rivendell.me on my LAN, and I do
ping bagend.rivendell.meon one computer, is that forwarded to 8.8.8.8 in this situation? Or does pfSense see that's in its own domain and resolve it there? -
If you have enabled DNS forwarding the query is forwarded to whatever DNS servers are defined on the system. That's whatever is in System > General and whatever your ISP is handing you via DHCP, if you have that enabled.
Did you actually at any time see anything fail to resolve? The issues you're describing, parts of websites not loading, things not responding to ping, may not be DNS issues at all.
Steve
-
@stephenw10 said in I Think This is a DNS Issue...:
If you have enabled DNS forwarding the query is forwarded to whatever DNS servers are defined on the system. That's whatever is in System > General and whatever your ISP is handing you via DHCP, if you have that enabled.
But it handles it if it can, then forwards what it can't, right? So if it's linked to the DHCP server and can resolve the request there, then it doesn't forward it?
Did you actually at any time see anything fail to resolve? The issues you're describing, parts of websites not loading, things not responding to ping, may not be DNS issues at all.
Yes. There were times I would get no resolution for host or nslookup or ping. Other times host and nslookup would work but ping would not find the domain or machine. I had times where I'd have that issue within my LAN and times when I'd have it outside of my LAN.
-
Yes it will return anything that's defined locally directly. So host overrides and dhcp leases if you have that enabled.
Steve
