Windows Clients cannot access the internet, very strange unexpected DNS problem.
-
@johnpoz Hmm JohnPoz, you are never gonna believe this. DNS lost its grip, yesterday and the day before. Rebooted twice.
The bottom right task square turned into the world icon. Couldn't connect to any webpage, DNS server unavailable, don't know what caused it. Didn't do troubleshoot either.I don't mind rebooting the firewall once in a while, but if the website comes online with other future stuff, then I'm beginning to worry...
-
@IrixOS you should never have to reboot the firewalll, unless your updating it to be honest..
My pfsense has been up 82 Days 14 Hours 23 Minutes 31 Seconds, and I even had a power outage - but it wasn't long enough that my ups couldn't cover it.
Pfsense rebooted when I updated to 23.09.1, which came out 85 days ago, so I was a couple days behind when it dropped ;) when I got around to doing the update..
If you have an issue with anything - the last thing I would do is reboot pfsense, after you have gathered info and not able to recover by any other means.. If you just rebooted and it then works you have no clue to what was the actual cause.. A reboot of pfsense should be your last thing you do, or if you can not access it at all - not via gui, not via ssh, and also console.. You want info of what is going on before you just reboot something..
-
Well I totally agree with that.
The firewall seems to be unresponsive. Didn't touch anything since the last time we have been troubleshooting.
I don't expect you to go through all the troubleshooting again . The dns server doesn't query.
Frankly I don't know what to think about it right now. It shouldn't behave like that.
-
@IrixOS your on the gui - sure seems like its responsive to me..
Did you try just restarting unbound? do you have internet access even? can you ping 8.8.8.8 from pfsense?
I would have to read over this whole thread to recall what was going on... I can not remember if you have pfsense set up to forward or not.. If its default resolving, then do a dig bing.com +trace so you could see where pfsense is failing in the resolve process.
[23.09.1-RELEASE][admin@sg4860.home.arpa]/root: dig www.bing.com +trace ; <<>> DiG 9.18.16 <<>> www.bing.com +trace ;; global options: +cmd . 85959 IN NS b.root-servers.net. . 85959 IN NS m.root-servers.net. . 85959 IN NS f.root-servers.net. . 85959 IN NS i.root-servers.net. . 85959 IN NS l.root-servers.net. . 85959 IN NS d.root-servers.net. . 85959 IN NS a.root-servers.net. . 85959 IN NS g.root-servers.net. . 85959 IN NS e.root-servers.net. . 85959 IN NS h.root-servers.net. . 85959 IN NS c.root-servers.net. . 85959 IN NS j.root-servers.net. . 85959 IN NS k.root-servers.net. . 85959 IN RRSIG NS 8 0 518400 20240317050000 20240304040000 30903 . p2Z7UhKDT1TGl4a8EAUU1BUrh2fO7VosuHjtHeZxUYmWu/m7iWM7CxG+ /4kfAXn7a3LdKbYTJwt8LdGHJ9F/QKAQ7GjWLlISNPnh3tfgPInoE/sE NpxeV8v0CUvd29gwjZc615XVrzoeyjrVw62Qgzt4+XYiKBFGYXrdC+5L NsZvzeFMGASw8A4QiBTuxYan3f3E++URjF0n7K7O7YhMXPJ5Yuj9rn+k 7WyFJS9Orqrlk8Mqk1tssnSIAMkFe11vTzK/6TvF+NMHIq8J1fv73ZbJ cO2lxdAZv005n+MNz0OMdfubCb8p9iWcCulFYG6sZUzUNmQ+Pcu6IgW3 csxrJw== ;; Received 525 bytes from 127.0.0.1#53(127.0.0.1) in 0 ms com. 172800 IN NS e.gtld-servers.net. com. 172800 IN NS b.gtld-servers.net. com. 172800 IN NS j.gtld-servers.net. com. 172800 IN NS m.gtld-servers.net. com. 172800 IN NS i.gtld-servers.net. com. 172800 IN NS f.gtld-servers.net. com. 172800 IN NS a.gtld-servers.net. com. 172800 IN NS g.gtld-servers.net. com. 172800 IN NS h.gtld-servers.net. com. 172800 IN NS l.gtld-servers.net. com. 172800 IN NS k.gtld-servers.net. com. 172800 IN NS c.gtld-servers.net. com. 172800 IN NS d.gtld-servers.net. com. 86400 IN DS 19718 13 2 8ACBB0CD28F41250A80A491389424D341522D946B0DA0C0291F2D3D7 71D7805A com. 86400 IN RRSIG DS 8 1 86400 20240317050000 20240304040000 30903 . McuSOdxedTIMS8425wT5wRvxIjy9ME426TNSH5qLj1O9pSBBp6OedWXO 1Ye4gn50Ur9FszAsBQ8prkEcqmJNu7mMv3/EzG6PEylJLujrCTxFn2r1 PwivXhfVQY9Aig2c/kS4zAKovDLI2F6hKqkZf17+7pa8wIYpbtVr3Y2Z lRTQSy/GJQ7kscBvnbLHGjHM+pbtp7gf0zhRA5wbCJRqQsWK0Nz866+v c/w0et44EAIRR9iQtljqSIJWmZIheXuC8RO9ZvXlCd8fQJlGen8Kb0Oa Fy8ufrmeNfixNbxR44ncxFqnOU27JZZqQyYnLEHNh8VPFWvdRrl5whdh AtmvaA== ;; Received 1172 bytes from 198.41.0.4#53(a.root-servers.net) in 10 ms bing.com. 172800 IN NS dns1.p09.nsone.net. bing.com. 172800 IN NS dns2.p09.nsone.net. bing.com. 172800 IN NS dns3.p09.nsone.net. bing.com. 172800 IN NS dns4.p09.nsone.net. bing.com. 172800 IN NS ns1-204.azure-dns.com. bing.com. 172800 IN NS ns2-204.azure-dns.net. bing.com. 172800 IN NS ns4-204.azure-dns.info. bing.com. 172800 IN NS ns3-204.azure-dns.org. CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN NSEC3 1 1 0 - CK0Q2D6NI4I7EQH8NA30NS61O48UL8G5 NS SOA RRSIG DNSKEY NSEC3PARAM CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN RRSIG NSEC3 13 2 86400 20240309052607 20240302041607 4534 com. Yno27N6Iyp51X80Wzajfgd8RC57n9zrUGUSfsm1e27HJE+nIFfAHaCBA ea8iXE50HL5TG3xyoq80Y9ixPgwSbg== 5UI7CV5HJHQLPAI73U56DMAO7830VJGD.com. 86400 IN NSEC3 1 1 0 - 5UI7FG7S6MDP7SO5PCHDU0CMCN3K4VOA NS DS RRSIG 5UI7CV5HJHQLPAI73U56DMAO7830VJGD.com. 86400 IN RRSIG NSEC3 13 2 86400 20240308071931 20240301060931 4534 com. bxaYe+AsATtZu+pk+DYfRGcrIFgv5xSRIUAY0qMC+cqL0EYn0PFyASk4 K1DhyvwOUBNP+ithuzt2AE3q/ZYdwg== ;; Received 666 bytes from 192.55.83.30#53(m.gtld-servers.net) in 10 ms www.bing.com. 21600 IN CNAME www-www.bing.com.trafficmanager.net. ;; Received 90 bytes from 208.84.5.204#53(ns4-204.azure-dns.info) in 13 ms [23.09.1-RELEASE][admin@sg4860.home.arpa]/root:Then you would have to follow that cname it points too www-www.bing.com.trafficmanager.net., if your forwarding do you have dnssec enabled - that can cause problems... If your doing dnssec and your ime is off you could have problems, etc..
-
@johnpoz Did you mention time like time settings?
I got this mini firewall with pfsense+ already installed from china.
When inspecting the device, I noticed the BIOS time was wrong. Doesn't matter which save option you choose from the menu, it still does not retain the time setting.
I contacted their support and there was some woman pulling tricks to me and said that the time has to be changed to the Chinese time schedule.Excuse me
?What I did was change the date with the BSD CLI in pfsense, but that was yesterday.
-
@IrixOS when you do dnssec there is a validation, if the the box doing the validation, ie pfsense time is off - then yeah validation can fail.. But really shouldn't matter with the bios.. But if time drifts on pfsense, yeah you could maybe be running into where it works until it drifts out too far..
If unbound fails to resolve you need to figure out why, vs just rebooting and hoping it fixes itself. Can your clients resolve pfsense name? This should matter about external anyway. If unbound is running it should always resolve..
I am showing this currently with nslookup and www.bing.com.. I look to be getting an answer but having some sort of issue with the cnames..
[23.09.1-RELEASE][admin@sg4860.home.arpa]/root: nslookup www.bing.com Server: 127.0.0.1 Address: 127.0.0.1#53 Non-authoritative answer: www.bing.com canonical name = www-www.bing.com.trafficmanager.net. www-www.bing.com.trafficmanager.net canonical name = www-bing-com.dual-a-0001.a-msedge.net. www-bing-com.dual-a-0001.a-msedge.net canonical name = dual-a-0001.a-msedge.net. Name: dual-a-0001.a-msedge.net Address: 13.107.21.200 Name: dual-a-0001.a-msedge.net Address: 204.79.197.200 ** server can't find dual-a-0001.a-msedge.net: SERVFAIL60 second ttl - wtf people..
dual-a-0001.a-msedge.net. 60 IN A 204.79.197.200 dual-a-0001.a-msedge.net. 60 IN A 13.107.21.200stuff like this can cause problems - how hard is it people to run dns ;)
net to a-msedge.net: The following NS name(s) were found in the authoritative NS RRset, but not in the delegation NS RRset (i.e., in the net zone): ns3.a-msedge.net net to a-msedge.net: The glue address(es) for ns2.a-msedge.net (131.253.21.1) differed from its authoritative address(es) (204.79.197.2). -
All of the sudden and as expected...
What can I say. Actually I am struggling with this same issue for quite some time, even years. Now I cannot even vpn anymore.
Didn't touch anything. It bet dns works if I directly connect to the pfsene LAN ip with a laptop in a /30 subnet. I don't quite get its relation with dns.I can't imagine this issue existing on a corporate network.
I am facing another issue along with the internet connectivity issue.
I am a megalomaniac, with a crazy idea having a cisco three tier model network into my home. Ended up with 3 catalayst 3750 and four 4948 series, a couple of virtual servers.
The sound in each room is crazy, it's highly overkill, but I like it the sound.
I looked up the power consumption on the specifications sheet, up to 212Watts per unit. According to AI chat, it's excessive and I am very scared for the energy bill.I got some energy bill a few years ago. About 3000 euro's. Lady said you had the average power consumption for a company. Looked at my stuff, turned white and shut of the whole network with the differential switch. Turned it it was an administrative mistake by the company.
Well this all might turn into reality with the current setup, what do you think? Normally home switches consume a lot less.Don't know what to do right now, planning to deploy a apache webserver for commercial purpose, I am beginning to wonder if it's all worth it. Pfsense didn't came free, it was also the hardware.
All connections and settings are set, there cannot be any fault by my own, it should work unless there is some unknown compatibility issue between cisco and Netgate. It is just a hypothesis. I am very careful by saying this. I see no relation.I remember before my recent network refit, blamed the old hardware for connection problems through pfsense because it was failing, I lost my remote windows desktops too many times over vpn, when the round turning circle appeared in the middle, then I knew there was a connection problem somehow somewhere.
No at the moment I am directly connected with the VDSL modem. Too bad,....
Pfsense can't access the net anymore, even after reboot, done what you asked, the screenshots is the result before the reboot...
-
@IrixOS if you can not even ping googledns your internet is not working, so no you wouldn't be able to resolve anything.. Not even sure why your asking about dns problems - when clearly you don't even have internet access working.
-
@johnpoz I am quite sure internet work because openvpn works, but DNS from the inside to the outside does not. Yesterday nothing worked, I slept this over one night and manually stopped the dns daemon the next day and rebooted, all came backup and dns now works.
I am 100% sure there is something with the pfsense dns resolver, definitely.
-
@IrixOS said in Windows Clients cannot access the internet, very strange unexpected DNS problem.:
internet work because openvpn works,
So your routing traffic over a vpn service? That could cause issues with resolving for sure, many of those services only allow their dns to be used, etc. Make sure your dns does not route over your vpn.
Unbound has zero to do with you pinging googledns, ie 8.8.8.8 - clearly in your post you could not talk to them
-
@johnpoz Look, when the bottom square computer turns into a world icon, the I know there is a problem.
So three things occur:
- No internet access in the browser
- The SERVFAIL message in dnslookup from both the client and dnslookup in pfsense.
- From both the client and pfsense at the command line, ping to 8.8.8.8 fails with the the TTL error.
The VPN was just to test if i can access the firewall and beyond because you talked about my loss of ip connectivity as well.
It just all of a sudden internal clients are not able to resolve and I tried to reproduce the error as expected after a couple of days and I did.
Stopped and restarted unbound daemon and suddenly I have that square icon at the bottom of windows again and I'm online. -
This post is deleted! -
@IrixOS said in Windows Clients cannot access the internet, very strange unexpected DNS problem.:
From both the client and pfsense at the command line, ping to 8.8.8.8 fails with the the TTL error.
This error you described in the quoted text really sounds like either an ISP issue or something going weird with your VPN setup.
If you can't get a repy from a
pingcommand directly to an IP address, then your basic Layer 2/3 connectivity is broken for the client you are trying thepingcommand from. At that point DNS andunboundare totally and completely out of the picture.You may be attacking this problem from the wrong end. Instead of worrying about
unbound, you need to see first what is happening to Layer 2/3 connectivity (that is, why is apingto an outside IP address not working?). Theunbounddaemon should not break Layer 2/3 connectivity for a client.Think about this logically and troubleshoot in a logical manner.
- When the problem occurs, don't restart anything. First try a simple
ping <pfSense_LAN_IP_address>. Does that work? - Next try
ping 8.8.8.8. Does that work?
If neither of the above work, then most certainly DNS resolving is going to be broken and Windows is going to show the globe icon (for no Internet). At that point you need to be troubleshooting Layer 2/3 connectivity to see why the basic
pingto a hard-coded address is not working. - When the problem occurs, don't restart anything. First try a simple
-
@bmeeks Yes not able to ping an external ip address from a client is strange, even though all connections are set and working, the firewall is reachable....There must be some ISP issue....I can hardly believe it's the internal routing.
-
@IrixOS said in Windows Clients cannot access the internet, very strange unexpected DNS problem.:
@bmeeks Yes not able to ping an external ip address from a client is strange, even though all connections are set and working, the firewall is reachable....There must be some ISP issue....I can hardly believe it's the internal routing.
Then I would concentrate all my troubleshooting efforts on figuring out why external connectivity is broken at the basic Layer 2/3 level. Could be something with routing, could certainly be an ISP issue, or it might be the VPN setup in some fashion.
Only after you can 100% reliably ping an external IP address all the way through the network should you start looking at DNS and
unboundissues. -
The client is connected to a switch configured with a local route (L) and advertised into OSPF and propagated the default route to all ospf routers on the ASBR that is directly connected with pfsense.
I also had this issue on a past network setup, but instead with SVIs at that time.
You could be right, it's either the cisco hardware or some ISP isue, the thing is if I connect a laptop or a pc directly to the LAN interface in a /30 subnet, then it works.Programming the switch is very straightforward, what else can I do to troubleshoot with the tools that exist in cisco IOS?
-
@IrixOS from your post above you show a ttl expired from 10.216.64.17 what device is this - is this upstream of pfsense, or some router on your network?
That normally points to a routing loop..
Also you could have some asymmetrical routing going on.. Which depending on what is talking to what, and if there is a stateful firewall in the mix.. Stateful firewalls don't like asymmetrical routing because there is no state, etc.. or with only seeing one side of the traffic the state can expire depending.
But @bmeeks is right on the money (as always) you need to troubleshoot your connectivity issues before you go looking to what can be wrong with unbound.. Unbound is not going to function as it should if your connectivity is broken... And not being able to ping 8.8.8.8 screams of connectivity problem!!
-
@IrixOS said in Windows Clients cannot access the internet, very strange unexpected DNS problem.:
The client is connected to a switch configured with a local route (L) and advertised into OSPF and propagated the default route to all ospf routers on the ASBR that is directly connected with pfsense.
I also had this issue on a past network setup, but instead with SVIs at that time.
You could be right, it's either the cisco hardware or some ISP isue, the thing is if I connect a laptop or a pc directly to the LAN interface in a /30 subnet, then it works.Programming the switch is very straightforward, what else can I do to troubleshoot with the tools that exist in cisco IOS?
It sounds to me that you may have a routing problem. And that problem may take a little bit to manifest itself as all the network equipment does its OSPF stuff. That's not my area of networking strength. @johnpoz will be much more help there as he does this kind of stuff all the time.
But I do know that these routing protocols are dynamic in that the devices participating periodically recheck the paths to calculate the shortest one. On the surface it seems that at some point they calculate something that is "suboptimal"
in terms of staying connected. Restarting and/or disconnecting a port would force a new OSPF algorithm run, and on that run they calculate correctly but then get lost again later and the cycle repeats. -
@bmeeks Yes Cisco use that 'suboptimal' term in all their concepts all the f* time
-
@bmeeks good insight.. Depending for sure - you could get different paths taken, or path could change - it would all come down to the actual setup.. And if there is even multiple paths that could be taken..
But yeah you could be on to something with the routing changing to why seeing issue sometimes and not others.
