When router reboots, clients get DHCP and LOCAL LAN DNS but no internet until i toggle "Allow DNS server list to be overridden by DHCP/PPP on WAN or remote OpenVPN server" and restart NIC

[Eddie55]

EDIT: I think this HAS to be a bug
TLDR: AFTER a pfsense reboot my pfsense router gets a valid WAN IP everyhing works fine, i can ping local computers and ping external websites using DNS without issue
My LOCAL clients get DHCP and a valid local IP address and LOCAL DNS works fine BUT no internet> i cant ping anything external via ip or dns
THE ONLY WAY to get my LOCAL DHCP clients internet access working is to simply go to
System > General Setup > Allow DNS server list to be overridden by DHCP/PPP on WAN or remote OpenVPN server
And either ENABLE or DISABLE this option, it DOES NOT matter if its checked or unchecked!
I just have to make sure to do the opposite of whatever it is currently set at and the click SAVE
and then
I need to restart the network device on the clients

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=

ORIGINAL

So i was happily using pfsense for many years on my old SG-2440 until it died
I decided to get this
https://cwwk.net/collections/frontpage/products/intel-12th-generation-n-series-8-core-new-member-n95-n100-n200-n305-fanless-mini-host-soft-router?variant=44409609322728
2.7.0-RELEASE (amd64)built on Wed Jun 28 03:53:34 UTC 2023FreeBSD 14.0-CURRENT

Network Setup
Fiber > AT&T U-verse Pace 5268AC Residential Gateway (192.168.1.254 )(Set to DMZplus mode to which forwards the public IP address to the PfSense Box) > Pfsense box (192.168.0.1)> unifi 24 port switch > all my desktops and Unifi AP are connected here (192.168.0.x/24)

Everything seems to work great EXCEPT
Sometimes when the power to the IT closet goes out my AT&T U-verse Pace 5268AC Residential Gateway AND PFsense router restart

When myAT&T U-verse Pace 5268AC Residential Gateway AND PFsense router turn back my pfsense router gets a normal public IP and i can ping external sites like google using DNS
BUT
The local network devices get DHCP but for some reason cant ping external sites like google
THE LAN works PERFECTLY fine and i can ping things VIA name and IP address LOCALLY

THE ONLY WAY to get my clients back working is to simply go to
System > General Setup > Allow DNS server list to be overridden by DHCP/PPP on WAN or remote OpenVPN server
And either ENABLE or DISABLE this option, it DOES NOT matter if its checked or unchecked!
I just have to make sure to do the opposite of whatever it is currently set at and the click SAVE

ONCE i do that then i need to restart the network interface of the clients
ANY ONE OF THE 3 METHODS WORK FINE
or
unplug and replug the network cable
or
restart the interface (sudo systemctl restart NetworkManager.service)
or
restart the device

This HAS to be a bug!

This is after i power cycle the IT closet, local network works perfectly fine but i cant ping external dns or IP

[Gertjan]

@Eddie55

Who/what is this :

Your upstream WAN ISP router ?

This is scary :

What is this enp5s0 interface ?
dhclient did received a lease, in the 192.168.0.0/24 network range. From who ?
That's the same as the LAN network ....

if enp5s0 is your WAN , you've just created Ethernet's most know issue : for a router, every interface has to have its own dedicated network, non overlapping with the others.

edit : and why use sudo ?
Are you not using the admin (root) account to access pfSense ?

[stephenw10]

That's on a client behind pfSense. 192.168.0.1 is the pfSense LAN IP. 192.168.1.254 is the upstream AT&T router.

What's telling is that when you try to ping something external like 8.8.8.8 you get network unreachable. That's not a good choice though because you are using that as a DNS server in pfSense so it will (or should) be adding a static route for it. Try pinging something that isn't defined like 8.8.4.4. If you still get 'Network Unreachable' check the routing table on the client.

Steve

[Eddie55]

@Gertjan
The terminal screen shot is showing my Desktop computer able to ping anything internal with ip and dns
HOWEVER
its showing that i cant ping anything external

Please remember its not just my desktop but all computers/laptops/roku cant get internet UNTIL

I toggle the
Allow DNS server list to be overridden by DHCP/PPP on WAN or remote OpenVPN server
Doesnt matter if its enabled or disabled
i just have to change its state and click save

THEN
i have to restart the NIC of the device

THEN
i have internet

[Eddie55]

@stephenw10
Thank you for your interest in this
I will test this tonight when i get home

And just to be clear right now my network is running perfectly fine with

Allow DNS server list to be overridden by DHCP/PPP on WAN or remote OpenVPN server > Disabled

so i do NOT have 192.168.1.254 in my DNS anymore

IF i power cycle my router again i will need to enable it and click save to get it to work

[stephenw10]

Yeah it sounds like you are actually just restarting/reloading something. Not sure it's actually DNS though since pinging by IP is also failing.

I would normally suspect pfSense it losing its default route but it would not be able to ping out itself if that were the case.

[Eddie55]

@stephenw10
yup basically the pfsense router can ping external sites with name and IP address without issue its just not letting any local client do the same
BUT the local devices all get local ip addresses and can find each other with DNS...

What is also wierd is that normally once the router gets internet all the devices under it just start working, i do find it WIERD that i need to restart the NIC of each device in order to get internet....maybe its DHCP that is giving out some bad information to get to the internet?

Maybe DHCP is handing out good enough information for local to work but not good enough for internet to work,
OR pfsense if blocking the clients
how would i test that theory

[stephenw10]

If it was blocking traffic you would see it in the firewall log, unless you had disabled default block logging or added a custom block rule that doesn't log.

The errors you're seeing on the client though look like something client side. You might try using dig against the firewall directly to bypass whatever systemd is doing. I have found it can get confused if it starts to see failures.

Steve