unbound quits resolving, have to restart

beerguzzle

Gertjan, Ok this is a lot of information to digest at the moment. I am running Kea DHCP, so I don't have the "Register DHCP leases in DNS" checkbox. I could not find any reference to it; this feature seems to be part of ISC DHCP.

I have had my Netgate for a couple of years now and my Mac mini (Intel) has always been the only thing connected to the LAN interface, wire to wire. This problem started with 24.03. I did recently buy a switch for the OPT interface, because I added more things on that network. But the unbound restart problem predated the switch.

One wrinkle is that I move around a lot in the summer, and my 1100 goes with me. It is in one of two places, using two different ISPs (I recently changed one of them). So it gets shut down via the terminal interface, packed up, and restarted at the other site. Maybe it is time for me to buy a second 1100.

Your Munin graph has prodded me to think of Munin or MRTG. I will try to pick out unbound restart times from the syslogs first. So my mission now is to look at how often unbound restarts and maybe why. Thanks.

Gertjan

@beerguzzle said in unbound quits resolving, have to restart:

Gertjan, Ok this is a lot of information to digest at the moment. I am running Kea DHCP, so I don't have the "Register DHCP leases in DNS" checkbox. I could not find any reference to it; this feature seems to be part of ISC DHCP.

KEA is fine, I guess .... It can't restart unbound, so that 's ok.

@beerguzzle said in unbound quits resolving, have to restart:

my Mac mini (Intel) has always been the only thing connected to the LAN interface, wire to wire.

Then understand that on every LAN (pfSense, MAC Mini) event == up and down events, a lot of process gets restarted. This includes unbound.
Normally, that ok-ish.
Just, I'm not a fan of such a setup, as this can creates issues that I don't have / don't know.

@beerguzzle said in unbound quits resolving, have to restart:

So it gets shut down via the terminal interface

That's the way to do it.

@beerguzzle said in unbound quits resolving, have to restart:

So my mission now is to look at how often unbound restarts and maybe why.

Exact.
You don't need Munin to see these.
Here is a one line :

cat /var/log/resolver.log | grep 'start'

which means : list the file /var/log/resolver.log and pass it through 'grep' that searches the lines which contain 'start'.

Munin : i'm not sure if I can advise it's usage, as most perl and other Munin "FreeBSD" dependent packages are not in the pfSense-FreeBSD packages server anymore.
This means you have to point the package server 'pointer' to the main "FreeBSD 15" packages servers, and that can be dangerous. I had to pull in many packages that needed other dependency packages, a small hundred of them. At any moment, a package could 'destroy' the system as core FreeBSD of pfSense is somewhat difference as a native FreeBSD system.
At any moment I was ready to 're install' from the ground up, if needed.
I like Munin as I know how it works (it's old), have I have other stuff already using munin also, and know how to add more functionality = it's always a matter of 'writing another script file in whatever language avaible'.

beerguzzle

bumping my log level up to 2 cluttered up my ability to see restarts, so I put it back to level 1. Doing a

cd /var/log
grep -i start resolver.log* (yes I speak Unix)

Just showed some of today's activity -- 2 restarts. Plus I changed locations today (pack up 1100, Mac mini, wifi router, then travel)

Per advice, I put my Mac Mini on the switch when I set things up this afternoon. So I'll see if a switch helps.

Gertjan

@beerguzzle said in unbound quits resolving, have to restart:

bumping my log level up to 2 cluttered up my ability

Not only that, it will 'explode' the size of the file.
But, as soon as the pre set maximum size is reached, it will get rotated.

It's possible to take counter measures : make the file size bigger, but that's probably not really an option on the 1100, it has a small storage.

shows my resolver.log has the details since juin 19, or something like 3 weeks.
When I set debug level 2, it will be less then a day, even with a non default, way bigger 2048 Mbytes log file size - the default is 512 Kb I guess.

HLPPC

@Gertjan have you packet captured the ISP's DHCP options and tried to make them match in the FreeBSD/pfSense files? Also, is you unbound allowing a remote control option on port 953 over localhost? I am not 100% certain of conflicts there, BUT, I know there is a remote control feature buried in there and the combination of Kea instead of ISC and whatever localhost is doing may cause issues.

Also, my ISP has had some success changing my interface local hostnames, checkable with netstat -r while using DHCP. Could be they are picky, or it could be a nonce or off-by-one c code issue.

Sometimes changing everything to attlocal.net or hostname and domain to pfsense.attlocal.net or dsldevice has worked for me if you are allowing the isp to override dns. If not maybe not bothering with dhcp is more valid or just using dnsmasq.

One decent way to set up dnsmasq is to port forward everything LAN to localhost port 54 and forward everything to the ISP's assigned DNS servers or the ISPs router, however, sometimes the ISP's router has some pretty crazy foreign vpns and crap connected to it and strict NAT options. The ISP may send ICMP error messages to you based on bad checksums.

Also, are you using IPv6? It may be a better option to disable it until you get IPv4 working.

HLPPC

@Gertjan alt text

CVE-2021-23017
6.8A security issue in nginx resolver was identified, which might allow an attacker who is able to forge UDP packets from the DNS server to cause 1-byte memory overwrite, resulting in worker process crash or potential other impact

Fixed: Update NGINX to address CVE-2021-23017 #12061

https://docs.netgate.com/pfsense/releases/22-01_2-6-0.html

https://redmine.pfsense.org/issues/12061

HLPPC

@Gertjan I'd reinstall all firmware but that is just me.

HLPPC

@Gertjan if your ISP has syslogs and you aren't scared of modern ASLR attacks you could send them to a local server and see what errors they have. Or send them directly to the pfSense or send pfSense's syslogs to the ISP router.

HLPPC

@Gertjan Do you hide netgate version identity or have any weird ISP IP passthrough or cascaded router options? Sometimes those work pretty well. A gaming console can tell you if NAT is actually working correctly. I can get either to work but it takes time.

HLPPC

@HLPPC there is also NAT pinhole-ing in some ISP routers for public IPs. https://en.wikipedia.org/wiki/Firewall_pinhole

HLPPC

@Gertjan if you are blocking bogons and local addresses on WAN, a double nat may not be possible with an ISP router.