DNS Resover stops running - Could not deliver signal HUP
-
Hi.
I'm running PC Engines APU2 / BIOS Version: v4.12.0.3 / Version 2.4.5-RELEASE-p1 (amd64)
PfBlockerNG V3.x.x.x running.
Got sometimes DNS resolver service stopping by itself.
System log show :Jan 12 17:22:31 pfSense dhcpleases: Could not deliver signal HUP to process because its pidfile (/var/run/unbound.pid) does not exist, No such process. Jan 12 17:22:31 pfSense dhcpleases: Could not deliver signal HUP to process because its pidfile (/var/run/unbound.pid) does not exist, No such process.
Go it again today just after pfBlockerNG update from v3.0.0.7 to 3.0.0.8
Jan 12 17:20:38 pfSense php: /etc/rc.packages: Successfully installed package: pfBlockerNG-devel. Jan 12 17:20:38 pfSense pkg-static: pfSense-pkg-pfBlockerNG-devel upgraded: 3.0.0_7 -> 3.0.0_8 Jan 12 17:20:40 pfSense check_reload_status: Reloading filter Jan 12 17:20:40 pfSense check_reload_status: Starting packages Jan 12 17:20:41 pfSense php-fpm[55341]: /rc.start_packages: Restarting/Starting all packages. Jan 12 17:20:41 pfSense bandwidthd: Monitoring subnet 192.168.20.0 with netmask 255.255.255.0 Jan 12 17:20:41 pfSense bandwidthd: Monitoring subnet 192.168.20.0 with netmask 255.255.255.0 Jan 12 17:20:41 pfSense bandwidthd: Opening igb2 Jan 12 17:20:41 pfSense bandwidthd: Opening igb2 Jan 12 17:20:41 pfSense bandwidthd: Opening igb2 Jan 12 17:20:41 pfSense bandwidthd: Opening igb2 Jan 12 17:20:41 pfSense bandwidthd: Packet Encoding: Ethernet Jan 12 17:20:41 pfSense bandwidthd: Packet Encoding: Ethernet Jan 12 17:20:41 pfSense bandwidthd: Opening igb2 Jan 12 17:20:41 pfSense bandwidthd: Opening igb2 Jan 12 17:20:41 pfSense bandwidthd: Packet Encoding: Ethernet Jan 12 17:20:41 pfSense bandwidthd: Packet Encoding: Ethernet Jan 12 17:20:41 pfSense bandwidthd: Opening igb2 Jan 12 17:20:41 pfSense bandwidthd: Packet Encoding: Ethernet Jan 12 17:20:41 pfSense bandwidthd: Opening igb2 Jan 12 17:20:41 pfSense bandwidthd: Packet Encoding: Ethernet Jan 12 17:20:41 pfSense bandwidthd: Packet Encoding: Ethernet Jan 12 17:20:41 pfSense bandwidthd: Packet Encoding: Ethernet Jan 12 17:20:43 pfSense lighttpd_pfb: [pfBlockerNG] DNSBL Webserver started Jan 12 17:20:44 pfSense php: [pfBlockerNG] DNSBL parser daemon started Jan 12 17:21:01 pfSense php-fpm[55341]: [pfBlockerNG] Starting firewall filter daemon Jan 12 17:21:01 pfSense lighttpd_pfb: [pfBlockerNG] DNSBL Webserver started Jan 12 17:21:03 pfSense php_pfb: [pfBlockerNG] filterlog daemon started Jan 12 17:21:03 pfSense php: [pfBlockerNG] DNSBL parser daemon started Jan 12 17:22:31 pfSense dhcpleases: Could not deliver signal HUP to process because its pidfile (/var/run/unbound.pid) does not exist, No such process. Jan 12 17:22:31 pfSense dhcpleases: Could not deliver signal HUP to process because its pidfile (/var/run/unbound.pid) does not exist, No such process.
Here are some logs from resolver.log
Jan 12 17:19:29 pfSense unbound: [40152:0] notice: init module 0: validator Jan 12 17:19:29 pfSense unbound: [40152:0] notice: init module 1: iterator Jan 12 17:19:30 pfSense unbound: [40152:0] info: start of service (unbound 1.10.1). Jan 12 17:19:30 pfSense unbound: [40152:3] info: generate keytag query _ta-4f66. NULL IN Jan 12 17:20:20 pfSense unbound: [26825:0] warning: unbound is already running as pid 40152. Jan 12 17:20:25 pfSense unbound: [27501:0] notice: init module 0: validator Jan 12 17:20:25 pfSense unbound: [27501:0] notice: init module 1: iterator Jan 12 17:20:25 pfSense unbound: [27501:0] info: start of service (unbound 1.10.1). Jan 12 17:20:26 pfSense unbound: [27501:2] info: generate keytag query _ta-4f66. NULL IN Jan 12 17:23:30 pfSense unbound: [39013:0] notice: init module 0: validator Jan 12 17:23:30 pfSense unbound: [39013:0] notice: init module 1: iterator Jan 12 17:23:30 pfSense unbound: [39013:0] info: start of service (unbound 1.10.1). Jan 12 17:23:30 pfSense unbound: [39013:0] info: service stopped (unbound 1.10.1).
Here are some logs from dhcpd.log
Jan 12 17:22:31 pfSense dhcpd: DHCPACK on 192.168.20.73 to 1c:65:9d:5b:48:c5 (Vostro001) via igb2 Jan 12 17:22:31 pfSense dhcpleases: Sending HUP signal to dns daemon(27501) Jan 12 17:22:31 pfSense dhcpleases: Could not deliver signal HUP to process because its pidfile (/var/run/unbound.pid) does not exist, No such process. Jan 12 17:22:31 pfSense dhcpleases: Sending HUP signal to dns daemon(27501) Jan 12 17:22:31 pfSense dhcpleases: Could not deliver signal HUP to process because its pidfile (/var/run/unbound.pid) does not exist, No such process. Jan 12 17:23:25 pfSense dhcpleases: /etc/hosts changed size from original! Jan 12 17:23:25 pfSense dhcpleases: Sending HUP signal to dns daemon(27501) Jan 12 17:23:25 pfSense dhcpleases: Could not deliver signal HUP to process because its pidfile (/var/run/unbound.pid) does not exist, No such process. Jan 12 17:23:25 pfSense dhcpleases: kqueue error: unknown Jan 12 17:23:25 pfSense dhcpleases: Sending HUP signal to dns daemon(39013) Jan 12 17:23:27 pfSense dhcpd: Internet Systems Consortium DHCP Server 4.4.1 Jan 12 17:23:27 pfSense dhcpd: Copyright 2004-2018 Internet Systems Consortium. Jan 12 17:23:27 pfSense dhcpd: All rights reserved.
This is very annoying.
Any help would be appreciated.Regards.
-
Specifically re: pfBlocker upgrade, per https://forum.netgate.com/topic/159094/pfblockerng-v3-0-0_6-update/4 there's a problem where unbound is stopped during package upgrade and doesn't restart. Just start it manually.
-
@teamits Thanks... Clear for me.
-
Several things to know / to check :
@autourdupc said in DNS Resover stops running - Could not deliver signal HUP:
pfSense dhcpleases
If you want to use the new Python mode, you have to stop the "dhcpleases" process anyway:
Goto Services > DNS Resolver > General Settings : uncheck DHCP Registration, validate and restart Unbound. Issue solved.Btw : even if you do not use the Python (excellent !) mode, I advise you to stop "dhcpleases" as it's implementation has limited - see also here - , like having unbound getting railgunned. ..... and that's probably your issue, because :
@autourdupc said in DNS Resover stops running - Could not deliver signal HUP:
Could not deliver signal HUP to process because its pidfile .....
That's bad.
So, it's "there" (the pid file exists with a pid) but it isn't there any more. Its doing a full time job of restarting (I'll bet ...). Another instance us running and it doesn't even had the time to init a pid file for it.
Check your unbound log, as it answers most interrogations and always tells the truth.
Or use the 'nerdy' way :clog /var/log/resolver.log | grep 'start'
and count the number of times unbound restarts per hour ? day ?
You'll be looking the intelligent way at the number of these lines :
@autourdupc said in DNS Resover stops running - Could not deliver signal HUP:
Jan 12 17:20:25 pfSense unbound: [27501:0] info: start of service (unbound 1.10.1).
If it's more then ones or twice a day : kill dhcpleases as stated above and your done again.
Another one : your resolver/unbound log shows something interesting :
@autourdupc said in DNS Resover stops running - Could not deliver signal HUP:
Jan 12 17:20:20 pfSense unbound: [26825:0] warning: unbound is already running as pid 40152.
which totally explains :
@autourdupc said in DNS Resover stops running - Could not deliver signal HUP:
Jan 12 17:22:31 pfSense dhcpleases: Could not deliver signal HUP to process because its pidfile (/var/run/unbound.pid) does not exist, No such process.
There is also this part, in the middle of your log fragment :
@autourdupc said in DNS Resover stops running - Could not deliver signal HUP:
Jan 12 17:20:26 pfSense unbound: [27501:2] info: generate keytag query _ta-4f66. NULL IN
Jan 12 17:23:30 pfSense unbound: [39013:0] notice: init module 0: validatorUnbound i"generate keytag " (whatever that means ^^)
And then, 3 ( 180 seconds !! ) later it continues with " init module 0: validator".This can mean several things :
You have a huge resource eater in your system. And you have :@autourdupc said in DNS Resover stops running - Could not deliver signal HUP:
Jan 12 17:20:41 pfSense bandwidthd: Packet Encoding: Ethernet
Nice package, bandwidth, but can an APU2 push this thing around ?
I can't tell if bandwidth, as I do not use these type of biscuit box size devices to run my pfSense. I tend to have it run on pure Intel stuff. An stay away from bandwidth like tools as it burns CPU cycles as snow in the sun.Also : running pfBlockerNG in 'old' unbound mode (== not python mode) will impact every restart of unbound. Normally, on a clean (no packages) pfSense setup, unbound starts up within a second.
Yours takes more then 3 minutes for sure.... that is bad.
You lso know now (one of the reasons) why the pfBlockerNG Python mode has been invented. Loading pfBlockerNG full with DNSBL feeds can even bring a serious powerful system to a nearly halt if pfB is in the unbound mode and unbound restarts.I'm not sure, but :
Jan 12 17:20:26 pfSense unbound: [27501:2] info: generate keytag query _ta-4f66. NULL IN
make me think it is contacting some server in the wild, to sync up with the root KSK for DNNSEC. ( See here - read the signed paper ) and that takes time ..... is your WAN connection that is bad ? at that moment ? Is your system crawling - at that moment ?
Anyway, these are just my observations, things I would check if I had my hand on the system.
-
@gertjan said in DNS Resover stops running - Could not deliver signal HUP:
If you want to use the new Python mode
Why not ?
What are the benefits to use Python mode ?In Python Module Order, what choice is the best ?
Pre validator ?
or
Post validator ?What is the difference ?
Laurent.
-
@autourdupc said in DNS Resover stops running - Could not deliver signal HUP:
Why not ?
What are the benefits to use Python mode ?unbound versus python mode :
To make a long story short : (python is) faster to handle DNS traffic, faster to (re)load, more control over the DNS filtering, better logging.Pre validator ?
or
Post validator ?See here. look up on that page "modue-config".
Leaving it to the default = pre-validating means that DNSSEC is activated.
It means that every DNS request that has to be resolved, a DNSSEC test is done first. After that, the DNS request is made. -
@gertjan said in DNS Resover stops running - Could not deliver signal HUP:
Leaving it to the default = pre-validating means that DNSSEC is activated.
DNSSEC is different from pre-validator.
DNSSEC need to be checked... Not the same setting, Right ?Regards,
-
@autourdupc said in DNS Resover stops running - Could not deliver signal HUP:
DNSSEC is different from pre-validator.
DNSSEC need to be checked... Not the same setting, Right ?With Pre validator : unbound.conf :
module-config: "validator python iterator"
With Pre validator : unbound.conf :
module-config: "python validator iterator"With "Enable DNSSEC Support" activated, the presence of "validator " is controlled.
So ..... yes, you're rightThe idea is to put the module named 'pyhton' up front, so its the first one to handle DNS requests.
Choosing among Pre or Post makes sense now.
If the "validator " (== DNSSEC) bails out becuase of validator errors, the DNS request would not make it up to pfBlockerNG, so it can at least log the DNS activity.The last, the iterator, is unbound itself as a resolver.
Btw : the name 'python' here is the pfBlockerNG integration.
At the bottom of unbound.conf you see :
# Python Module python: python-script: pfb_unbound.py
-
@gertjan
I did the settings... All is fine.
pfsense still running.Concerning BandwithD, it works also fine on my APU...
pfsense usage is for personal use, with less than 20 devices on my Network.As you can see, CPU usage is low... (9%..30% depending on what I do).
Only temp. is a little bit elevated due to the fact there is no cooling system.I also implemented zfs for better performances and reliability.