No startup Unbound

[Qinn]

Since some time, sorry I can not pin point it, after a reboot Unbound (DNS Resolver) isn't started. After a reboot It took a long time to get into the GUI, so I logged in using a null-modem, did a reboot using the console menu, did not see anything on the console output.

Finally, I got into the GUI and on the dashboard I saw that Unbound wasn't running, restarted it, it worked.

This happened every reboot, maybe someone can shed some light on it.

Saw this in the General logs

Feb 11 17:56:10 	php-fpm 	340 	/rc.newwanip: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '[1549904170] unbound[96196:0] error: can't bind socket: Address already in use for 127.0.0.1 port 953 [1549904170] unbound[96196:0] error: cannot open control interface 127.0.0.1 953 [1549904170] unbound[96196:0] fatal error: could not open ports'
Feb 11 17:55:58 	login 		login on ttyu0 as root 

Cheers Qinn

[johnpoz]

@qinn said in No startup Unbound:

Address already in use for 127.0.0.1 port 953

My take is you have a issue with bind also running they like to use the same control port 953. You can change the control port in the bind gui!

[Qinn]

Thanks for the reply John, but bind is not installed, it there anyway to find out why during booting pfSense Unbound won't come up?

[johnpoz]

Well look to see what else is using 953 then.

Do a simple sockstat when unbound is not running

[2.4.4-RELEASE][root@sg4860.local.lan]/: sockstat | grep :953
unbound  unbound    49220 27 tcp4   127.0.0.1:953         *:*
[2.4.4-RELEASE][root@sg4860.local.lan]/: 

You can see that unbound is using 953, your error is saying it can not use that port.

[Qinn]

Good idea

With Unbound running, result:

[2.4.4-RELEASE][root@pfSense.localdomain]/root: sockstat | grep :953
unbound  unbound    63945 22 tcp4   127.0.0.1:953         *:*
?        ?          ?     ?  tcp4   127.0.0.1:953         127.0.0.1:57064
?        ?          ?     ?  tcp4   127.0.0.1:953         127.0.0.1:1987
?        ?          ?     ?  tcp4   127.0.0.1:953         127.0.0.1:33822
?        ?          ?     ?  tcp4   127.0.0.1:953         127.0.0.1:33089
?        ?          ?     ?  tcp4   127.0.0.1:953         127.0.0.1:64753
?        ?          ?     ?  tcp4   127.0.0.1:953         127.0.0.1:18454
?        ?          ?     ?  tcp4   127.0.0.1:953         127.0.0.1:43445

Then a reboot from the console (option 5) below the result in the GUI:

Then:

2.4.4-RELEASE][root@pfSense.localdomain]/root: sockstat | grep :953
[2.4.4-RELEASE][root@pfSense.localdomain]/root:

...but nothing running on 953?

In the General log

General log
Feb 12 16:59:00 	sshd 	17591 	Accepted keyboard-interactive/pam for root from 192.168.1.100 port 14375 ssh2
Feb 12 16:58:59 	php-fpm 	340 	/rc.newwanip: Creating rrd update script
Feb 12 16:58:59 	php-fpm 	340 	/rc.newwanip: Resyncing OpenVPN instances for interface WAN.
Feb 12 16:58:57 	php-fpm 	340 	/rc.newwanip: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '[1549987137] unbound[17312:0] error: can't bind socket: Address already in use for 127.0.0.1 port 953 [1549987137] unbound[17312:0] error: cannot open control interface 127.0.0.1 953 [1549987137] unbound[17312:0] fatal error: could not open ports'
Feb 12 16:58:57 	sshd 	17591 	user root login class [preauth] 

In the DNS resolver log

Feb 12 16:58:16 	unbound 	72813:0 	debug: duplicate acl address ignored.
Feb 12 16:58:16 	unbound 	72813:0 	debug: drop user privileges, run as unbound
Feb 12 16:58:16 	unbound 	72813:0 	debug: chroot to /var/unbound
Feb 12 16:58:16 	unbound 	72813:0 	debug: chdir to /var/unbound
Feb 12 16:56:55 	unbound 	63945:0 	info: service stopped (unbound 1.8.1).
Feb 12 16:56:47 	unbound 	63945:0 	info: control cmd: stats_noreset
Feb 12 16:56:46 	unbound 	63945:0 	debug: new control connection from 127.0.0.1 port 35665
Feb 12 16:56:46 	unbound 	63945:0 	debug: cache memory msg=1903288 rrset=3843972 infra=2490483 val=635129 

There is something else maybe worth mentioning, it's not every time that Unbound won't start after a reboot, but it's a 50-50 with 3 other services

avahi
pfb_dnsbl
pfb_filter

So the one time Unbound is up and those 3 are down and the other time, it's the other way around?

[johnpoz]

My guess would be something with your pfblocker.. It restarts unbound when it updates - its possible that unbound is not actually stopping and left running..

What is this?

?        ?          ?     ?  tcp4   127.0.0.1:953         127.0.0.1:57064
?        ?          ?     ?  tcp4   127.0.0.1:953         127.0.0.1:1987
?        ?          ?     ?  tcp4   127.0.0.1:953         127.0.0.1:33822
?        ?          ?     ?  tcp4   127.0.0.1:953         127.0.0.1:33089

You can not see the pid of what is bound to 953?

[RonpfS]

When the widget send an unbound-control stats_noreset during an unbound reload, unbound may hang and may need to be kill -9 .

Change the pfblockerNG Widget DNSBL Resolver Query frequency from 5 sec to 60 or 120 sec.

You can also try to enable DNSBL Live Reload Resync so pfblockerNG wont reload unbound.

[Qinn]

[2.4.4-RELEASE][root@pfSense.localdomain]/root: sockstat | grep :953
unbound  unbound    63945 22 tcp4   127.0.0.1:953         *:*
?        ?          ?     ?  tcp4   127.0.0.1:953         127.0.0.1:57064

@johnpoz Your guess is as good as mine , only the first line is unbound?

[Qinn]

@ronpfs said in No startup Unbound:

DNSBL Resolver Query frequency

Thanks RonpfS for stepping in, I can't seem to locate the;

"Change the pfblockerNG Widget DNSBL Resolver Query frequency from 5 sec to 60 or 120 sec."

You've mentioned, of course I will try to enable DNSBL Live Reload Resync and report back.

[RonpfS]

@qinn said in No startup Unbound:

Thanks RonpfS for stepping in, I can't seem to locate the;
"Change the pfblockerNG Widget DNSBL Resolver Query frequency from 5 sec to 60 or 120 sec."

Click on the Wrench Icon on the pfBlockerNG widget

[Qinn]

@ronpfs said in No startup Unbound:

@qinn said in No startup Unbound:

Thanks RonpfS for stepping in, I can't seem to locate the;
"Change the pfblockerNG Widget DNSBL Resolver Query frequency from 5 sec to 60 or 120 sec."

Click on the Wrench Icon on the pfBlockerNG widget

Thanks, duhhhh, missed out that one, stupid me. I will try that first and if that don't work the
enable DNSBL Live Reload Resync, but it has to wait for tomorrow as some rsync's are running and after that backups are kickin in, so tomorrow I will try and report back!

Thanks again

[Qinn]

@RonpfS setting DNSBL Resolver Query frequency to 60 sec, didn't do much. Next tried the *DNSBL Live Reload Resync, but unfortunately it did not do much either. Still unbound is not running after a reboot...

Anyone any idea's?

Cheers Qinn

[Gertjan]

@qinn said in No startup Unbound:

Anyone any idea's?

Yep.
If you can works a day or two without DNSBL , disable it all together.
Reboot pfSense.
Check.

Doing so will rule out any problems related to DNSBL - or not ....

@qinn said in No startup Unbound:

Still unbound is not running after a reboot...

That is, it should start after a boot. The logs do shows this.
But shortly after that it gets probably restarted, so it stops, and then can't start again.

[Qinn]

@gertjan I stopped and disabled pfb_dnsbl and pfb_filter and rebooted 3 times, on all of them unbound was running and there were no errors in the General Log concerning Unbound and accessing the GUI/Dashboard after reboot now took a few seconds, instead of minutes. It seems very likely that the culprit has found, pfb_dnsbl and pfb_filter and Unbound don't play well , but now to a solution. I hope that @BBcan177 can shed some light on it.

[johnpoz]

@qinn said in No startup Unbound:

pfb_dnsbl and pfb_filter and Unbound don't play well

I would concur on this ;) no offense to BBcan177 great work on the package.. But it has become lets call it very complex ;)

[Gertjan]

@johnpoz said in No startup Unbound:

@qinn said in No startup Unbound:

pfb_dnsbl and pfb_filter and Unbound don't play well

I would concur on this ;) no offense to BBcan177 great work on the package.. But it has become lets call it very complex ;)

@metoo

But the fact is :
pfb_dnsbl import big or can I call them huge - lists.
And when he set them up, it kicks unbound, who starts to parse all these lists using. This needs a lot resources.
I do think a 'correct' setup of pfb_dnsbl would be a good solution.

[johnpoz]

@gertjan said in No startup Unbound:

pfb_dnsbl import big or can I call them huge - lists.

Yup would concur there as well ;)