Unbound resolver error: Can't assign requested address for 127.0.0.1

themadsalvi

@Gertjan said in Unbound resolver error: Can't assign requested address for 127.0.0.1:

unbound listens to port 953 :
[2.4.4-RELEASE][admin@pfsense.brit-hotel-fumel.net]/root: sockstat -4l | grep 'unbound'
unbound  unbound    28162 6  udp4   127.0.0.1:53          *:*
unbound  unbound    28162 7  tcp4   127.0.0.1:53          *:*
unbound  unbound    56848 6  udp4   *:53                  *:*
unbound  unbound    56848 7  tcp4   *:53                  *:*
unbound  unbound    56848 8  tcp4   127.0.0.1:953         *:*
This :

means probably that DNSBL tries to restart unbound, but it (re) started it to fast - unbound wasn't stopped (etc) and thus port '953' remains 'occupied'.
The new unbound instance can't grab it - and complains about it after stopping.

Run
[2.4.4-RELEASE][admin@pfsense.brit-hotel-fumel.net]/root: ps ax | grep 'unbound'
11558  -  Ss       0:00.22 /usr/local/sbin/unbound -c /var/unbound/unbound.conf
48881  1  S+       0:00.00 grep unbound
to see what happens on your pfSEnse.
If needed, stop unbound using the GUI, and if any zombies left, kill them.
Using the kill command and the process number.

edit : By any chance : your are not trying to overload unbound == very long startup time (by importing a huge number of DNSBL). On very small systems big lists can make unbound very slow to start, stop and just operate.

This is before I disabled the resolver

After I disabled it, the grep command came up with this
shell 2.PNG

Then I killed the remaining 79387 process. The other process came up with a "no such process". Did I do this right? This comes up when doing the grep command after restarting the unbound resolver

shell 3 after re-enable.PNG

RonpfS

@themadsalvi said in Unbound resolver error: Can't assign requested address for 127.0.0.1:

DNSBL is always out of sync,

Can you post the pfblockerng log during a Force Reload DNSBL so we can see why you get the Out of Sync errors ?

You can get those errors when you have duplicate Headers / Label in DNSBL.

How much memory do you have on that system ? 8GB can support around 1000000 DNSBL entries.

themadsalvi

@Gertjan @RonpfS said in Unbound resolver error: Can't assign requested address for 127.0.0.1:

@themadsalvi said in Unbound resolver error: Can't assign requested address for 127.0.0.1:

DNSBL is always out of sync,

Can you post the pfblockerng log during a Force Reload DNSBL so we can see why you get the Out of Sync errors ?

You can get those errors when you have duplicate Headers / Label in DNSBL.

How much memory do you have on that system ? 8GB can support around 1000000 DNSBL entries.

I currently have 4GB, which is 45% percent used according to pfsense. I can add more, but I have been running it since last year with no issues.

Below is a raw dump of the pfblockerng log in a text file(too many characters to do a full dump:

pfblockerng.txt

RonpfS

Those tables : pfB_PRI1_v4, pfB_PRI4_v4, pfB_PRI2_v4, DNSBL_pfB_PRI2_v4 - pfB_PRI2_v4, DNSBL_Abuse - pfB_Abuse_PS_v4 shouldn't be in DNSBL, they are IPv4 tables, remove them.

Disable BBC_DGA it's probably too big for your memory. And try another Force Reload DNSBL.

You have to monitor memory usage with Status Monitoring. The Dashboard only display "current" memory usage, the Monitoring will give you memory usage over time.

themadsalvi

@RonpfS

Removed those, and forced a reload, which still had the unbound resolver error.

This is the result in the status monitoring during and after reload

This is the force reload log
pfblockerng2.txt

RonpfS

You still have pfB_Abuse_PS_v4 to remove
Try again with BBC_DGA feed disabled.
If it still fails, then post your DNS Resolver config.

themadsalvi

@RonpfS @Gertjan
Here is the latest file for the reload, with all of the lists gone that you told me to delete. Same error pops up:
pfblockerng3.txt

Rsolver settings.

RonpfS

Did you try to remove the private-domain: line ?
On my box I have Prefetch Support and Prefetch DNS Key Support ticked.

themadsalvi

@RonpfS @Gertjan
I ended up taking the private domain line out(save and apply), then checking the prefetch support and Prefetch DNS Key Support boxes(save and apply changes). Tried the forced reload, with those changes, and the error persists.

RonpfS

In a shell or Diagnostics Command prompt, do a

ls -al /var/unbound /var/db/pfblockerng

themadsalvi

@RonpfS @Gertjan
I have placed the output below

Why are the last 4 so old?

RonpfS

@themadsalvi The 2012 timestamp looks suspicious compared to mine :

-rw-r-----   1 unbound  unbound       2459 Dec  8 19:42 unbound_control.key
-rw-r-----   1 unbound  unbound       1330 Dec  8 19:42 unbound_control.pem
-rw-r-----   1 unbound  unbound       2459 Dec  8 19:42 unbound_server.key
-rw-r-----   1 unbound  unbound       1318 Dec  8 19:42 unbound_server.pem

maybe it time to delete them, restart unbound or reboot pfsense.

themadsalvi

@RonpfS

what is the syntax for deleting the files in the shell?
rm -f /var/unbound/unbound_server.key?

is that the correct syntax?

Edit:
It looks like it was able to recreate the files

RonpfS

@themadsalvi

Rename them in case :

mv  /var/unbound/unbound_control.key /var/unbound/backup_unbound_control.key
mv  /var/unbound/unbound_control.pem /var/unbound/backup_unbound_control.pem
mv  /var/unbound/unbound_server.key /var/unbound/backup_unbound_server.key
mv  /var/unbound/unbound_server.pem /var/unbound/backup_unbound_server.pem

restart unbound, it should start, if not ... then move them back.
to remove them it's :

rm /var/unbound/unbound_server.pem

Also it's better to access the webgui with the pfsense IP address instead of using it's domain name when stopping and restarting DNS resolver.

themadsalvi

@RonpfS
unbound restarted ok, without any errors, but the DNSBL was still unable to reload without the error.
pfblockerng4.txt

I use the IP of Pfsense whenever I log into the web GUI, not sure why it uses the domain name when logging into shell

Grimson

What other packages are you using? Bind will conflict with unbound and if you use Service Watchdog make sure it does not monitor unbound.

RonpfS

This post is deleted!

RonpfS

Well ... I have no more clue why it doesn't reload unbound.
Maybe disable all feeds excepts Ads ?

What does ls -al /var/unbound look like now ?

themadsalvi

@RonpfS I placed the result of the rebuilt key and pem files, as well as how /var/unbound looks in my last post(out on lunch and on mobile, sorry)

@Grimson the one thing I find odd is it just started this over the weekend, after a power outage. It has been fine for the last 6 months, without any issue. I do not have bind,and have made sure that unbound is not being monitored by service watchdog. I have the regularly installed packages like pfblockerng-devel, snort, etc.

Grimson

@themadsalvi said in Unbound resolver error: Can't assign requested address for 127.0.0.1:

@Grimson the one thing I find odd is it just started this over the weekend, after a power outage.

So did you run fsck on the filesystem? https://docs.netgate.com/pfsense/en/latest/hardware/troubleshooting-disk-check-errors-fsck.html#manually-run-fsck

I have the regularly installed packages like pfblockerng-devel, snort, etc.

There are no regularly installed packages, a regular install comes without additional packages. So always mention the packages you are using when asking for help.

If following the above to check the filesystem doesn't work grab a config backup and do a fresh install to make sure the installation is in a good state.