Unbound resolver error: Can't assign requested address for 127.0.0.1

Gertjan

Hi,

Please confirm that you are using 2.4.4-p2.

I advise you to isolate the problem.
First : de activate packages like pfblocker.
In the pfSEnse GUI : goto Status => Services and stop the Resolver.
On another browser windows, visit Status > System > Logs > System > DNS Resolver
In the first browser window, start the Resolver.
In the second windows, hit Refesh (probably F5).

Show us the log (reversed order) :

Mar 26 15:46:59 	unbound 	12158:0 	info: start of service (unbound 1.8.1).
....
Mar 26 15:46:30 	unbound 	13394:0 	info: service stopped (unbound 1.8.1).

About the

The manual states :

# create certificate keys; do not recreate if they already exist.

So, delete these two files (2 x .key file / 2 x .pem file) and restart unbound.

Btw : this looks like a managed file system error. Certs are there, but can't be recreated - and contain wrong info.
It's probably time to use the console access **, and execute a couple of times fsck.

Power outage and the resulting hard shut down of pfSense ? Never saw that before.
pfSense behaves like a Windows PC these days : pull out the plug and you have a good chance it won't wake up again.
That's why UPS exist ... use them to remove a lot of possible problems.

** correction : if you do not use an UPS you should be an console access expert. That's one more reason why people prefer using an UPS ^^

themadsalvi

@Gertjan
these are the logs that I am able to get from the DNS resolver! resolver log 1.PNG dns resolver 2.PNG

I will get a better UPS, as I have a 600VA one at the moment. It was not enough for this situation

Edit: Placed the DNSBL reload error below for good measure

DNSBL error.PNG

I am in the shell, but unsure of commands to use to delete the files along with recreating them.

Gertjan

unbound listens to port 953 :

[2.4.4-RELEASE][admin@pfsense.brit-hotel-fumel.net]/root: sockstat -4l | grep 'unbound'
unbound  unbound    28162 6  udp4   127.0.0.1:53          *:*
unbound  unbound    28162 7  tcp4   127.0.0.1:53          *:*
unbound  unbound    56848 6  udp4   *:53                  *:*
unbound  unbound    56848 7  tcp4   *:53                  *:*
unbound  unbound    56848 8  tcp4   127.0.0.1:953         *:*

This :

means probably that DNSBL tries to restart unbound, but it (re) started it to fast - unbound wasn't stopped (etc) and thus port '953' remains 'occupied'.
The new unbound instance can't grab it - and complains about it after stopping.

Run

[2.4.4-RELEASE][admin@pfsense.brit-hotel-fumel.net]/root: ps ax | grep 'unbound'
11558  -  Ss       0:00.22 /usr/local/sbin/unbound -c /var/unbound/unbound.conf
48881  1  S+       0:00.00 grep unbound

to see what happens on your pfSEnse.
If needed, stop unbound using the GUI, and if any zombies left, kill them.
Using the kill command and the process number.

edit : By any chance : your are not trying to overload unbound == very long startup time (by importing a huge number of DNSBL). On very small systems big lists can make unbound very slow to start, stop and just operate.

themadsalvi

@Gertjan said in Unbound resolver error: Can't assign requested address for 127.0.0.1:

unbound listens to port 953 :
[2.4.4-RELEASE][admin@pfsense.brit-hotel-fumel.net]/root: sockstat -4l | grep 'unbound'
unbound  unbound    28162 6  udp4   127.0.0.1:53          *:*
unbound  unbound    28162 7  tcp4   127.0.0.1:53          *:*
unbound  unbound    56848 6  udp4   *:53                  *:*
unbound  unbound    56848 7  tcp4   *:53                  *:*
unbound  unbound    56848 8  tcp4   127.0.0.1:953         *:*
This :

means probably that DNSBL tries to restart unbound, but it (re) started it to fast - unbound wasn't stopped (etc) and thus port '953' remains 'occupied'.
The new unbound instance can't grab it - and complains about it after stopping.

Run
[2.4.4-RELEASE][admin@pfsense.brit-hotel-fumel.net]/root: ps ax | grep 'unbound'
11558  -  Ss       0:00.22 /usr/local/sbin/unbound -c /var/unbound/unbound.conf
48881  1  S+       0:00.00 grep unbound
to see what happens on your pfSEnse.
If needed, stop unbound using the GUI, and if any zombies left, kill them.
Using the kill command and the process number.

edit : By any chance : your are not trying to overload unbound == very long startup time (by importing a huge number of DNSBL). On very small systems big lists can make unbound very slow to start, stop and just operate.

This is before I disabled the resolver

After I disabled it, the grep command came up with this
shell 2.PNG

Then I killed the remaining 79387 process. The other process came up with a "no such process". Did I do this right? This comes up when doing the grep command after restarting the unbound resolver

shell 3 after re-enable.PNG

RonpfS

@themadsalvi said in Unbound resolver error: Can't assign requested address for 127.0.0.1:

DNSBL is always out of sync,

Can you post the pfblockerng log during a Force Reload DNSBL so we can see why you get the Out of Sync errors ?

You can get those errors when you have duplicate Headers / Label in DNSBL.

How much memory do you have on that system ? 8GB can support around 1000000 DNSBL entries.

themadsalvi

@Gertjan @RonpfS said in Unbound resolver error: Can't assign requested address for 127.0.0.1:

@themadsalvi said in Unbound resolver error: Can't assign requested address for 127.0.0.1:

DNSBL is always out of sync,

Can you post the pfblockerng log during a Force Reload DNSBL so we can see why you get the Out of Sync errors ?

You can get those errors when you have duplicate Headers / Label in DNSBL.

How much memory do you have on that system ? 8GB can support around 1000000 DNSBL entries.

I currently have 4GB, which is 45% percent used according to pfsense. I can add more, but I have been running it since last year with no issues.

Below is a raw dump of the pfblockerng log in a text file(too many characters to do a full dump:

pfblockerng.txt

RonpfS

Those tables : pfB_PRI1_v4, pfB_PRI4_v4, pfB_PRI2_v4, DNSBL_pfB_PRI2_v4 - pfB_PRI2_v4, DNSBL_Abuse - pfB_Abuse_PS_v4 shouldn't be in DNSBL, they are IPv4 tables, remove them.

Disable BBC_DGA it's probably too big for your memory. And try another Force Reload DNSBL.

You have to monitor memory usage with Status Monitoring. The Dashboard only display "current" memory usage, the Monitoring will give you memory usage over time.

themadsalvi

@RonpfS

Removed those, and forced a reload, which still had the unbound resolver error.

This is the result in the status monitoring during and after reload

This is the force reload log
pfblockerng2.txt

RonpfS

You still have pfB_Abuse_PS_v4 to remove
Try again with BBC_DGA feed disabled.
If it still fails, then post your DNS Resolver config.

themadsalvi

@RonpfS @Gertjan
Here is the latest file for the reload, with all of the lists gone that you told me to delete. Same error pops up:
pfblockerng3.txt

Rsolver settings.

RonpfS

Did you try to remove the private-domain: line ?
On my box I have Prefetch Support and Prefetch DNS Key Support ticked.

themadsalvi

@RonpfS @Gertjan
I ended up taking the private domain line out(save and apply), then checking the prefetch support and Prefetch DNS Key Support boxes(save and apply changes). Tried the forced reload, with those changes, and the error persists.

RonpfS

In a shell or Diagnostics Command prompt, do a

ls -al /var/unbound /var/db/pfblockerng

themadsalvi

@RonpfS @Gertjan
I have placed the output below

Why are the last 4 so old?

RonpfS

@themadsalvi The 2012 timestamp looks suspicious compared to mine :

-rw-r-----   1 unbound  unbound       2459 Dec  8 19:42 unbound_control.key
-rw-r-----   1 unbound  unbound       1330 Dec  8 19:42 unbound_control.pem
-rw-r-----   1 unbound  unbound       2459 Dec  8 19:42 unbound_server.key
-rw-r-----   1 unbound  unbound       1318 Dec  8 19:42 unbound_server.pem

maybe it time to delete them, restart unbound or reboot pfsense.

themadsalvi

@RonpfS

what is the syntax for deleting the files in the shell?
rm -f /var/unbound/unbound_server.key?

is that the correct syntax?

Edit:
It looks like it was able to recreate the files

RonpfS

@themadsalvi

Rename them in case :

mv  /var/unbound/unbound_control.key /var/unbound/backup_unbound_control.key
mv  /var/unbound/unbound_control.pem /var/unbound/backup_unbound_control.pem
mv  /var/unbound/unbound_server.key /var/unbound/backup_unbound_server.key
mv  /var/unbound/unbound_server.pem /var/unbound/backup_unbound_server.pem

restart unbound, it should start, if not ... then move them back.
to remove them it's :

rm /var/unbound/unbound_server.pem

Also it's better to access the webgui with the pfsense IP address instead of using it's domain name when stopping and restarting DNS resolver.

themadsalvi

@RonpfS
unbound restarted ok, without any errors, but the DNSBL was still unable to reload without the error.
pfblockerng4.txt

I use the IP of Pfsense whenever I log into the web GUI, not sure why it uses the domain name when logging into shell

Grimson

What other packages are you using? Bind will conflict with unbound and if you use Service Watchdog make sure it does not monitor unbound.

RonpfS

This post is deleted!