unbound DNS Resolver Will Not Start

bmeeks

@newUser2pfSense said in unbound DNS Resolver Will Not Start:

When I restart pfSense and watch the scrolling console screen, I see the following line:
Starting DNS Resolver...done.

The DNS resolver must be stopped just after this line gets displayed in the console.

As a test, I just disabled pfBlockerNG completely and restarted from the console. When I logged into the GUI, I see the same unbound DNS Resolver is stopped and again and it won't start. Is it safe to say it's not pfBlockerNG?

No, not safe to say that. pfBlockerNG may have left Unbound with a corrupt or incorrectly configured conf file. You need to look in the logs for the system and resolver (unbound) to see that it is reporting (if anything).

newUser2pfSense

I'm not sure if this helps or not but here are the last log file lines in the System Logs > System > DNS Resolver after updating and it repeats about 5 times:

Jun 10 09:15:18 unbound 22966:1 info: 127.0.0.1 _https._tcp.pkg.pfsense.org. SRV IN NOERROR 0.000000 1 123
Jun 10 09:15:18 unbound 22966:1 info: 127.0.0.1 files00.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:6 info: 127.0.0.1 files00.netgate.com. AAAA IN NOERROR 0.000000 1 65
Jun 10 09:15:18 unbound 22966:3 info: 127.0.0.1 files01.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:3 info: 127.0.0.1 files01.netgate.com. AAAA IN NOERROR 0.000000 1 65
Jun 10 09:15:18 unbound 22966:6 info: 127.0.0.1 files00.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:6 info: 127.0.0.1 files00.netgate.com. AAAA IN NOERROR 0.000000 1 65
Jun 10 09:15:18 unbound 22966:6 info: 127.0.0.1 files01.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:1 info: 127.0.0.1 files01.netgate.com. AAAA IN NOERROR 0.000000 1 65
Jun 10 09:15:18 unbound 22966:2 info: 127.0.0.1 files00.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:6 info: 127.0.0.1 files00.netgate.com. AAAA IN NOERROR 0.000000 1 65
Jun 10 09:15:18 unbound 22966:7 info: 127.0.0.1 files01.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:7 info: 127.0.0.1 files01.netgate.com. AAAA IN NOERROR 0.000000 1 65

Here's an interesting line in the System Logs > System > General

Jun 10 12:35:05 php-fpm 88498 /status_services.php: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '/var/unbound/unbound.conf:105: error: syntax error read /var/unbound/unbound.conf failed: 1 errors in configuration file [1591806905] unbound[31342:0] fatal error: Could not read config file: /var/unbound/unbound.conf. Maybe try unbound -dd, it stays on the commandline to see more errors, or unbound-checkconf'

When I enter "unbound-checkconf" (no quotes) in the Diagnostics > Command Prompt, I get the following:

unbound-checkconf: no errors in /usr/local/etc/unbound/unbound.conf

pete35

You can try to restore the pfsense configuration from your backup.

If there is none, just try to reconfigure the resolver and save it again, hopeing that it will repair the incorrect configuration file.

You can also look into the configuration file around the mentioned line to check if there is a missconfiguration or syntax error and try to correct it. You need to login on console to do that.

newUser2pfSense

So I made a mistake and didn't backup my configuration before updating. Wow, big mistake. That won't happen again! The backups that I do have are just over 3 months old and I'm not sure if they have my complete configuration or not.

How would I go about reconfiguring the resolver and then save it again? Any ideas?

bmeeks

@newUser2pfSense said in unbound DNS Resolver Will Not Start:

I'm not sure if this helps or not but here are the last log file lines in the System Logs > System > DNS Resolver after updating and it repeats about 5 times:

Jun 10 09:15:18 unbound 22966:1 info: 127.0.0.1 _https._tcp.pkg.pfsense.org. SRV IN NOERROR 0.000000 1 123
Jun 10 09:15:18 unbound 22966:1 info: 127.0.0.1 files00.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:6 info: 127.0.0.1 files00.netgate.com. AAAA IN NOERROR 0.000000 1 65
Jun 10 09:15:18 unbound 22966:3 info: 127.0.0.1 files01.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:3 info: 127.0.0.1 files01.netgate.com. AAAA IN NOERROR 0.000000 1 65
Jun 10 09:15:18 unbound 22966:6 info: 127.0.0.1 files00.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:6 info: 127.0.0.1 files00.netgate.com. AAAA IN NOERROR 0.000000 1 65
Jun 10 09:15:18 unbound 22966:6 info: 127.0.0.1 files01.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:1 info: 127.0.0.1 files01.netgate.com. AAAA IN NOERROR 0.000000 1 65
Jun 10 09:15:18 unbound 22966:2 info: 127.0.0.1 files00.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:6 info: 127.0.0.1 files00.netgate.com. AAAA IN NOERROR 0.000000 1 65
Jun 10 09:15:18 unbound 22966:7 info: 127.0.0.1 files01.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:7 info: 127.0.0.1 files01.netgate.com. AAAA IN NOERROR 0.000000 1 65

Here's an interesting line in the System Logs > System > General

Jun 10 12:35:05 php-fpm 88498 /status_services.php: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '/var/unbound/unbound.conf:105: error: syntax error read /var/unbound/unbound.conf failed: 1 errors in configuration file [1591806905] unbound[31342:0] fatal error: Could not read config file: /var/unbound/unbound.conf. Maybe try unbound -dd, it stays on the commandline to see more errors, or unbound-checkconf'

When I enter "unbound-checkconf" (no quotes) in the Diagnostics > Command Prompt, I get the following:

unbound-checkconf: no errors in /usr/local/etc/unbound/unbound.conf

You ran the "checkconf" on the wrong unbound.conf file. Look again at your error message. It does not like the unbound.conf file in '/var/unbound. So you will need to run the unbound-checkconf command against /var/unbound/unbound.conf to find out what's actually wrong.

See, here is the actual error message:

fatal error: Could not read config file: /var/unbound/unbound.conf.

Because you gave the unbound-checkconf utility no parameters, it checked the default file here:

no errors in /usr/local/etc/unbound/unbound.conf

but that is NOT the file that unbound uses when it actually runs. It runs from config files in /var/unbound.

pete35

Just change any number on the "advanced resolver options" under "advanced settings" save it and change it back. Save it again. Try to start unbound then and check the logs again.

newUser2pfSense

So I changed a value in the "Advanced Resolver Options", saved it, changed it back, saved it again.

In the Status > System Logs > System > DNS Resolver:

Nothing changed...no new lines at all.

In the Status > System Logs > System > General, these are the new lines:

Jun 10 13:30:24 php-fpm 357 /services_unbound_advanced.php: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '/var/unbound/unbound.conf:105: error: syntax error read /var/unbound/unbound.conf failed: 1 errors in configuration file [1591810224] unbound[36841:0] fatal error: Could not read config file: /var/unbound/unbound.conf. Maybe try unbound -dd, it stays on the commandline to see more errors, or unbound-checkconf'
Jun 10 13:30:51 check_reload_status Syncing firewall
Jun 10 13:30:54 php-fpm 356 /services_unbound_advanced.php: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '/var/unbound/unbound.conf:105: error: syntax error read /var/unbound/unbound.conf failed: 1 errors in configuration file [1591810254] unbound[69110:0] fatal error: Could not read config file: /var/unbound/unbound.conf. Maybe try unbound -dd, it stays on the commandline to see more errors, or unbound-checkconf'
Jun 10 13:31:02 php-fpm 356 /status_services.php: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '/var/unbound/unbound.conf:105: error: syntax error read /var/unbound/unbound.conf failed: 1 errors in configuration file [1591810262] unbound[82148:0] fatal error: Could not read config file: /var/unbound/unbound.conf. Maybe try unbound -dd, it stays on the commandline to see more errors, or unbound-checkconf'

In the Diagnostics > Command prompt, I ran the following command:

unbound-checkconf /var/unbound/unbound.conf

The result:
/var/unbound/unbound.conf:105: error: syntax error
read /var/unbound/unbound.conf failed: 1 errors in configuration file

pete35

Maybe post your unbound.conf here?

newUser2pfSense

From: /var/unbound/unbound.conf

##########################

Unbound Configuration

##########################

Server configuration

server:

chroot: /var/unbound
username: "unbound"
directory: "/var/unbound"
pidfile: "/var/run/unbound.pid"
use-syslog: yes
port: 53
verbosity: 2
hide-identity: yes
hide-version: yes
harden-glue: yes
do-ip4: yes
do-ip6: no
do-udp: yes
do-tcp: yes
do-daemonize: yes
module-config: "validator iterator"
unwanted-reply-threshold: 0
num-queries-per-thread: 512
jostle-timeout: 200
infra-host-ttl: 900
infra-cache-numhosts: 10000
outgoing-num-tcp: 10
incoming-num-tcp: 10
edns-buffer-size: 4096
cache-max-ttl: 86400
cache-min-ttl: 0
harden-dnssec-stripped: yes
msg-cache-size: 4m
rrset-cache-size: 8m

num-threads: 8
msg-cache-slabs: 8
rrset-cache-slabs: 8
infra-cache-slabs: 8
key-cache-slabs: 8
outgoing-range: 4096
#so-rcvbuf: 4m
auto-trust-anchor-file: /var/unbound/root.key
prefetch: no
prefetch-key: no
use-caps-for-id: no
serve-expired: no

Statistics

Unbound Statistics

statistics-interval: 0
extended-statistics: yes
statistics-cumulative: yes

TLS Configuration

tls-cert-bundle: "/etc/ssl/cert.pem"

Interface IP(s) to bind to

interface-automatic: yes
interface: 0.0.0.0
interface: ::0

Outgoing interfaces to be used

DNS Rebinding

For DNS Rebinding prevention

private-address: 127.0.0.0/8
private-address: 10.0.0.0/8
private-address: ::ffff:a00:0/104
private-address: 172.16.0.0/12
private-address: ::ffff:ac10:0/108
private-address: 169.254.0.0/16
private-address: ::ffff:a9fe:0/112
private-address: 192.168.0.0/16
private-address: ::ffff:c0a8:0/112
private-address: fd00::/8
private-address: fe80::/10

Access lists

include: /var/unbound/access_lists.conf

Static host entries

include: /var/unbound/host_entries.conf

dhcp lease entries

include: /var/unbound/dhcpleases_entries.conf

Domain overrides

include: /var/unbound/domainoverrides.conf

Forwarding

forward-zone:
name: "."
forward-tls-upstream: yes
forward-addr: 9.9.9.9@853
forward-addr: 149.112.112.112@853

Unbound custom options

log-replies: yes

Remote Control Config

include: /var/unbound/remotecontrol.conf

bmeeks

@newUser2pfSense said in unbound DNS Resolver Will Not Start:

From: /var/unbound/unbound.conf

##########################

Unbound Configuration

##########################

Server configuration

server:

chroot: /var/unbound
username: "unbound"
directory: "/var/unbound"
pidfile: "/var/run/unbound.pid"
use-syslog: yes
port: 53
verbosity: 2
hide-identity: yes
hide-version: yes
harden-glue: yes
do-ip4: yes
do-ip6: no
do-udp: yes
do-tcp: yes
do-daemonize: yes
module-config: "validator iterator"
unwanted-reply-threshold: 0
num-queries-per-thread: 512
jostle-timeout: 200
infra-host-ttl: 900
infra-cache-numhosts: 10000
outgoing-num-tcp: 10
incoming-num-tcp: 10
edns-buffer-size: 4096
cache-max-ttl: 86400
cache-min-ttl: 0
harden-dnssec-stripped: yes
msg-cache-size: 4m
rrset-cache-size: 8m

num-threads: 8
msg-cache-slabs: 8
rrset-cache-slabs: 8
infra-cache-slabs: 8
key-cache-slabs: 8
outgoing-range: 4096
#so-rcvbuf: 4m
auto-trust-anchor-file: /var/unbound/root.key
prefetch: no
prefetch-key: no
use-caps-for-id: no
serve-expired: no

Statistics

Unbound Statistics

statistics-interval: 0
extended-statistics: yes
statistics-cumulative: yes

TLS Configuration

tls-cert-bundle: "/etc/ssl/cert.pem"

Interface IP(s) to bind to

interface-automatic: yes
interface: 0.0.0.0
interface: ::0

Outgoing interfaces to be used

DNS Rebinding

For DNS Rebinding prevention

private-address: 127.0.0.0/8
private-address: 10.0.0.0/8
private-address: ::ffff:a00:0/104
private-address: 172.16.0.0/12
private-address: ::ffff:ac10:0/108
private-address: 169.254.0.0/16
private-address: ::ffff:a9fe:0/112
private-address: 192.168.0.0/16
private-address: ::ffff:c0a8:0/112
private-address: fd00::/8
private-address: fe80::/10

Access lists

include: /var/unbound/access_lists.conf

Static host entries

include: /var/unbound/host_entries.conf

dhcp lease entries

include: /var/unbound/dhcpleases_entries.conf

Domain overrides

include: /var/unbound/domainoverrides.conf

Forwarding

forward-zone:
name: "."
forward-tls-upstream: yes
forward-addr: 9.9.9.9@853
forward-addr: 149.112.112.112@853

Unbound custom options

log-replies: yes

Remote Control Config

include: /var/unbound/remotecontrol.conf

You need to open this file in an editor and find line #105. That's where the error is. Look again at the error message you are given from the logs:

the output was '/var/unbound/unbound.conf:105: error: syntax error

The ":105" part is the line number where the syntax error is located.

newUser2pfSense

I copied the entire contents and pasted it into a text file in an editor and I believe line 105 is:

log-replies: yes

pete35

Go to the custom options in the unbound gui and remove this line. Save and restart unbound.

bmeeks

@newUser2pfSense said in unbound DNS Resolver Will Not Start:

I copied the entire contents and pasted it into a text file in an editor and I believe line 105 is:

log-replies: yes

I don't see anything wrong on that line, but there may be extra control or other characters present that either did not get copied into your post or show up as whitespace and thus are invisible. Clearly there is something on line 105 that unbound does not care for. Could be something just in front of, or just behind, that line number.

bmeeks

User @pete35 is on the right track with his suggestions. You need to wipe out all of the custom conf stuff that pfBlockerNG would have added to unbound's conf file.

newUser2pfSense

So I went to Services > DNS Resolver > General settings and deleted the log-replies: yes.

As soon as I went to the dashboard, the Services Status > unbound DNS Resolver is now started.

bmeeks

@newUser2pfSense said in unbound DNS Resolver Will Not Start:

So I went to Services > DNS Resolver > General settings and deleted the log-replies: yes.

As soon as I went to the dashboard, the Services Status > unbound DNS Resolver is now started.

Great! There must have been something else lurking on that line because according to the man page I found on Google that option is a vaild one.

pete35

So insert that line again, but dont copy it from anywhere, just type it in. save and Restart, if that is ok then, you can enable pfblocker.

newUser2pfSense

Ok, so I tried this two times and no luck -
I went to Diagnostics > Edit File and browsed to the file and typed the line back in and saved the file. As a test to make sure the line stayed in the file, I restarted pfSense. When logging back into the GUI, the unbound DNS Resolver is working, however, when I check the file, the line is not there.

Going to Services > DNS Resolver > General Settings > Custom options, I tried typing the line in there and saving and I eceived the following error message:

The following input errors were detected:

The generated config file cannot be parsed by unbound. Please correct the following errors:
/var/unbound/test/unbound.conf:105: error: syntax error
read /var/unbound/test/unbound.conf failed: 1 errors in configuration file

Hmm, /test/ is in this path. When I look in the file with the /test/ in the path, the line is there.

pete35

Hmm, i dont think you need that option. If it is ok for you, just forget about it. Enable pfblocker. Sugestion: in advance to reconfigure anything or do an update, please do a backup of a running config.

bmeeks

@newUser2pfSense said in unbound DNS Resolver Will Not Start:

Ok, so I tried this two times and no luck -
I went to Diagnostics > Edit File and browsed to the file and typed the line back in and saved the file. As a test to make sure the line stayed in the file, I restarted pfSense. When logging back into the GUI, the unbound DNS Resolver is working, however, when I check the file, the line is not there.

Going to Services > DNS Resolver > General Settings > Custom options, I tried typing the line in there and saving and I eceived the following error message:

The following input errors were detected:

The generated config file cannot be parsed by unbound. Please correct the following errors:

/var/unbound/test/unbound.conf:105: error: syntax error

read /var/unbound/test/unbound.conf failed: 1 errors in configuration file

Hmm, /test/ is in this path. When I look in the file with the /test/ in the path, the line is there.

Manually typing into the config files will not be persistent. The GUI code recreates the conf files for all packages each time you start/stop the service or otherwise modify something in pfSense. All configuration is stored in the firewall's config.xml file and read out from there when creating or recreating conf files. When you go into SERVICES > DNS Resolver > General Settings > Custom Options and type in something, then when you click Save the conf file is recreated from scratch. Any changes you make directly on the filesystem (such as when using DIAGNOSTICS > EDIT FILE) are overwritten. This is true for all packages, and is something new users typically get confused by. You edit something on the command line and yet it doesn't "stay edited".