unbound DNS Resolver Will Not Start

newUser2pfSense · Jun 10, 2020, 4:10 PM

Using the console, I just updated pfSense to 2.4.5-RELEASE-p1 (amd64) from 2.4.5. The update went well. However, in the Dashboard GUI for Services Status, I see the unbound DNS Resolver is in a stopped state. I've pressed the Start Service button several times and it won't start. login-to-view
The DNS Resolver is checked enabled in Services > DNS Resolver.

I have a pretty vanilla pfSense installation. I have the following packages installed:
nmap 1.4.4_1
openvpn-client-export 1.4.23
pfBlockerNG-devel 2.2.5_32
suricata 5.0.2_2

pfBlockerNG gets rid of ads for me. I used the Configuring Quad9 on pfSense tutorial found here:
https://linuxincluded.com/configuring-quad9-on-pfsense/
and am using Quad9 DNS servers: 9.9.9.9, 149.112.112.112.

I'll be the first to admit that I'm not a network guru by any stretch of the imagination. Seeing that I can't get out to the internet, any ideas what may have gone wrong and how to fix it? Any suggestions would be most helpful. Thank you.

pete35 · Jun 10, 2020, 4:20 PM

Maybe Pfblocker includes some configurations under custom options of unbound. Sometimes the syntax is wrong there. You can try to remove them and start unbound (the resolver) again. If this works, try to correct the syntax there.

newUser2pfSense · Jun 10, 2020, 4:42 PM

When I restart pfSense and watch the scrolling console screen, I see the following line:
Starting DNS Resolver...done.

The DNS resolver must be stopped just after this line gets displayed in the console.

As a test, I just disabled pfBlockerNG completely and restarted from the console. When I logged into the GUI, I see the same unbound DNS Resolver is stopped and again it won't start. Is it safe to say it's not pfBlockerNG?

pete35 · Jun 10, 2020, 4:44 PM

Maybe unbound logs why it cant start, check the logs under "status/system logs/system/dns resolver".

The syntax in "custom options" maybe wrong with and without pfblocker, if there is anything in there and unbound refuses to start. Usually there will be an entry in the logs.

bmeeks · Jun 10, 2020, 4:43 PM

@newUser2pfSense said in unbound DNS Resolver Will Not Start:

When I restart pfSense and watch the scrolling console screen, I see the following line:
Starting DNS Resolver...done.

The DNS resolver must be stopped just after this line gets displayed in the console.

As a test, I just disabled pfBlockerNG completely and restarted from the console. When I logged into the GUI, I see the same unbound DNS Resolver is stopped and again and it won't start. Is it safe to say it's not pfBlockerNG?

No, not safe to say that. pfBlockerNG may have left Unbound with a corrupt or incorrectly configured conf file. You need to look in the logs for the system and resolver (unbound) to see that it is reporting (if anything).

newUser2pfSense · Jun 10, 2020, 5:05 PM

I'm not sure if this helps or not but here are the last log file lines in the System Logs > System > DNS Resolver after updating and it repeats about 5 times:

Jun 10 09:15:18 unbound 22966:1 info: 127.0.0.1 _https._tcp.pkg.pfsense.org. SRV IN NOERROR 0.000000 1 123
Jun 10 09:15:18 unbound 22966:1 info: 127.0.0.1 files00.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:6 info: 127.0.0.1 files00.netgate.com. AAAA IN NOERROR 0.000000 1 65
Jun 10 09:15:18 unbound 22966:3 info: 127.0.0.1 files01.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:3 info: 127.0.0.1 files01.netgate.com. AAAA IN NOERROR 0.000000 1 65
Jun 10 09:15:18 unbound 22966:6 info: 127.0.0.1 files00.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:6 info: 127.0.0.1 files00.netgate.com. AAAA IN NOERROR 0.000000 1 65
Jun 10 09:15:18 unbound 22966:6 info: 127.0.0.1 files01.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:1 info: 127.0.0.1 files01.netgate.com. AAAA IN NOERROR 0.000000 1 65
Jun 10 09:15:18 unbound 22966:2 info: 127.0.0.1 files00.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:6 info: 127.0.0.1 files00.netgate.com. AAAA IN NOERROR 0.000000 1 65
Jun 10 09:15:18 unbound 22966:7 info: 127.0.0.1 files01.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:7 info: 127.0.0.1 files01.netgate.com. AAAA IN NOERROR 0.000000 1 65

Here's an interesting line in the System Logs > System > General

Jun 10 12:35:05 php-fpm 88498 /status_services.php: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '/var/unbound/unbound.conf:105: error: syntax error read /var/unbound/unbound.conf failed: 1 errors in configuration file [1591806905] unbound[31342:0] fatal error: Could not read config file: /var/unbound/unbound.conf. Maybe try unbound -dd, it stays on the commandline to see more errors, or unbound-checkconf'

When I enter "unbound-checkconf" (no quotes) in the Diagnostics > Command Prompt, I get the following:

unbound-checkconf: no errors in /usr/local/etc/unbound/unbound.conf

pete35 · Jun 10, 2020, 5:07 PM

You can try to restore the pfsense configuration from your backup.

If there is none, just try to reconfigure the resolver and save it again, hopeing that it will repair the incorrect configuration file.

You can also look into the configuration file around the mentioned line to check if there is a missconfiguration or syntax error and try to correct it. You need to login on console to do that.

newUser2pfSense · Jun 10, 2020, 5:18 PM

So I made a mistake and didn't backup my configuration before updating. Wow, big mistake. That won't happen again! The backups that I do have are just over 3 months old and I'm not sure if they have my complete configuration or not.

How would I go about reconfiguring the resolver and then save it again? Any ideas?

bmeeks · Jun 10, 2020, 5:26 PM

@newUser2pfSense said in unbound DNS Resolver Will Not Start:

I'm not sure if this helps or not but here are the last log file lines in the System Logs > System > DNS Resolver after updating and it repeats about 5 times:

Jun 10 09:15:18 unbound 22966:1 info: 127.0.0.1 _https._tcp.pkg.pfsense.org. SRV IN NOERROR 0.000000 1 123
Jun 10 09:15:18 unbound 22966:1 info: 127.0.0.1 files00.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:6 info: 127.0.0.1 files00.netgate.com. AAAA IN NOERROR 0.000000 1 65
Jun 10 09:15:18 unbound 22966:3 info: 127.0.0.1 files01.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:3 info: 127.0.0.1 files01.netgate.com. AAAA IN NOERROR 0.000000 1 65
Jun 10 09:15:18 unbound 22966:6 info: 127.0.0.1 files00.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:6 info: 127.0.0.1 files00.netgate.com. AAAA IN NOERROR 0.000000 1 65
Jun 10 09:15:18 unbound 22966:6 info: 127.0.0.1 files01.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:1 info: 127.0.0.1 files01.netgate.com. AAAA IN NOERROR 0.000000 1 65
Jun 10 09:15:18 unbound 22966:2 info: 127.0.0.1 files00.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:6 info: 127.0.0.1 files00.netgate.com. AAAA IN NOERROR 0.000000 1 65
Jun 10 09:15:18 unbound 22966:7 info: 127.0.0.1 files01.netgate.com. A IN NOERROR 0.000000 1 53
Jun 10 09:15:18 unbound 22966:7 info: 127.0.0.1 files01.netgate.com. AAAA IN NOERROR 0.000000 1 65

Here's an interesting line in the System Logs > System > General

Jun 10 12:35:05 php-fpm 88498 /status_services.php: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '/var/unbound/unbound.conf:105: error: syntax error read /var/unbound/unbound.conf failed: 1 errors in configuration file [1591806905] unbound[31342:0] fatal error: Could not read config file: /var/unbound/unbound.conf. Maybe try unbound -dd, it stays on the commandline to see more errors, or unbound-checkconf'

When I enter "unbound-checkconf" (no quotes) in the Diagnostics > Command Prompt, I get the following:

unbound-checkconf: no errors in /usr/local/etc/unbound/unbound.conf

You ran the "checkconf" on the wrong unbound.conf file. Look again at your error message. It does not like the unbound.conf file in '/var/unbound. So you will need to run the unbound-checkconf command against /var/unbound/unbound.conf to find out what's actually wrong.

See, here is the actual error message:

fatal error: Could not read config file: /var/unbound/unbound.conf.

Because you gave the unbound-checkconf utility no parameters, it checked the default file here:

no errors in /usr/local/etc/unbound/unbound.conf

but that is NOT the file that unbound uses when it actually runs. It runs from config files in /var/unbound.

pete35 · Jun 10, 2020, 5:27 PM

Just change any number on the "advanced resolver options" under "advanced settings" save it and change it back. Save it again. Try to start unbound then and check the logs again.

newUser2pfSense · Jun 10, 2020, 5:40 PM

So I changed a value in the "Advanced Resolver Options", saved it, changed it back, saved it again.

In the Status > System Logs > System > DNS Resolver:

Nothing changed...no new lines at all.

In the Status > System Logs > System > General, these are the new lines:

Jun 10 13:30:24 php-fpm 357 /services_unbound_advanced.php: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '/var/unbound/unbound.conf:105: error: syntax error read /var/unbound/unbound.conf failed: 1 errors in configuration file [1591810224] unbound[36841:0] fatal error: Could not read config file: /var/unbound/unbound.conf. Maybe try unbound -dd, it stays on the commandline to see more errors, or unbound-checkconf'
Jun 10 13:30:51 check_reload_status Syncing firewall
Jun 10 13:30:54 php-fpm 356 /services_unbound_advanced.php: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '/var/unbound/unbound.conf:105: error: syntax error read /var/unbound/unbound.conf failed: 1 errors in configuration file [1591810254] unbound[69110:0] fatal error: Could not read config file: /var/unbound/unbound.conf. Maybe try unbound -dd, it stays on the commandline to see more errors, or unbound-checkconf'
Jun 10 13:31:02 php-fpm 356 /status_services.php: The command '/usr/local/sbin/unbound -c /var/unbound/unbound.conf' returned exit code '1', the output was '/var/unbound/unbound.conf:105: error: syntax error read /var/unbound/unbound.conf failed: 1 errors in configuration file [1591810262] unbound[82148:0] fatal error: Could not read config file: /var/unbound/unbound.conf. Maybe try unbound -dd, it stays on the commandline to see more errors, or unbound-checkconf'

In the Diagnostics > Command prompt, I ran the following command:

unbound-checkconf /var/unbound/unbound.conf

The result:
/var/unbound/unbound.conf:105: error: syntax error
read /var/unbound/unbound.conf failed: 1 errors in configuration file

pete35 · Jun 10, 2020, 5:40 PM

Maybe post your unbound.conf here?

newUser2pfSense · Jun 10, 2020, 5:46 PM

From: /var/unbound/unbound.conf

##########################

Unbound Configuration

##########################

Server configuration

server:

chroot: /var/unbound
username: "unbound"
directory: "/var/unbound"
pidfile: "/var/run/unbound.pid"
use-syslog: yes
port: 53
verbosity: 2
hide-identity: yes
hide-version: yes
harden-glue: yes
do-ip4: yes
do-ip6: no
do-udp: yes
do-tcp: yes
do-daemonize: yes
module-config: "validator iterator"
unwanted-reply-threshold: 0
num-queries-per-thread: 512
jostle-timeout: 200
infra-host-ttl: 900
infra-cache-numhosts: 10000
outgoing-num-tcp: 10
incoming-num-tcp: 10
edns-buffer-size: 4096
cache-max-ttl: 86400
cache-min-ttl: 0
harden-dnssec-stripped: yes
msg-cache-size: 4m
rrset-cache-size: 8m

num-threads: 8
msg-cache-slabs: 8
rrset-cache-slabs: 8
infra-cache-slabs: 8
key-cache-slabs: 8
outgoing-range: 4096
#so-rcvbuf: 4m
auto-trust-anchor-file: /var/unbound/root.key
prefetch: no
prefetch-key: no
use-caps-for-id: no
serve-expired: no

Statistics

Unbound Statistics

statistics-interval: 0
extended-statistics: yes
statistics-cumulative: yes

TLS Configuration

tls-cert-bundle: "/etc/ssl/cert.pem"

Interface IP(s) to bind to

interface-automatic: yes
interface: 0.0.0.0
interface: ::0

Outgoing interfaces to be used

DNS Rebinding

For DNS Rebinding prevention

private-address: 127.0.0.0/8
private-address: 10.0.0.0/8
private-address: ::ffff:a00:0/104
private-address: 172.16.0.0/12
private-address: ::ffff:ac10:0/108
private-address: 169.254.0.0/16
private-address: ::ffff:a9fe:0/112
private-address: 192.168.0.0/16
private-address: ::ffff:c0a8:0/112
private-address: fd00::/8
private-address: fe80::/10

Access lists

include: /var/unbound/access_lists.conf

Static host entries

include: /var/unbound/host_entries.conf

dhcp lease entries

include: /var/unbound/dhcpleases_entries.conf

Domain overrides

include: /var/unbound/domainoverrides.conf

Forwarding

forward-zone:
name: "."
forward-tls-upstream: yes
forward-addr: 9.9.9.9@853
forward-addr: 149.112.112.112@853

Unbound custom options

log-replies: yes

Remote Control Config

include: /var/unbound/remotecontrol.conf

bmeeks · Jun 10, 2020, 5:49 PM

@newUser2pfSense said in unbound DNS Resolver Will Not Start:

From: /var/unbound/unbound.conf

##########################

Unbound Configuration

##########################

Server configuration

server:

chroot: /var/unbound
username: "unbound"
directory: "/var/unbound"
pidfile: "/var/run/unbound.pid"
use-syslog: yes
port: 53
verbosity: 2
hide-identity: yes
hide-version: yes
harden-glue: yes
do-ip4: yes
do-ip6: no
do-udp: yes
do-tcp: yes
do-daemonize: yes
module-config: "validator iterator"
unwanted-reply-threshold: 0
num-queries-per-thread: 512
jostle-timeout: 200
infra-host-ttl: 900
infra-cache-numhosts: 10000
outgoing-num-tcp: 10
incoming-num-tcp: 10
edns-buffer-size: 4096
cache-max-ttl: 86400
cache-min-ttl: 0
harden-dnssec-stripped: yes
msg-cache-size: 4m
rrset-cache-size: 8m

num-threads: 8
msg-cache-slabs: 8
rrset-cache-slabs: 8
infra-cache-slabs: 8
key-cache-slabs: 8
outgoing-range: 4096
#so-rcvbuf: 4m
auto-trust-anchor-file: /var/unbound/root.key
prefetch: no
prefetch-key: no
use-caps-for-id: no
serve-expired: no

Statistics

Unbound Statistics

statistics-interval: 0
extended-statistics: yes
statistics-cumulative: yes

TLS Configuration

tls-cert-bundle: "/etc/ssl/cert.pem"

Interface IP(s) to bind to

interface-automatic: yes
interface: 0.0.0.0
interface: ::0

Outgoing interfaces to be used

DNS Rebinding

For DNS Rebinding prevention

private-address: 127.0.0.0/8
private-address: 10.0.0.0/8
private-address: ::ffff:a00:0/104
private-address: 172.16.0.0/12
private-address: ::ffff:ac10:0/108
private-address: 169.254.0.0/16
private-address: ::ffff:a9fe:0/112
private-address: 192.168.0.0/16
private-address: ::ffff:c0a8:0/112
private-address: fd00::/8
private-address: fe80::/10

Access lists

include: /var/unbound/access_lists.conf

Static host entries

include: /var/unbound/host_entries.conf

dhcp lease entries

include: /var/unbound/dhcpleases_entries.conf

Domain overrides

include: /var/unbound/domainoverrides.conf

Forwarding

forward-zone:
name: "."
forward-tls-upstream: yes
forward-addr: 9.9.9.9@853
forward-addr: 149.112.112.112@853

Unbound custom options

log-replies: yes

Remote Control Config

include: /var/unbound/remotecontrol.conf

You need to open this file in an editor and find line #105. That's where the error is. Look again at the error message you are given from the logs:

the output was '/var/unbound/unbound.conf:105: error: syntax error

The ":105" part is the line number where the syntax error is located.

newUser2pfSense · Jun 10, 2020, 5:51 PM

I copied the entire contents and pasted it into a text file in an editor and I believe line 105 is:

log-replies: yes

pete35 · Jun 10, 2020, 5:53 PM

Go to the custom options in the unbound gui and remove this line. Save and restart unbound.

bmeeks · Jun 10, 2020, 5:54 PM

@newUser2pfSense said in unbound DNS Resolver Will Not Start:

I copied the entire contents and pasted it into a text file in an editor and I believe line 105 is:

log-replies: yes

I don't see anything wrong on that line, but there may be extra control or other characters present that either did not get copied into your post or show up as whitespace and thus are invisible. Clearly there is something on line 105 that unbound does not care for. Could be something just in front of, or just behind, that line number.

bmeeks · Jun 10, 2020, 5:57 PM

User @pete35 is on the right track with his suggestions. You need to wipe out all of the custom conf stuff that pfBlockerNG would have added to unbound's conf file.

newUser2pfSense · Jun 10, 2020, 5:58 PM

So I went to Services > DNS Resolver > General settings and deleted the log-replies: yes.

As soon as I went to the dashboard, the Services Status > unbound DNS Resolver is now started.

bmeeks · Jun 10, 2020, 6:00 PM

@newUser2pfSense said in unbound DNS Resolver Will Not Start:

So I went to Services > DNS Resolver > General settings and deleted the log-replies: yes.

As soon as I went to the dashboard, the Services Status > unbound DNS Resolver is now started.

Great! There must have been something else lurking on that line because according to the man page I found on Google that option is a vaild one.