Unbound access issue? (Unable to check for updates) No client issues!

yattengate

I am running out of ideas on what can be wrong (and what actually caused that).

The system (2.7.0) was working fine and the only trigger to this issue as far as I could recall was ISP's public IP change.
I restarted pfSense since and see no effects.

DNS resolution works:

I can nslookup netgate.com <my pfsense local IP> from machine attached to pfSense and it responds immediately.
I can resolve netgate.com from "Diagnostics / DNS Lookup" and timing from localhost is 0 msec, but the page takes 30+ seconds to reload.
I can run pfSense-upgrade -d via SSH and it succeeds, though it is also feels slow and I don't have a benchmark to compare to.
All clients work in general, though I suspect I have some issues with FW rule with FQDN aliases due to same problem.

"System / Update / System Update" and Home Page both show Unable to check for updates and both are very slow to load.

It looks to me like pfSense itself cannot properly talk to its Unbound, times out and defaults to configured external DNS servers?
Can this be possible?
I even added a floating rule to ensure no FW intervention to no effect.

Screenshot from 2023-10-25 16-34-56.png

unbound.conf below

##########################

Unbound Configuration

##########################

Server configuration

server:

chroot: /var/unbound
username: "unbound"
directory: "/var/unbound"
pidfile: "/var/run/unbound.pid"
use-syslog: yes
port: 53
verbosity: 5
hide-identity: yes
hide-version: yes
harden-glue: yes
do-ip4: yes
do-ip6: no
do-udp: yes
do-tcp: yes
do-daemonize: yes
module-config: "validator iterator"
unwanted-reply-threshold: 0
num-queries-per-thread: 512
jostle-timeout: 200
infra-keep-probing: yes
infra-host-ttl: 900
infra-cache-numhosts: 10000
outgoing-num-tcp: 10
incoming-num-tcp: 10
edns-buffer-size: 4096
cache-max-ttl: 86400
cache-min-ttl: 0
harden-dnssec-stripped: yes
msg-cache-size: 4m
rrset-cache-size: 8m
qname-minimisation: yes

num-threads: 4
msg-cache-slabs: 4
rrset-cache-slabs: 4
infra-cache-slabs: 4
key-cache-slabs: 4
outgoing-range: 4096
#so-rcvbuf: 4m
auto-trust-anchor-file: /var/unbound/root.key
prefetch: no
prefetch-key: yes
use-caps-for-id: no
serve-expired: yes
aggressive-nsec: no

Statistics

Unbound Statistics

statistics-interval: 0
extended-statistics: yes
statistics-cumulative: yes

TLS Configuration

tls-cert-bundle: "/etc/ssl/cert.pem"
tls-port: 853
tls-service-pem: "/var/unbound/sslcert.crt"
tls-service-key: "/var/unbound/sslcert.key"

Interface IP addresses to bind to

interface: 172.25.1.1
interface: 172.25.1.1@853
interface: 172.29.0.1
interface: 172.29.0.1@853
interface: 172.25.15.1
interface: 172.25.15.1@853
interface: 172.27.0.1
interface: 172.27.0.1@853
interface: 172.28.0.1
interface: 172.28.0.1@853
interface: 172.25.17.1
interface: 172.25.17.1@853
interface: 172.25.10.1
interface: 172.25.10.1@853
interface: 172.29.2.1
interface: 172.29.2.1@853
interface: 172.25.13.1
interface: 172.25.13.1@853
interface: 172.29.1.1
interface: 172.29.1.1@853
interface: 172.25.20.1
interface: 172.25.20.1@853
interface: 172.25.14.1
interface: 172.25.14.1@853
interface: 172.25.19.1
interface: 172.25.19.1@853
interface: 172.26.0.1
interface: 172.26.0.1@853
interface: 172.25.12.1
interface: 172.25.12.1@853
interface: 127.0.0.1
interface: 127.0.0.1@853
interface: ::1
interface: ::1@853

Outgoing interfaces to be used

outgoing-interface: <ISP public IP here>

DNS Rebinding

For DNS Rebinding prevention

private-address: 127.0.0.0/8
private-address: 10.0.0.0/8
private-address: ::ffff:a00:0/104
private-address: 172.16.0.0/12
private-address: ::ffff:ac10:0/108
private-address: 169.254.0.0/16
private-address: ::ffff:a9fe:0/112
private-address: 192.168.0.0/16
private-address: ::ffff:c0a8:0/112
private-address: fd00::/8
private-address: fe80::/10

Access lists

include: /var/unbound/access_lists.conf

Static host entries

include: /var/unbound/host_entries.conf

dhcp lease entries

include: /var/unbound/dhcpleases_entries.conf

OpenVPN client entries

include: /var/unbound/openvpn.*.conf

Domain overrides

include: /var/unbound/domainoverrides.conf

Unbound custom options

server:local-zone: "use-application-dns.net" static
server:local-zone: "cloudflare-dns.com" static
server:local-zone: "mozilla.cloudflare-dns.com" static

server:include: /var/unbound/pfb_dnsbl.*conf

Remote Control Config

include: /var/unbound/remotecontrol.conf

yattengate

Found two lines logged, which may be related; how to interpret this?

debug: ignoring duplicate RR: localhost. A 127.0.0.1
debug: donotq: 127.0.0.0/8

yattengate

I hate the magic, but the problem self-resolved :(

I got another public IP update and everything started working again.

For the record, the IP change was 174.93.x.y (good) -> 70.53.x.y (bad) -> 174.95.x.y (good)

I checked the range registration on ARIN and it is at least 2020; did not find it in bogon list.