Unbound not responding on all chosen interfaces after reboot

DBMandrake

Ok I've just realised that the code I'm looking at on Github is new and doesn't represent what is running in 2.6.0. When I look at /etc/rc.linkup on my running 2.6.0 system it does have code to restart unbound on linkup - although it doesn't seem to be working for me on startup:

                switch ($argument2) {
                        case "stop":
                        case "down":
                                log_error("DEVD Ethernet detached event for {$iface}");
                                interface_bring_down($iface);
                                break;
                        case "start":
                        case "up":
                                log_error("DEVD Ethernet attached event for {$iface}");
                                log_error("HOTPLUG: Configuring interface {$iface}");
                                // Do not try to readd to bridge otherwise em(4) has problems
                                interface_configure($iface, true, true);

                                /* Make sure gw monitor is configured */
                                if ($ip6addr == 'slaac' ||
                                    $ip6addr == 'dhcp6') {
                                        setup_gateways_monitor();
                                }
                                /* restart unbound on interface recover,
                                 * https://redmine.pfsense.org/issues/11547 */
                                if (isset($config['unbound']['enable']) &&
                                    (in_array($iface, explode(',', $config['unbound']['active_interface'])) ||
                                    in_array($iface, explode(',', $config['unbound']['outgoing_interface'])))) {
                                        services_unbound_configure();
                                }
                                break;
                }
        }
}

It looks like this whole code area is being significantly re-written for 2.7.0.

Gertjan

@dbmandrake

About the log lines :

Nov 29 09:18:48 pfSense-Home check_reload_status[409]: Linkup starting igb0
and more

Why would your igb1=LAN would have to handle a "link state changed to DOWN" and "link state changed to UP" this late in the startup of the system ?
LAN is, normally, a statically set interface and gets initialized in very early booting process.
Typically, it goes to a switch that should be the 'always' on.

If unbound was told (according the config file) to bind to 192.168.1.1 and it can't during startup, it will complain in the log file.
You didn't mention that, so, according to unbound, all is well : config told it to bind to 127.0.0.1 and that's it.

When your system boots, an unbound.conf is created that says : listen only to local host IPv4 = 127.0.0.1 then it's the pfSense low level GUI that fails to build a correct config file.
Or the info stored is wrong, see the config.xml file ( /cf/conf/config.xml )

This is a not-good situation, as you've sue the info differently in the GUI, resolver settings page.

I've read https://redmine.pfsense.org/issues/13254
And https://redmine.pfsense.org/issues/12613 which is related.

There is a solution.
Put a switch between your pfSense LAN port and your other devices.
This will stop the flapping on the LAN interface.
Also : use this : select All :

as you know by now that you can trust your firewall and no one will be able to contact your DNS resolver "from the out side", the WAN interface(s)
So select All and call it a day ;)

You unbound.conf interfaces list

:# Interface IP(s) to bind to
interface: 127.0.0.1
interface: 127.0.0.1@853
interface: ::1
interface: ::1@853

Let's forget about 853 (used by the less known 'rndc') :

:# Interface IP(s) to bind to
interface: 127.0.0.1
interface: ::1

so unbound listens only to 127.0.0.1 IPv4 and ::1 IPv6.

Or, it should be something like (I removed the 853 stuff) (my list) :

# Interface IP(s) to bind to
interface: 192.168.1.1
interface: 2001:470:beef:5c0:2::1
interface: 192.168.100.1
interface: 192.168.100.1@853
interface: 2001:470:dead:100::1
interface: 192.168.2.1
interface: 192.168.3.1
interface: 2001:470:dead:3::1
interface: fe80::92ec:77ff:fe29:392c%igc0
interface: fe80::92ec:77ff:fe29:392e%igc2
interface: fe80::92ec:77ff:fe29:392d%igc1
interface: fe80::92ec:77ff:fe29:392c%ovpns1
interface: 127.0.0.1
interface: ::1

When I rebooted, I saw the same interfaces list.

which is ok for me as I have selected :

I'm am using IPv6 and IPv4 ;)

Btw : I'm using a Netgate device, the 4100, so I had to leave 2.6.0, I'm using 22.05 these days.

@dbmandrake said in Unbound not responding on all chosen interfaces after reboot:

It looks like this whole code area is being significantly re-written for 2.7.0.

Let's reserve that for future fun ^^

robotox

Hi,
I seem to have a similar problem.
In my case unbound is set to use Outgoing Interfaces that are VPNs.
I guess unbound ignores them as they haven't come up yet during its start.
I need to later restart the unbound service.
I can't even find proof of that.
But I've read about this behavior a couple of times.
So I guess it is definitely not related with one user with specific setup.
Thanks.

Gertjan

@robotox

With unbound set to listen to "All" interfaces, it will (I should say : should) listen on all available interfaces when the system boots and unbound starts.

Also : when using a OpenVPN client, see here https://www.youtube.com/watch?v=ulRgecz0UsQ at 9 minutes and 28 seconds : an 'OpenVPN' interface has to be created so the system and unbound has another interface.
In theory, as I didn't test this, I'm not using any OpenVPN client, when the OpenVPN clients start, and 'interface' event will happens : unbound gets restarted and now it will find the OpenVPN interface and 'bind' (use) to it. Did you saw such an event ?

robotox

@gertjan said in Unbound not responding on all chosen interfaces after reboot:

unbound gets restarted and now it will find the OpenVPN interface and 'bind' (use) to it. Did you saw such an event ?

Thank for your time.
I don't use "All" interfaces so that all DNS queries are routed to the VPNs (I have at the moment a total of 3 to test redundancy) for privacy reasons. It is said that this is the only way to prevent DNS leaks to the actual ISP at WAN.
Unbound doesn't seem to restart at all after initial boot for any reason. Watchdog not helpful in this case as the service is really running.
I just increased log level from 3 to 5 but I don't think it will show me any new information for what I am looking for.
Will shutdown and boot again to have a new look at it.
Thanks again.

robotox

@gertjan
Tried removing the physical WAN cable and it brings unbound to apparently resolve itself.

Tried removing all VPNs but leaving only 1.
Unbound starts 2 seconds before every interface and does nothing after that.

Thank you once more.

DBMandrake

@robotox said in Unbound not responding on all chosen interfaces after reboot:

@gertjan said in Unbound not responding on all chosen interfaces after reboot:

unbound gets restarted and now it will find the OpenVPN interface and 'bind' (use) to it. Did you saw such an event ?

Thank for your time.
I don't use "All" interfaces so that all DNS queries are routed to the VPNs (I have at the moment a total of 3 to test redundancy) for privacy reasons. It is said that this is the only way to prevent DNS leaks to the actual ISP at WAN.

Why not just enable forwarder mode in the unbound configuration and tell it to forward to a DNS server on the other end of your VPN ? (Presumably head office) That way it will not try to query root servers.

Also you could use egress rules in floating rules to block outgoing queries on the WAN interface so that even if unbound tried to send dns queries outside your VPN's they would be blocked.

Because unbound runs on the firewall itself only egress rules can block its query traffic.

robotox

@dbmandrake
Hi, unbound is in forwarding mode with selected Outgoing Interfaces -- not "All".
Tried with or without declaring the VPN provider's DNSs.
Floating rules are set up to prevent leaks to WAN.
As far as I understood, those would apply to the client devices only -- the firewall's setup for Unbound is managed separately.
As far as I understood, to achieve the desired solution I need to unselect "All" and select only those desired interfaces.
Thanks.

skogs

@dbmandrake
This is potentially wildly off topic; but are we attempting to spoof MACs?
I could potentially see it initially come up with real MAC; then swap to fake MAC and bork things.

robotox

Added my plea here https://redmine.pfsense.org/issues/13707.

tomeq82

I'm with that problem too. I'm using pfsense+ 23.01 but it happened on 2.6 too. My setup is a virtual pfSense running on qemu (Linux) with explictly passed through intel dual port 1 Gig adapter to the pfSense so it works as bare-metal NIC.

Each and every reboot of pfSense guest, machine reboot, renders Unbound unusable - it starts but it doesn't resolve a thing. Only restart of the service helps immediately.

My setup is similar to those described in the other places - listen on LAN, localhost, outgoing interface WAN. I tried various workarounds for this eg. setting gateway monitoring address to something few hops further than default ISP gw - like 1.1.1.1 but no avail... the issue persist.

robotox

Hi,
I now have an SG-2100 with 23.05.1 for the same setup and still the same problem.
Unbound fails to start as I have OpenVPNs as Outgoing Network Interfaces.
Still trying to get attention at https://redmine.pfsense.org/issues/13707.

DBMandrake

While it's technically only a workaround and I would hope this will get fixed one day, the pragmatic solution is to just set "Network Interfaces" and "Outgoing Network Interfaces" to All, and simply use firewall rules to block / allow access to the DNS server from client devices.

That way, no matter how interfaces go up or down during boot or later on (including VPN's going up and down after the system has booted) unbound will always bind to all interfaces, but access will be dictated by the firewall rules for each interface.

This is how I have been running ever since reporting the issue, and in some ways blocking using firewall rules is a more explicit and secure way to prevent access to the DNS server for clients who should not have access than relying on unbound to bind to the correct interfaces in its own configuration.

Given the default firewall rules for an interface are to block, this is fairly easy because unless you add an allow rule access to the DNS server is blocked by default including attempts to query from the WAN side.

Gertjan

@DBMandrake

Consider also using Unbound ACL rules.

DBMandrake

@Gertjan Much less secure, because unbound still receives and processes the packets and then decides whether they should be ignored or responded to based on its own configuration file.

If there was ever a problem like a buffer overflow found in unbound it would be vulnerable to attack from clients that are "blocked" by the ACL list but allowed by firewall rules.

Firewall rules on the other hand are absolute, and do not allow any packets to reach unbound for processing and would prevent such exploitation. So if you're going to bind to all interfaces (as in this workaround) why not just set access to unbound using firewall rules. I would not rely on unbounds own ACL's except to allow remote subnets which are normally denied by default. I would not rely on it as a means of blocking.

Gertjan

@DBMandrake

Now that's what I call 'considering'

robotox

Thank you for bringing the thread back to life!
But in my case, the problem being with Outgoing Interfaces, rules won't apply to the firewall.

robotox

Now testing the SG-2100 with 23.05.1 for the similar setup but with multiple Wireguards instead of multiple OpenVPNs.
Unbound starts correctly.
I am guessing that Wireguard is faster than OpenVPN starting at boot.
Thanks again.