OpenVPN + WireGuard breaking DNS resolver. [SOLVED]
-
So you are adding only 193.138.218.74 and 10.8.0.1 as DNS servers in System > General Setup and setting Unbound in Forwarding mode?
Have you set those as being on the OpenVPN gateway?Unbound is probably trying to reach them before the VPN is established resulting in bad states on WAN or over Wireguard. You will need to see exactly what's happening in the failure situation and probably add rules to prevent that.
Steve
-
@stephenw10 said in OpenVPN + WireGuard breaking DNS resolver.:
So you are adding only 193.138.218.74 and 10.8.0.1 as DNS servers in System > General Setup and setting Unbound in Forwarding mode?
Exactly.
And i also tried using just one of those IPs as well.
Have you set those as being on the OpenVPN gateway?
If we are talking about optional Gateway setting in System / General Setup, then yes. I tried using WireGuard as a gateway, rebooted, nothing. Then i set gateway to OpenVPN, rebooted, nothing. I did the same for both DNS IPs.
Unbound is probably trying to reach them before the VPN is established resulting in bad states on WAN or over Wireguard.
Just to be clear. I did clean install of pfSense without WireGuard and same thing happens. Then, i removed OpenVPN interface and disabled OpenVPN client completely. Then, i installed WireGuard package, set it up, rebooted pfSense box, and it all works flawlessly.
You will need to see exactly what's happening in the failure situation and probably add rules to prevent that.
Steve
Failure starts after reboot. Should i post logs from Status / System Logs / System / DNS Resolver section right after reboot ?
Thanks Steve.
-
What I would do is look at the firewall states for outbound DNS connections in the failed condition and see which interfaces they are on. You can probably only connect to those DNS servers over the VPN so any connections that are leaving the WAN directly will fail.
-
@stephenw10 said in OpenVPN + WireGuard breaking DNS resolver.:
What I would do is look at the firewall states for outbound DNS connections in the failed condition and see which interfaces they are on. You can probably only connect to those DNS servers over the VPN so any connections that are leaving the WAN directly will fail.
@stephenw10 you were absolutely right. Here is the part of the log.
WAN icmp 192.168.5.2:23885 -> 192.168.5.1:23885 0:0 WAN udp 192.168.5.2:123 -> 194.58.205.148:123 MULTIPLE:MULTIPLE WAN udp 192.168.5.2:123 -> 194.58.203.148:123 MULTIPLE:MULTIPLE lo0 icmp 10.66.205.79:24517 -> 10.66.205.79:24517 0:0 lo0 icmp 10.16.0.2:24079 -> 193.138.218.74:24079 0:0 MULLVADVPN_WG udp 10.66.205.79:20140 (10.10.50.3:42613) -> 188.120.127.79:443 MULTIPLE:MULTIPLE MULLVADVPN_WG udp 10.66.205.79:3124 (10.10.50.3:49582) -> 188.120.127.142:443 MULTIPLE:MULTIPLE MULLVADVPN_WG tcp 10.66.205.79:12740 (10.10.50.3:44841) -> 142.250.74.78:443 ESTABLISHED:ESTABLISHED WAN udp 192.168.5.2:31671 -> 193.138.218.74:53 MULTIPLE:SINGLE WAN udp 192.168.5.2:29132 -> 193.138.218.74:53 MULTIPLE:SINGLE WAN udp 192.168.5.2:58525 -> 193.138.218.74:53 MULTIPLE:SINGLE WAN udp 192.168.5.2:33339 -> 193.138.218.74:53 MULTIPLE:SINGLE WAN udp 192.168.5.2:42101 -> 193.138.218.74:53 MULTIPLE:SINGLE WAN udp 192.168.5.2:64598 -> 193.138.218.74:53 MULTIPLE:SINGLE WAN udp 192.168.5.2:33082 -> 193.138.218.74:53 MULTIPLE:SINGLE WAN tcp 192.168.5.2:56200 -> 193.138.218.74:53 FIN_WAIT_2:FIN_WAIT_2 WAN udp 192.168.5.2:29890 -> 193.138.218.74:53 MULTIPLE:SINGLE WAN udp 192.168.5.2:13777 -> 193.138.218.74:53 MULTIPLE:SINGLE WAN tcp 192.168.5.2:49574 -> 193.138.218.74:53 ESTABLISHED:ESTABLISHED WAN udp 192.168.5.2:28760 -> 193.138.218.74:53 MULTIPLE:SINGLE WAN udp 192.168.5.2:8692 -> 193.138.218.74:53 MULTIPLE:SINGLE WAN tcp 192.168.5.2:15643 -> 193.138.218.74:53 ESTABLISHED:ESTABLISHED WAN udp 192.168.5.2:8340 -> 193.138.218.74:53 MULTIPLE:SINGLE WAN tcp 192.168.5.2:16416 -> 193.138.218.74:53 ESTABLISHED:ESTABLISHED WAN udp 192.168.5.2:123 -> 193.182.111.141:123 MULTIPLE:SINGLE MULLVADVPN_WG tcp 10.66.205.79:55823 (10.10.50.3:46476) -> 188.120.127.78:443 ESTABLISHED:ESTABLISHED MULLVADVPN_WG tcp 10.66.205.79:39780 (10.10.50.3:44047) -> 142.250.74.67:80 ESTABLISHED:ESTABLISHED MULLVADVPN_WG tcp 10.66.205.79:23684 (10.10.50.3:42393) -> 142.250.74.164:443 ESTABLISHED:ESTABLISHED MULLVADVPN_WG udp 10.66.205.79:53224 (10.10.50.3:46297) -> 216.58.207.214:443 MULTIPLE:MULTIPLE MULLVADVPN_WG udp 10.66.205.79:24477 (10.10.50.3:49690) -> 188.120.127.76:443 MULTIPLE:MULTIPLE MULLVADVPN_WG udp 10.66.205.79:10898 (10.10.50.3:46000) -> 142.250.74.170:443 MULTIPLE:MULTIPLE MULLVADVPN_WG udp 10.66.205.79:34243 (10.10.50.3:44996) -> 142.250.74.164:443 MULTIPLE:MULTIPLE MULLVADVPN_WG udp 10.66.205.79:7771 (10.10.50.3:45347) -> 216.58.207.206:443 MULTIPLE:MULTIPLE WAN udp 192.168.5.2:58284 -> 193.138.218.74:53 MULTIPLE:SINGLE
I can clearly see that IP address 193.138.218.74, which is MullvadVPN DNS, is attempting to establish connection on 192.168.5.2 which is my WAN address. After i go to System > General and hit save button, those requests redirected to to MULLVADVPN_WG and MULLVADOVPN interfaces. And thats when it all starts working correctly.
Do i need to add
sleep xx
Where xx is the number of seconds in rc.bootup file, or theres more elegant way to solve this ?
Thanks.
-
I would try adding a floating outbound block rule on WAN to match that and prevent any states opening on WAN.
-
@stephenw10 said in OpenVPN + WireGuard breaking DNS resolver.:
I would try adding a floating outbound block rule on WAN to match that and prevent any states opening on WAN.
I tried but it didnt work. BlockWAN is the alias containing both MullvadVPN DNS IPs. I tried block and reject, and i enabled Quick option.
After reboot, state table still shows DNS requests made on port 53 on on my WAN IP 192.168.5.2. It seems like these DNS requests are taking place before firewall rules have the chance to block them.
Again, once the system is fully up, i just click save in System / General and everything starts working.
-
The source port should be 'any' there not 53. If the look at the states created the source port is a random high numbered port.
Steve
-
@stephenw10 said in OpenVPN + WireGuard breaking DNS resolver.:
The source port should be 'any' there not 53. If the look at the states created the source port is a random high numbered port.
Steve
I fixed that, but this rule blocks DNS resolution completely.
🔒 Log in to viewI tried setting port to "any" in destination, but same thing happens. Going to System / General and hitting save no longer works if this rule is enabled.
I experimented some more, and i made a tiny progress. If i set my DNS Server Settings like this
and then i select Mullvad OpenVPN interface as gateway in firewall rules, everything works fine after reboot. However, roles have been switched now. And all clients that use WireGuard gateway dont have DNS resolution or any traffic for that matter. WireGuard applet now shows 0 peers.
There is also no handshake
And going to System / General doesnt help like before.
-
@nimrod said in OpenVPN + WireGuard breaking DNS resolver.:
and then i select Mullvad OpenVPN interface as gateway in firewall rules, everything works fine after reboot.
In what firewall rules? Traffic from the firewall itself cannot be policy routed. It will always use the system routing table which is why I was asking you about the static route to the DNS server added by setting a gateway against them.
Steve
-
@stephenw10 said in OpenVPN + WireGuard breaking DNS resolver.:
@nimrod said in OpenVPN + WireGuard breaking DNS resolver.:
and then i select Mullvad OpenVPN interface as gateway in firewall rules, everything works fine after reboot.
In what firewall rules? Traffic from the firewall itself cannot be policy routed. It will always use the system routing table which is why I was asking you about the static route to the DNS server added by setting a gateway against them.
Steve
I think we didnt understand each other because im bad at explaining things. Sorry. Here is the entire configuration for the scenario that i described in the first post.
This is the status of the Mullvad OpenVPN client once its configured using their instructions.
🔒 Log in to view
As you can see, it connects with no issues.Here is the interface configuration of the Mullvad OpenVPN client.
🔒 Log in to view
Here is the OpenVPN gateway that gets automatically created when i add and enable OpenVPN interface. I just added MullvadDNS IP as the gateway monitoring address.This is the configuration under System / Advanced /
/ MiscellaneousThese are the manual NAT rules. Only local subnets are using OpenVPN and sometimes i switch to WG as well. I also have a WAN NAT rule so i can access DSL modem web configuration.
These are the firewall rules on the LAN interface aka Local Subnets.
🔒 Log in to view
These are the firewall rules on the WiFi interface aka WiFi Networks subnet. As you can see here all wifi devices are going through Mullvad WireGuard gateway. I just censored their descriptions and aliases containing their IP addresses.And these are the firewall rules on the final subnet. A single PC running FreeBSD.
These are the settings under System / General Setup
These are the settings in Services / DNS Resolver / General Settings
🔒 Log in to view
🔒 Log in to viewThis is the status of WireGuard VPN
🔒 Log in to view
This is how it all looks like in System / Routing / Gateways
🔒 Log in to viewThis is how it all looks like after reboot, however, none of the 3 subnets have DNS resolution.
As i said before, i go to System / General Setup, i change nothing, hit the save button, and thats it.
Local Subnets start working without any issues going through OpenVPN gateway, and the other two subnets start working without any issues going through WireGuard gateway. Confirmend by using MullvadVPN connection check.
Its obvious that this setup works. Its just that it stops working after reboot. What am i doing wrong here ?
-
When you add an alternative monitoring IP to a gateway a static route to that IP via the gateway is added to ensure it's actually monitoring the correct gateway. Since you are using the Mulvard DNS server there it means it can only ever connect over the WG VPN.
You have not set 'Skip rules when gateway is down' which means that means that if a gateway does go down the rules are just created without a gateway which here means traffic would just leave over the WAN. That's probably not what you want.
Steve
-
@stephenw10 said in OpenVPN + WireGuard breaking DNS resolver.:
When you add an alternative monitoring IP to a gateway a static route to that IP via the gateway is added to ensure it's actually monitoring the correct gateway. Since you are using the Mulvard DNS server there it means it can only ever connect over the WG VPN.
Unbelivalble. I cant belive this was causing the issue.
You have not set 'Skip rules when gateway is down' which means that means that if a gateway does go down the rules are just created without a gateway which here means traffic would just leave over the WAN. That's probably not what you want.
Steve
Thank for very much for your help @stephenw10 !!!
Cheers.
-
No worries. Let me know if that helps. There easily be more interactions happening there based on the connection timing.
Steve
-
@stephenw10 said in OpenVPN + WireGuard breaking DNS resolver.:
No worries. Let me know if that helps. There easily be more interactions happening there based on the connection timing.
Steve
It works !!
I removed the monitoring IP`s on both gateways, and i enabled "Do not create rules when gateway is down" in System / Advanced / Miscellaneous.
After reboot, both WireGuard and OpenVPN clients connected as usual and all subnets are going through their designated gateways.
Once again, thank you @stephenw10 !!!
-