Missing connectivity after setting up wireguard

IamLunchbox · Jun 23, 2022, 7:33 AM

Dear community,
two times in a row I set up wireguard and each time got a strange behaviour from the firewall after doing.
My steps were:

Installing the package
Setting up the tunnel
peer configuration
corresponding interface
gateway for wireguard
static route for the wireguard ip range
Proxy ARP for the wireguard ip range

Why proxy-arp, gateway and static route? A hypervisor service on the vm-host is using a lan-interface to hop onto the wireguard connection and push updates to the remote system.

After allowing the traffic in, my peer can reach me and I him (site-to-site vpn is my goal in the end), but the firewall stops to connect to the outside world. When I tcpdump on the hypervisor it seems, that the firewall starts to use the exit ip 0.0.0.0 for dns resolution...so the log is flooded with udp requests to my designated dns-servers, which obviously don't get any routing or response.

Disabling wireguard did not help at all, I always have roll back the firewall to get dns working again. Restarting unbound lead to no success either - it just restarts the mentioned traffic above.

Is this behavior known? Is this possibly a misconfiguration of wireguard, a bug or another fault within the routing settings?

I am grateful for any advice :)

dbosiljevac · Jul 19, 2022, 7:26 PM

@iamlunchbox I am having this issue as well.

Everything was working fine for a few days but then, last night, I did a reboot of my pfSense and connectivity to the outside world stopped from the firewall. I wasn't able to ping or do DNS resolution. I rebooted a few times and the issue was still there. Thankfully all the machines behind the firewall that weren't set up to use the firewall as a DNS resolver were working fine and traffic was passing normally.

I disabled the WireGuard service (without rebooting the firewall) and it still didn't work.

Through some sleuthing, we figured out that the source IP was being set as 0.0.0.0 on pings. When we set the source IP to our WAN IP on the pings, everything worked just fine.

This morning, for fun, I disabled WireGuard and then rebooted the pfSense and everything was back to normal again. Not sure where the problem lies here.

IamLunchbox · Jul 24, 2022, 6:34 PM

@dbosiljevac Hi, thank you for confirming, that this problem exists. Sorry for taking so long to reply, but I was pretty busy privately.

I stopped using wireguard for some time then and went for another solution: Connecting my proxmox host directly to the backup server through a wireguard tunnel on the given hosts.

Now, about a month later, I retried to set up wireguard to connect external hosts into my infrastructure. I could not reproduce the problem, but i diverged from my original setup in one way:
I set the wireguard IP in the dedicated interface and did not touch static routes and gateways at all. Originally, I used this to route traffic onto the hypervisor through wireguard. But having a working setup now I believe, that this was the culprit. Hope this helps you. =)

dbosiljevac · Jul 25, 2022, 5:25 PM

@iamlunchbox Hey there... Thanks for the reply.

I ended up bypassing my pfSense for WireGuard and now do a port forward from my firewall IP to a Linux VM that I'm using as my WG server. It's been working beautifully ever since I did that.

IamLunchbox · Aug 4, 2022, 10:40 AM

I have since been able to reproduce the issue, having the same problem but less setup:

Install the package
Set up the tunnel (Assign interface, configure a peer)
Set up a WAN-Rule to allow connections and do a test.
Done

After the pfsense is rebooted, my WAN-Gateway for ipv4 goes down (Services -> Gateways) and the firewall is not able to do an ipv4 based update-check or ipv4 based ping (using Select Interface automatically). Everything works using ipv6, I tested the ipv4 functionality using the context switch in Advanced setup -> Networking.

On the other hand, certain ipv4 services still work, namely:

name resolution using ipv4
pinging, if the WAN-interface is explicitly chosen

Disabling wireguard does not solve the issue, uninstalling and rebooting does though. Logs give no hint as of why this issue arises. Does anybody have an idea? Or shall I issue a github-issue to the corresponding wireguard-plugin?