pfSense 2.5.0 boxes with WireGuard keep crashing (both!)

z3us

@jimp Thanks so far! No trafficshaping active, already checked that.
I've attached two files from box NMG and four files from box UDN.
UDN crashed 4 times, the other box 2 times.
NMG box is also not receiving a DHCP address on the WAN interface after the reboot right after the crash. I have to disable and enable the DHCP setting on the interface to make it work again.

Thanks a lot in advance, hope you can find the issue![6_1614242663203_UDN_textdump.tar.4](Uploading 100%) [5_1614242663203_UDN_textdump.tar.3](Uploading 100%) [4_1614242663203_UDN_textdump.tar.2](Uploading 100%) [3_1614242663202_UDN_textdump.tar.1](Uploading 100%) [2_1614242663202_UDN_textdump.tar.0](Uploading 100%) [1_1614242663202_NMG_textdump.tar.1](Uploading 100%) [0_1614242663200_NMG_textdump.tar.0](Uploading 100%)

z3us

Both boxes also have these weird log messages every second. This happened right after enabling WireGuard....

Feb 25 09:46:37 kernel matchaddr failed
Feb 25 09:46:36 kernel matchaddr failed
Feb 25 09:46:36 kernel matchaddr failed
Feb 25 09:46:35 kernel matchaddr failed
Feb 25 09:46:35 kernel matchaddr failed
Feb 25 09:46:34 kernel matchaddr failed
Feb 25 09:46:34 kernel matchaddr failed

jimp

Looks like those uploads didn't complete properly, can you try again? Maybe one at a time to be certain they each work?

You might have to rename them so they are named textdump.1.tar instead of textdump.tar.1

z3us

NMG_textdump0.tar

z3us

NMG_textdump1.tar

z3us

UDN_textdump0.tar

z3us

UDN_textdump1.tar

z3us

UDN_textdump2.tar

z3us

UDN_textdump3.tar

z3us

UDN_textdump4.tar

z3us

UDN.zip

z3us

NMG.zip

jimp

OK, all of those did have identical backtraces.

Is there anything special you're doing with these tunnels that might be triggering it? Any services using WireGuard specifically? Or just lots of continued traffic?

I ask because unless we can find a way to reproduce it we can't necessarily be sure we've found and fixed the problem. Thus far I don't think anyone internally has his the particular panic.

jimp

I started an issue on Redmine for this crash:

https://redmine.pfsense.org/issues/11538

z3us

@jimp said in pfSense 2.5.0 boxes with WireGuard keep crashing (both!):

OK, all of those did have identical backtraces.

Is there anything special you're doing with these tunnels that might be triggering it? Any services using WireGuard specifically? Or just lots of continued traffic?

I ask because unless we can find a way to reproduce it we can't necessarily be sure we've found and fixed the problem. Thus far I don't think anyone internally has his the particular panic.

I understand your question, but there is very few traffic on this tunnel. Just some smb (filesharing) en domain controller logins. UDN has a domain controller and NMG has 3 desktops (domain members). I think there's not more than 1GB traffic in a day.

jimp

Is it all regular L3 traffic from one subnet to another? Or could some of it be trying to send broadcast or multicast on the WireGuard interface?

I wouldn't think so, since it can't be bridged and that would typically involve something like Samba running on the firewall (which it can't) but it makes me curious.

Also what entries do you have in "Allowed IPs" on both sides? Is it empty? Or do you have the remote subnets listed?

If you have the Allowed IPs list filled in, could something be trying to route across WireGuard that isn't listed in Allowed IPs?

z3us

@jimp Yes we go from one subnet to another and there's nothing being send to the WireGuard interfaces. On both sides I've entered the remote subnet as the allowed IP. We don't use other subnets than those who are entered on the allowed IP lists.

stafast

I'm actually having this exact same issue. I setup Wireguard to experiment from work to my house to see difference between IPSEC. My work router has been crashing much more than my home router, but they are both crashing with similar crash dump to this. I disabled the interface and WireGuard on both sides, will see if the crashes stop.

z3us

I've disabled WireGuard for now and activated OpenVPN. So far so good, no issues since 24 hours.....

bobwondernut

Same issue here on SG-5100s - two wireguard peers (three pfsense endpoints total), similar backtraces. Interface created for the wireguard shared subnet, and MSS clamped to 1420 on the WG interface (it was not clamping in the reverse direction when looking at SYN packet traces otherwise.)

Allowed IP of each peer set only to the remote CIDR. Peer WireGuard address set to the peer's IP in the WG subnet (which is set as a common /24 among all 3 endpoints.) Distinct PSKs per peer pair.