PFsense 2.6. WG handshake stops and only way to make it work again is to restore the same pfsense config
-
Ok so this makes no sense to me. My pfsense has been running wireguard for over a year now without any problems. But once in a while WG clients cannot handshake with the pfsense wg server.
No config has changed on the pfsense. It has been rebooted several times before so there is no pending change witing to be applied etc. Yet once in a while (twice in a week this time) wg clients could not connect to the pf sense wg server.
Restarting the WG service, resetting states or even a Reboot of the pfsense doestn fix the issue. The last two this happened i had to restore an old config and that fixed it. However note that im running the same config right now! So before i go ahead and restore the config again, i would like to see if i can troubleshoot this.
Are there any wireguard related logs anywhere on the pfsense? This issue doesnt make any sense to me.
I even tried taking a network trace on WAN port for incoming traffic on the WG port but i see nothing which is odd! im pretty sure my tcpdump command is good because i can trace traffic for other ports just fine.
WG interface is up:
tun_wg0: flags=80c1<UP,RUNNING,NOARP,MULTICAST> metric 0 mtu 1420 description: WG_VPN options=80000<LINKSTATE> inet 10.0.2.1 netmask 0xffffff00 groups: wg WireGuard nd6 options=101<PERFORMNUD,NO_DAD>Log on pf sense show:
Jan 14 01:07:34 php-fpm 422 /wg/vpn_wg_settings.php: Configuration Change: vpnuser@10.0.1.2 (Local Database): [pfSense-pkg-WireGuard] Applied package default settings as necessary. Jan 14 01:06:33 php 56692 /usr/local/pkg/wireguard/includes/wg_service.inc: Gateway, none 'available' for inet6, use the first one configured. 'WAN_DHCP6' Jan 14 01:06:33 php 56692 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Applied package default settings as necessary. Jan 14 01:06:33 php 56692 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed Unbound ACL group (WireGuard). Jan 14 01:06:33 php 56692 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed Unbound ACL group (WireGuard). Jan 14 01:06:33 php 56692 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed interface group (WireGuard). Jan 14 01:06:33 php 56692 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed interface group (WireGuard). Jan 14 01:06:33 php 56692 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed earlyshellcmd(s). Jan 14 01:06:33 php 56692 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed earlyshellcmd(s). Jan 14 01:06:33 kernel tun_wg0: link state changed to UP Jan 14 01:06:33 kernel wg0: changing name to 'tun_wg0' Jan 14 01:06:33 php 56692 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Applied package default settings as necessary. Jan 14 01:06:33 php 56692 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed Unbound ACL group (WireGuard). Jan 14 01:06:33 php 56692 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed Unbound ACL group (WireGuard). Jan 14 01:06:33 php 56692 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed interface group (WireGuard). Jan 14 01:06:33 php 56692 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed interface group (WireGuard). Jan 14 01:06:33 php 56692 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed earlyshellcmd(s). Jan 14 01:06:32 php 56692 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed earlyshellcmd(s). Jan 14 01:05:46 kernel tun_wg0: link state changed to DOWN Jan 7 19:56:14 php-fpm 421 /wg/vpn_wg_tunnels.php: Successful login for user 'vpnuser' from: 10.0.2.3 (Local Database)All that up/down is me starting and stopping the WG service or resetting the state table on pfsense.
-
Update.
And just like that it started to work by itself. Now im thinking perhaps the issue is not on the pfsense.
There is an uplink Netgear router to which pfsense's WAN port is connected to. pfsense is in the DMZ of that netgear router.
So im thinking that fact that i was not seeing any WG incoming traffic on the pfsense when a WG client tried to connect is maybe because the netgear router was not forwarding the UDP traffic to pfsense?
Thats the only thing that would make sense. I just dont know for sure till i happens again and ill try rebooting that router instead. -
@alirz Your tcpdump or a packet capture in the pfSense UI is all the evidence you need. If you capture no incoming packets to the Wireguard port when clients cannot connect, then pfSense/wireguard service is definitively is not to blame. It cannot answer/connect to clients it does not know are trying to connect :-)
There’s probably some kind af packetfiltering active in the Netgear even though it is a DMZ setup.
When you have the issue, does a reboot of the Netgear temporarily solve it? -
@keyser Reboot of the netgear router will be the first thing i try the next time this issue occurs. I had never thought about the possibility of the issue being on the netgear router before, so ill be testing and verifying that next time.
