Wireguard fails after reboot (2.8.0)
-
@Buffalo0207 The problem seems to be remarkably similar to one discussed in 2023 Wireguard Site-to-Site Gateways disabled after reboot - service not starting
-
@stephenw10 Can you tell me exactly where the main log is and I will download it for you. I also have WinSCP, so can access it from there if this is easier.
-
@Misterb Thanks for this. I will add a cron job to automatically install the WG package.
-
It should appear in the main system log in Status > System Logs > System > General. For example:
-
@stephenw10 This is immediately after I rebooted without reinstalling WG. Let me know if you need another one after WG has been reinstalled.
-
So what new logs appear there if you try to start WG when it fails?
-
@stephenw10 This is immediately after I tried restarting WG (and fails) from the UI Service Status page, without restarting dpinger.
-
Hmm, that was some time it booted?
It's triggering a bunch of rc.bootup events which seems odd. Perhaps you have something preventing it ever reaching 'bootup complete'.
Do you see 'bootup complete' in the system logs after booting?
Do you see /var/run/booting still present?
-
@stephenw10 Can you tell me exactly where I can see this information?
-
Bootup Complete should appear in the main system log like:
You can check the boot file from the CLI like:
[2.8.0-RELEASE][admin@t70.stevew.lan]/root: ls /var/run charon.ctl ld-elf.so.hints charon.pid ld-elf32.so.hints charon.vici log charon.wlst logpriv check_reload_status miniupnpd.pid cron.pid nginx-webConfigurator.pid daemon-charon.pid ntpd.pid daemon_sshguard.pid openvpn_server1.pid devd.pid pfSense_version devd.pipe pfSense_version.rc devd.seqpacket.pipe php-fpm.pid dhclient.igb0.pid php-fpm.socket dmesg.boot ping_hosts.pid dnsbl.pid powerd.pid dnsmasq.pid sshd.pid dpinger_VTI0_VTIV4~10.66.11.1~10.66.11.2.pid sshguard.pid dpinger_VTI0_VTIV4~10.66.11.1~10.66.11.2.sock suricata-ctrl-socket-60565 dpinger_WAN_DHCP~172.21.16.239~172.21.16.1.pid suricata_igb160565.pid dpinger_WAN_DHCP~172.21.16.239~172.21.16.1.sock syslog.pid expire_accounts.pid unbound.pid filter_reload_status update_alias_url_data.pid filterdns.pid updaterrd.sh.pid filterlog.pid utmp ipsec_keepalive.pid utx.active -
@stephenw10 Sorry for the delayed reply - I have just got back from a business trip. Anyway, this is the output from the CLI
[2.8.1-RC][root@pfSense.mymain.local]/root: ls /var/run
check_reload_status
cron.pid
daemon_sshguard.pid
devd.pid
devd.pipe
devd.seqpacket.pipe
dhclient.igb0.pid
dmesg.boot
dnsbl.pid
dpinger_VPNUNLIMITED_L2TP~10.240.0.2~10.240.0.1.pid
dpinger_VPNUNLIMITED_L2TP~10.240.0.2~10.240.0.1.sock
dpinger_WANV6_TUNNELV6~2001:470:1f08:84a::2~2001:470:1f08:84a::1.pid
dpinger_WANV6_TUNNELV6~2001:470:1f08:84a::2~2001:470:1f08:84a::1.sock
dpinger_WAN_DHCP~82.13.203.142~82.13.202.1.pid
dpinger_WAN_DHCP~82.13.203.142~82.13.202.1.sock
dpinger_wg1GW~10.102.1.114~10.102.1.114.pid
dpinger_wg1GW~10.102.1.114~10.102.1.114.sock
dpinger_wg2GW~10.102.100.206~10.102.100.206.pid
dpinger_wg2GW~10.102.100.206~10.102.100.206.sock
expire_accounts.pid
filter_reload_status
filterlog.pid
ipsec_keepalive.pid
kea
kea2fib6.cache
kea4-ctrl-socket
kea4-ctrl-socket.lock
kea6-ctrl-socket
kea6-ctrl-socket.lock
l2tp_opt9.pid
ld-elf.so.hints
ld-elf32.so.hints
log
logpriv
mdns-bridge.pid
miniupnpd.pid
nginx-webConfigurator.pid
ntpd.pid
pfSense_version
pfSense_version.rc
php-fpm.pid
php-fpm.socket
ping_hosts.pid
radvd.pid
sshd.pid
sshguard.pid
syslog.pid
unbound.pid
update_alias_url_data.pid
updaterrd.sh.pid
utmp
utx.active
wireguardd.pid
[2.8.1-RC][root@pfSense.mymain.local]/root: [2.8.1-RC][root@pfSense.mymain.local]/root: ls /var/run
kea4-ctrl-socket.lock
kea6-ctrl-socket
kea6-ctrl-socket.lock
l2tp_opt9.pid
[2.8.1-RC][root@pfSense.mymain.local]/root:: Too many arguments.
[2.8.1-RC][root@pfSense.mymain.local]/root: check_reload_status
ld-elf.so.hints
ld-elf32.so.hints
log
logpriv
mdns-bridge.pid
miniupnpd.pid
nginx-webConfigurator.pid
ntpd.pid
pfSense_version
pfSense_version.rc
php-fpm.pid
php-fpm.socket
ping_hosts.pid
radvd.pid
sshd.pid
sshguard.pid
syslog.pid
unbound.pid
update_alias_url_data.pid
updaterrd.sh.pid
utmp
utx.active
wireguardd.pid -
OK so the file is not present which indicates it has completed boot.
Do you see the 'bootup complete' line in the system logs?
-
-
This post is deleted! -
@stephenw10 I decided to do a fresh install of pfsense 2.8.1, and restored my original config.xml. The PHP_errors have now completely disappeared. However, the wireguard package still does not start at bootup.
I installed the shellcmd package and noticed that wireguard has an earlyshellcmd at bootup. I think it starts, but then immediately stops as I noticed that both my wireguard gateways that I have setup on tun_wg1 and tun_wg2 in System>Routing are 'stopped' (greyed out) after a reboot - I have to manually start them both and then start Wireguard on the opening GUI page.
I have uninstalled wireguard and reinstalled it, but the issue still persists.
Could there be an issue in my config.xml file? I am happy to send you the relevant parts of the xml file if you tell me what to send.
Otherwise, in the meantime, would you know the line of code that i can use in shellcmd to autommatically restart the two wireguard gateway interfaces so I don't have to manually start them on every reboot?
-
It feels more like some sort of race condition at boot. Like Wireguard tries to start on an interface that hasn't been setup yet, maybe a VPN or other tunnel type. That GIF tunnel perhaps? Can you try disabling that as a test?
-
This post is deleted! -
@stephenw10 I think we are getting somewhere and maybe to the route of the issue.
I disabled the GIF tunnel (Hurricane Electric) as you suggested, rebooted, and the wireguard gateways and wireguard service status were stopped. However, when i started the GIF tunnel again, the wireguard gateways and wireguard service status automatically started up.
There is a secondary issue however - once the wireguard service starts, the GIF Tunnel changes to 'Offline, Packetloss'. If I then restart the GIF tunnel, it remains 'Offline, Packetloss'.
There definitely seems to be a link between the wireguard service and the GIF tunnel.
-
Hmm, is there some IP/subnet conflict between the GIF and Wireguard tunnels?
-
@stephenw10 Not that I can see. The GIF tunnel (Hurricane Electric) is using 216.66.80.26 and the two wireguard tunnels use 10.102.1.114 and 10.102.100.206.
