Openvpn link flapping and constant dropped connections since updating to rc0



  • I've been running the beta 2.1 builds since last year and have been updating about once a month.  I've never had an issue until I've updated to RC0.

    The main issue I'm facing is active connections are being dropped throughout the day.  For example IRC connections and video game connections get dropped and need to be reconnected.  Sometimes several hours go between these hiccups but sometimes several happen within minutes of each other.

    The system log shows check_reload_status running during these moments.  And it appears to be tied to openvpn.  I'm running two openvpn servers, one for site-to-site connections and the other for remote access.

    Here is the relevant portion of the system log.  this corresponds to when my clients had their connections dropped.

    Jun 2 10:07:32	check_reload_status: updating dyndns heNetGWGWv6,WAN_DHCP
    Jun 2 10:07:32	check_reload_status: Restarting ipsec tunnels
    Jun 2 10:07:32	check_reload_status: Restarting OpenVPN tunnels/interfaces
    Jun 2 10:07:32	check_reload_status: Reloading filter
    Jun 2 10:07:32	check_reload_status: updating dyndns WAN_DHCP,heNetGWGWv6
    Jun 2 10:07:32	check_reload_status: Restarting OpenVPN tunnels/interfaces
    Jun 2 10:07:34	php: : OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use heNetGWGWv6,WAN_DHCP.
    Jun 2 10:07:35	php: : OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP,heNetGWGWv6.
    Jun 2 10:08:18	check_reload_status: updating dyndns heNetGWGWv6,WAN_DHCP
    Jun 2 10:08:18	check_reload_status: Restarting ipsec tunnels
    Jun 2 10:08:18	check_reload_status: Restarting OpenVPN tunnels/interfaces
    Jun 2 10:08:18	check_reload_status: Reloading filter
    Jun 2 10:08:19	check_reload_status: updating dyndns WAN_DHCP,heNetGWGWv6
    Jun 2 10:08:19	check_reload_status: Restarting OpenVPN tunnels/interfaces
    Jun 2 10:08:21	php: : OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use heNetGWGWv6,WAN_DHCP.
    Jun 2 10:08:22	php: : OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP,heNetGWGWv6.
    Jun 2 10:09:22	check_reload_status: updating dyndns heNetGWGWv6,WAN_DHCP
    Jun 2 10:09:22	check_reload_status: Restarting ipsec tunnels
    Jun 2 10:09:22	check_reload_status: Restarting OpenVPN tunnels/interfaces
    Jun 2 10:09:22	check_reload_status: Reloading filter
    Jun 2 10:09:24	php: : OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use heNetGWGWv6,WAN_DHCP.
    Jun 2 10:09:28	check_reload_status: updating dyndns WAN_DHCP
    Jun 2 10:09:28	check_reload_status: Restarting OpenVPN tunnels/interfaces
    Jun 2 10:09:31	php: : phpDynDNS (all.dnsomatic.com): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Jun 2 10:09:31	php: : OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP.
    Jun 2 10:09:31	php: : OpenVPN: Resync server1 defServer
    Jun 2 10:09:31	kernel: ovpns1: link state changed to DOWN
    Jun 2 10:09:31	php: : OpenVPN: Resync server2 sitetosite
    Jun 2 10:09:31	kernel: ovpns1: link state changed to UP
    Jun 2 10:09:31	kernel: ovpns2: link state changed to DOWN
    Jun 2 10:09:31	check_reload_status: rc.newwanip starting ovpns1
    Jun 2 10:09:31	kernel: ovpns2: link state changed to UP
    Jun 2 10:09:32	check_reload_status: rc.newwanip starting ovpns2
    Jun 2 10:09:32	php: : phpDynDNS (77348): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Jun 2 10:09:33	php: : phpDynDNS (XXXXXXXX): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Jun 2 10:09:34	php: : rc.newwanip: Informational is starting ovpns1.
    Jun 2 10:09:34	php: : rc.newwanip: on (IP address: 10.13.0.1) (interface: ) (real interface: ovpns1).
    Jun 2 10:09:34	check_reload_status: Reloading filter
    Jun 2 10:09:34	php: : pfSense package system has detected an ip change -> 10.13.0.1 ... Restarting packages.
    Jun 2 10:09:34	check_reload_status: Starting packages
    Jun 2 10:09:34	php: : rc.newwanip: Informational is starting ovpns2.
    Jun 2 10:09:34	php: : rc.newwanip: on (IP address: 10.15.0.1) (interface: ) (real interface: ovpns2).
    Jun 2 10:09:34	php: : pfSense package system has detected an ip change -> 10.15.0.1 ... Restarting packages.
    Jun 2 10:09:36	php: : Restarting/Starting all packages.
    Jun 2 10:09:36	radiusd[80763]: Signalled to terminate
    Jun 2 10:09:36	radiusd[80763]: Exiting normally.
    Jun 2 10:09:37	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:09:39	check_reload_status: updating dyndns WAN_DHCP
    Jun 2 10:09:39	check_reload_status: Restarting ipsec tunnels
    Jun 2 10:09:39	check_reload_status: Restarting OpenVPN tunnels/interfaces
    Jun 2 10:09:40	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:09:40	check_reload_status: updating dyndns WAN_DHCP,heNetGWGWv6
    Jun 2 10:09:40	check_reload_status: Restarting OpenVPN tunnels/interfaces
    Jun 2 10:09:42	php: : OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP.
    Jun 2 10:09:42	php: : OpenVPN: Resync server1 defServer
    Jun 2 10:09:42	php: : phpDynDNS (all.dnsomatic.com): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Jun 2 10:09:42	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:09:42	kernel: ovpns1: link state changed to DOWN
    Jun 2 10:09:42	php: : OpenVPN: Resync server2 sitetosite
    Jun 2 10:09:42	kernel: ovpns1: link state changed to UP
    Jun 2 10:09:42	check_reload_status: rc.newwanip starting ovpns1
    Jun 2 10:09:42	kernel: ovpns2: link state changed to DOWN
    Jun 2 10:09:43	kernel: ovpns2: link state changed to UP
    Jun 2 10:09:43	check_reload_status: rc.newwanip starting ovpns2
    Jun 2 10:09:43	php: : OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP,heNetGWGWv6.
    Jun 2 10:09:43	php: : phpDynDNS (77348): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Jun 2 10:09:44	php: : phpDynDNS (XXXXXXXX): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Jun 2 10:09:44	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:09:45	php: : rc.newwanip: Informational is starting ovpns1.
    Jun 2 10:09:45	php: : rc.newwanip: on (IP address: 10.13.0.1) (interface: ) (real interface: ovpns1).
    Jun 2 10:09:45	check_reload_status: Reloading filter
    Jun 2 10:09:45	php: : pfSense package system has detected an ip change -> 10.13.0.1 ... Restarting packages.
    Jun 2 10:09:45	check_reload_status: Starting packages
    Jun 2 10:09:45	php: : rc.newwanip: Informational is starting ovpns2.
    Jun 2 10:09:45	php: : rc.newwanip: on (IP address: 10.15.0.1) (interface: ) (real interface: ovpns2).
    Jun 2 10:09:45	php: : pfSense package system has detected an ip change -> 10.15.0.1 ... Restarting packages.
    Jun 2 10:09:46	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:09:47	php: : Restarting/Starting all packages.
    Jun 2 10:09:48	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:09:48	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:09:49	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:09:50	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:09:51	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:09:52	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:09:53	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:09:54	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:09:55	radiusd[54944]: Loaded virtual server <default>
    Jun 2 10:09:55	radiusd[55088]: Ready to process requests.
    Jun 2 10:09:56	radiusd[55088]: Signalled to terminate
    Jun 2 10:09:56	radiusd[55088]: Exiting normally.
    Jun 2 10:09:56	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:09:58	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:09:59	upsmon[71055]: Signal 15: exiting
    Jun 2 10:09:59	upsd[70675]: User monuser@127.0.0.1 logged out from UPS [localups]
    Jun 2 10:09:59	upsd[70675]: mainloop: Interrupted system call
    Jun 2 10:09:59	upsd[70675]: Signal 15: exiting
    Jun 2 10:09:59	usbhid-ups[70443]: Signal 15: exiting
    Jun 2 10:10:00	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:10:02	check_reload_status: updating dyndns heNetGWGWv6,WAN_DHCP
    Jun 2 10:10:02	check_reload_status: Restarting ipsec tunnels
    Jun 2 10:10:02	check_reload_status: Restarting OpenVPN tunnels/interfaces
    Jun 2 10:10:02	check_reload_status: Reloading filter
    Jun 2 10:10:03	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:10:04	usbhid-ups[90795]: Startup successful
    Jun 2 10:10:04	upsd[90867]: listening on ::1 port 3493
    Jun 2 10:10:04	upsd[90867]: listening on 127.0.0.1 port 3493
    Jun 2 10:10:04	upsd[90867]: Connected to UPS [localups]: usbhid-ups-localups
    Jun 2 10:10:04	upsd[91207]: Startup successful
    Jun 2 10:10:04	upsmon[91518]: Startup successful
    Jun 2 10:10:04	upsd[91207]: User monuser@127.0.0.1 logged into UPS [localups]
    Jun 2 10:10:04	php: : OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use heNetGWGWv6,WAN_DHCP.
    Jun 2 10:10:05	radiusd[97711]: Loaded virtual server <default>
    Jun 2 10:10:05	radiusd[97960]: Ready to process requests.
    Jun 2 10:10:09	check_reload_status: updating dyndns WAN_DHCP,heNetGWGWv6
    Jun 2 10:10:09	check_reload_status: Restarting OpenVPN tunnels/interfaces
    Jun 2 10:10:09	upsmon[91546]: Signal 15: exiting
    Jun 2 10:10:09	upsd[91207]: User monuser@127.0.0.1 logged out from UPS [localups]
    Jun 2 10:10:09	upsd[91207]: mainloop: Interrupted system call
    Jun 2 10:10:09	upsd[91207]: Signal 15: exiting
    Jun 2 10:10:09	usbhid-ups[90795]: Signal 15: exiting
    Jun 2 10:10:11	php: : OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP,heNetGWGWv6.
    Jun 2 10:10:13	usbhid-ups[9755]: Startup successful
    Jun 2 10:10:13	upsd[9927]: listening on ::1 port 3493
    Jun 2 10:10:13	upsd[9927]: listening on 127.0.0.1 port 3493
    Jun 2 10:10:13	upsd[9927]: Connected to UPS [localups]: usbhid-ups-localups
    Jun 2 10:10:13	upsd[10208]: Startup successful
    Jun 2 10:10:13	upsmon[10638]: Startup successful
    Jun 2 10:10:13	upsd[10208]: User monuser@127.0.0.1 logged into UPS [localups]
    Jun 2 10:10:55	radiusd[97960]: Login OK: [XXXX] (from client local port 0)
    Jun 2 10:10:59	check_reload_status: updating dyndns heNetGWGWv6,WAN_DHCP
    Jun 2 10:10:59	check_reload_status: Restarting ipsec tunnels
    Jun 2 10:10:59	check_reload_status: Restarting OpenVPN tunnels/interfaces
    Jun 2 10:10:59	check_reload_status: Reloading filter
    Jun 2 10:10:59	check_reload_status: updating dyndns WAN_DHCP,heNetGWGWv6
    Jun 2 10:10:59	check_reload_status: Restarting OpenVPN tunnels/interfaces
    Jun 2 10:11:01	php: : OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use heNetGWGWv6,WAN_DHCP.
    Jun 2 10:11:01	php: : OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP,heNetGWGWv6.
    Jun 2 10:13:21	check_reload_status: updating dyndns heNetGWGWv6
    Jun 2 10:13:21	check_reload_status: Restarting ipsec tunnels
    Jun 2 10:13:21	check_reload_status: Restarting OpenVPN tunnels/interfaces
    Jun 2 10:13:21	check_reload_status: Reloading filter
    Jun 2 10:13:23	php: : OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use heNetGWGWv6.
    Jun 2 10:13:41	check_reload_status: updating dyndns heNetGWGWv6
    Jun 2 10:13:41	check_reload_status: Restarting ipsec tunnels
    Jun 2 10:13:41	check_reload_status: Restarting OpenVPN tunnels/interfaces
    Jun 2 10:13:41	check_reload_status: Reloading filter
    Jun 2 10:13:42	check_reload_status: updating dyndns WAN_DHCP
    Jun 2 10:13:42	check_reload_status: Restarting OpenVPN tunnels/interfaces
    Jun 2 10:13:43	php: : OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use heNetGWGWv6.
    Jun 2 10:13:44	php: : OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP.
    Jun 2 10:13:44	php: : OpenVPN: Resync server1 defServer
    Jun 2 10:13:44	php: : phpDynDNS (all.dnsomatic.com): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Jun 2 10:13:44	kernel: ovpns1: link state changed to DOWN
    Jun 2 10:13:44	php: : OpenVPN: Resync server2 sitetosite
    Jun 2 10:13:45	kernel: ovpns1: link state changed to UP
    Jun 2 10:13:45	kernel: in6_purgeaddr: link-local all-nodesmulticast address deletion error
    Jun 2 10:13:45	kernel: in6_purgeaddr: node-local all-nodesmulticast address deletion error
    Jun 2 10:13:45	kernel: ovpns2: link state changed to DOWN
    Jun 2 10:13:45	check_reload_status: rc.newwanip starting ovpns1
    Jun 2 10:13:45	kernel: pfr_unroute_kentry: delete failed.
    Jun 2 10:13:45	kernel: ovpns2: link state changed to UP
    Jun 2 10:13:45	check_reload_status: rc.newwanip starting ovpns2
    Jun 2 10:13:45	php: : phpDynDNS (77348): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Jun 2 10:13:46	php: : phpDynDNS (XXXXXXXX): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Jun 2 10:13:47	php: : rc.newwanip: Informational is starting ovpns1.
    Jun 2 10:13:47	php: : rc.newwanip: on (IP address: 10.13.0.1) (interface: ) (real interface: ovpns1).
    Jun 2 10:13:47	php: : pfSense package system has detected an ip change -> 10.13.0.1 ... Restarting packages.
    Jun 2 10:13:47	check_reload_status: Starting packages
    Jun 2 10:13:47	php: : rc.newwanip: Informational is starting ovpns2.
    Jun 2 10:13:47	php: : rc.newwanip: on (IP address: 10.15.0.1) (interface: ) (real interface: ovpns2).
    Jun 2 10:13:47	php: : pfSense package system has detected an ip change -> 10.15.0.1 ... Restarting packages.
    Jun 2 10:13:49	php: : Restarting/Starting all packages.
    Jun 2 10:13:49	radiusd[97960]: Signalled to terminate
    Jun 2 10:13:49	radiusd[97960]: Exiting normally.
    Jun 2 10:13:50	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:13:52	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:13:54	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:13:56	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:13:58	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:14:00	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:14:02	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:14:05	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:14:07	radiusd[85400]: Loaded virtual server <default>
    Jun 2 10:14:07	radiusd[85689]: Ready to process requests.
    Jun 2 10:14:11	upsmon[10824]: Signal 15: exiting
    Jun 2 10:14:11	upsd[10208]: User monuser@127.0.0.1 logged out from UPS [localups]
    Jun 2 10:14:11	upsd[10208]: mainloop: Interrupted system call
    Jun 2 10:14:11	upsd[10208]: Signal 15: exiting
    Jun 2 10:14:11	usbhid-ups[9755]: Signal 15: exiting
    Jun 2 10:14:15	usbhid-ups[93223]: Startup successful
    Jun 2 10:14:15	upsd[93250]: listening on ::1 port 3493
    Jun 2 10:14:15	upsd[93250]: listening on 127.0.0.1 port 3493
    Jun 2 10:14:15	upsd[93250]: Connected to UPS [localups]: usbhid-ups-localups
    Jun 2 10:14:15	upsd[93536]: Startup successful
    Jun 2 10:14:15	upsmon[94013]: Startup successful
    Jun 2 10:14:15	upsd[93536]: User monuser@127.0.0.1 logged into UPS [localups]
    Jun 2 10:14:52	check_reload_status: updating dyndns heNetGWGWv6
    Jun 2 10:14:52	check_reload_status: Restarting ipsec tunnels
    Jun 2 10:14:52	check_reload_status: Restarting OpenVPN tunnels/interfaces
    Jun 2 10:14:52	check_reload_status: Reloading filter
    Jun 2 10:14:54	php: : OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use heNetGWGWv6.
    Jun 2 10:14:56	radiusd[85689]: Login OK: [XXXXXX] (from client local port 0)
    Jun 2 10:15:18	check_reload_status: updating dyndns WAN_DHCP
    Jun 2 10:15:18	check_reload_status: Restarting ipsec tunnels
    Jun 2 10:15:18	check_reload_status: Restarting OpenVPN tunnels/interfaces
    Jun 2 10:15:18	check_reload_status: Reloading filter
    Jun 2 10:15:20	php: : OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP.
    Jun 2 10:15:20	php: : OpenVPN: Resync server1 defServer
    Jun 2 10:15:21	php: : phpDynDNS (all.dnsomatic.com): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Jun 2 10:15:21	kernel: in6_purgeaddr: link-local all-nodesmulticast address deletion error
    Jun 2 10:15:21	kernel: in6_purgeaddr: node-local all-nodesmulticast address deletion error
    Jun 2 10:15:21	kernel: ovpns1: link state changed to DOWN
    Jun 2 10:15:21	php: : OpenVPN: Resync server2 sitetosite
    Jun 2 10:15:21	kernel: ovpns1: link state changed to UP
    Jun 2 10:15:21	kernel: ovpns2: link state changed to DOWN
    Jun 2 10:15:21	check_reload_status: rc.newwanip starting ovpns1
    Jun 2 10:15:21	kernel: ovpns2: link state changed to UP
    Jun 2 10:15:21	check_reload_status: rc.newwanip starting ovpns2
    Jun 2 10:15:22	php: : phpDynDNS (77348): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Jun 2 10:15:23	php: : phpDynDNS (XXXXXXX): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
    Jun 2 10:15:23	php: : rc.newwanip: Informational is starting ovpns1.
    Jun 2 10:15:23	php: : rc.newwanip: on (IP address: 10.13.0.1) (interface: ) (real interface: ovpns1).
    Jun 2 10:15:23	php: : pfSense package system has detected an ip change -> 10.13.0.1 ... Restarting packages.
    Jun 2 10:15:23	check_reload_status: Starting packages
    Jun 2 10:15:24	php: : rc.newwanip: Informational is starting ovpns2.
    Jun 2 10:15:24	php: : rc.newwanip: on (IP address: 10.15.0.1) (interface: ) (real interface: ovpns2).
    Jun 2 10:15:24	php: : pfSense package system has detected an ip change -> 10.15.0.1 ... Restarting packages.
    Jun 2 10:15:26	php: : Restarting/Starting all packages.
    Jun 2 10:15:26	radiusd[85689]: Signalled to terminate
    Jun 2 10:15:26	radiusd[85689]: Exiting normally.
    Jun 2 10:15:26	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:15:28	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:15:30	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:15:31	check_reload_status: updating dyndns heNetGWGWv6
    Jun 2 10:15:31	check_reload_status: Restarting ipsec tunnels
    Jun 2 10:15:31	check_reload_status: Restarting OpenVPN tunnels/interfaces
    Jun 2 10:15:31	check_reload_status: Reloading filter
    Jun 2 10:15:32	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:15:33	php: : OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use heNetGWGWv6.
    Jun 2 10:15:35	radiusd[67634]: Loaded virtual server <default>
    Jun 2 10:15:35	radiusd[71056]: Ready to process requests.
    Jun 2 10:15:35	radiusd[71056]: Signalled to terminate
    Jun 2 10:15:35	radiusd[71056]: Exiting normally.
    Jun 2 10:15:35	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:15:37	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:15:39	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:15:41	php: : The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Jun 2 10:15:44	radiusd[94793]: Loaded virtual server <default>
    Jun 2 10:15:44	radiusd[94794]: Ready to process requests.
    Jun 2 10:15:47	upsmon[94257]: Signal 15: exiting
    Jun 2 10:15:47	upsd[93536]: User monuser@127.0.0.1 logged out from UPS [localups]
    Jun 2 10:15:47	upsd[93536]: mainloop: Interrupted system call
    Jun 2 10:15:47	upsd[93536]: Signal 15: exiting
    Jun 2 10:15:47	usbhid-ups[93223]: Signal 15: exiting
    Jun 2 10:15:52	usbhid-ups[26043]: Startup successful
    Jun 2 10:15:52	upsd[26127]: listening on ::1 port 3493
    Jun 2 10:15:52	upsd[26127]: listening on 127.0.0.1 port 3493
    Jun 2 10:15:52	upsd[26127]: Connected to UPS [localups]: usbhid-ups-localups
    Jun 2 10:15:52	upsd[26240]: Startup successful
    Jun 2 10:15:52	upsmon[26601]: Startup successful
    Jun 2 10:15:52	upsd[26240]: User monuser@127.0.0.1 logged into UPS [localups]
    Jun 2 10:16:32	radiusd[94794]: Login OK: [XXXXXX] (from client local port 0)</default></default></default></default></default>
    

    And here is the dmesg

    ovpns1: link state changed to DOWN
    ovpns1: link state changed to UP
    ovpns2: link state changed to DOWN
    ovpns2: link state changed to UP
    ovpns1: link state changed to DOWN
    ovpns1: link state changed to UP
    ovpns2: link state changed to DOWN
    ovpns2: link state changed to UP
    ovpns1: link state changed to DOWN
    ovpns1: link state changed to UP
    in6_purgeaddr: link-local all-nodesmulticast address deletion error
    in6_purgeaddr: node-local all-nodesmulticast address deletion error
    ovpns2: link state changed to DOWN
    pfr_unroute_kentry: delete failed.
    ovpns2: link state changed to UP
    in6_purgeaddr: link-local all-nodesmulticast address deletion error
    in6_purgeaddr: node-local all-nodesmulticast address deletion error
    ovpns1: link state changed to DOWN
    ovpns1: link state changed to UP
    ovpns2: link state changed to DOWN
    ovpns2: link state changed to UP
    pflog0: promiscuous mode enabled
    in6_purgeaddr: link-local all-nodesmulticast address deletion error
    in6_purgeaddr: node-local all-nodesmulticast address deletion error
    ovpns1: link state changed to DOWN
    ovpns1: link state changed to UP
    ovpns2: link state changed to DOWN
    ovpns2: link state changed to UP
    in6_purgeaddr: link-local all-nodesmulticast address deletion error
    in6_purgeaddr: node-local all-nodesmulticast address deletion error
    ovpns1: link state changed to DOWN
    ovpns1: link state changed to UP
    ovpns2: link state changed to DOWN
    ovpns2: link state changed to UP
    

    Unfortunately the dmesg doesn't have timestamps, so I cannot say how close together these link state changes are.

    To rule out errant configuration issues.  Yesterday I reformatted my router and reinstalled a fresh copy of RC0 from May 31st.  I reconfigured everything by hand (no config restore) and yet I'm still having the issue.



  • OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use heNetGWGWv6,WAN_DHCP.
    

    That message means that the gateways heNetGWGWv6 and WAN_DHCP have gone away/come back. In this case, heNetGWGWv6 is probably running over WAN_DHCP. So when WAN_DHCP goes AWOL, so does heNetGWGWv6. Even if the IP address that WAN receives remains the same, any OpenVPN links that use that gateway need to be restarted - OpenVPN does not like the IP address it is bound to, to go away and come back.
    I suspect that some "event" is happening on WAN to set this off. The question is, what?



  • Have you had any luck with this?

    I believe I am experiencing a very similar if not identical problem and can't seem to resolve it.



  • @abeauchamp:

    Have you had any luck with this?

    I believe I am experiencing a very similar if not identical problem and can't seem to resolve it.

    Sadly no.  I've no idea what's causing it.  Although after reinstalling it I feel the number of occurrences has decreased.  Perhaps this weekend I'll have time to look more deeply but I'm not familiar enough with freebsd or networking in general to get far.

    Worst case, I have a full backup from last Feb saved.  I may just restore that and wait for RC1 or even RC2.



  • If you go into Status > Systems Logs > System / Gateways are you getting a lot of:

    Jun 3 17:44:49 apinger: ALARM: WANGW(X.X.X.X) *** delay ***
    Jun 3 17:44:58 apinger: alarm canceled: WANGW(X.X.X.X) *** delay ***
    Jun 3 17:46:58 apinger: ALARM: WANGW(X.X.X.X) *** delay ***
    Jun 3 17:47:13 apinger: alarm canceled: WANGW(9X.X.X.X) *** delay ***

    I was seeing a bunch of these corresponding with my OpenVPN connection drops and the check_reload_status messages appearing and the drop connections.

    I went into System > Routing > Gateways and adjusted the Frequency Probe from the default 1 second to 10 seconds and Down from the default of 10 seconds to 30 seconds and I haven't seen my issue of dropped connections and OpenVPN flapping for almost 24 hours now.

    I will keep you updated to see if anything changes.



  • Yes I do see a bunch of those.  I'm not sure what they mean however.

    If the frequency of dropped connections increases again, I'll try increasing the values you suggested.



  • http://lists.pfsense.org/pipermail/list/2012-September/002934.html

    Take a look at this link it might help explain it a bit better.  You can even disable gateway monitoring if your are not using failover.