I have my WAN Gateway and running OpenVPN for my other gateway.  At random, my internet kill-switch kicks in because OpenVPN is restarting.

May 15 19:25:09	rc.gateway_alarm	98632	>>> Gateway alarm: VPN_WAN_VPNV4 (Addr:REDACTED Alarm:1 RTT:31347ms RTTsd:5964ms Loss:21%)
May 15 19:25:09	check_reload_status		updating dyndns VPN_WAN_VPNV4
May 15 19:25:09	check_reload_status		Restarting ipsec tunnels
May 15 19:25:09	check_reload_status		Restarting OpenVPN tunnels/interfaces
May 15 19:25:09	check_reload_status		Reloading filter
May 15 19:25:10	php-fpm	243	/rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use VPN_WAN_VPNV4.

I thought it was a memory issue, I did turn up a few things to push my rig to see what I could get away with but even with things turned down (log retention, number of entries for pfblocker, things like that) it still keeps cycling itself.

I will say I only started having this issue in the past few weeks.  I am only running the pfBlocker package, but I do have some large lists.  I initially thought a cron job was causing a memory issue to make this go down however, I switched the cron jobs to once a day at 2am but still experienced the issue.

Either of you guys have similar setups that can correlate potential causes so we can start to work towards a solution?  Or smarter minds than my own are definitely in this community so I would love to hear back from someone that can tell me where my dumb mistake lies - I will glady wear egg on my face if it means network gains some stability.

Thanks!

Something triggers when check_reload_status is run that causes dpinger issues.

Not a biggy, but I too have noticed this.

Feb 6 20:48:20	rc.gateway_alarm	72290	>>> Gateway alarm: WAN_PPPOE (Addr:X.X.X.X Alarm:0 RTT:3906ms RTTsd:1128ms Loss:0%)
Feb 6 20:48:20	check_reload_status		updating dyndns WAN_PPPOE
Feb 6 20:48:20	check_reload_status		Restarting ipsec tunnels
Feb 6 20:48:20	check_reload_status		Restarting OpenVPN tunnels/interfaces
Feb 6 20:48:20	check_reload_status		Reloading filter
Feb 6 20:48:21	php-fpm		/rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_PPPOE.
Feb 6 21:46:58	php-fpm		/index.php: Successful login for user 'admin' from: 192.168.0.106
Feb 7 01:46:58	rc.gateway_alarm	95168	>>> Gateway alarm: WAN_PPPOE (Addr:X.X.X.X Alarm:1 RTT:134683ms RTTsd:486359ms Loss:0%)
Feb 7 01:46:58	check_reload_status		updating dyndns WAN_PPPOE
Feb 7 01:46:58	check_reload_status		Restarting ipsec tunnels
Feb 7 01:46:58	check_reload_status		Restarting OpenVPN tunnels/interfaces
Feb 7 01:46:58	check_reload_status		Reloading filter
Feb 7 01:46:59	php-fpm		/rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_PPPOE.
Feb 7 01:47:12	rc.gateway_alarm	97132	>>> Gateway alarm: WAN_PPPOE (Addr:X.X.X.X Alarm:1 RTT:219457ms RTTsd:556905ms Loss:11%)
Feb 7 01:47:12	check_reload_status		updating dyndns WAN_PPPOE
Feb 7 01:47:12	check_reload_status		Restarting ipsec tunnels
Feb 7 01:47:12	check_reload_status		Restarting OpenVPN tunnels/interfaces
Feb 7 01:47:12	check_reload_status		Reloading filter

Well, adding inactive directive and forcing restart on ping failure did not improve the situation.

inactive 120
remap-usr1 SIGHUP

To recap.  I run an OpenVPN client on pfSense which is meant to be always on.
Every random amount of time, depending on ISP whatever, the connection stops working.
OpenVPN itself rarely notices the problem, even with the settings above [in addition to the usual keepalive].
If I am de-authed and it exits, my watchdog catches it, but otherwise the OpenVPN logs don't show a problem.
Attempting to use the network shows packets not flowing.
dpinger always notices the problem, and fires the gateway alarm.
…which does nothing, hence my posting here.
Going back through lengthy logs shows dpinger always noticing, reporting, firing alarm.  And then nothing until manual intervention.

So, I made this patch:

--- /etc/rc.openvpn     2017-11-03 19:46:16.254944000 -0700
+++ /tmp/rc.openvpn     2017-11-03 19:49:20.564839000 -0700
@@ -40,17 +40,18 @@
        if (isset($ovpn_settings['disable'])) {
                $resync_needed = false;
        } else {
-               if (!empty($interface)) {
-                       $mode_id = $mode . $ovpn_settings['vpnid'];
-                       $fpath = "{$g['varetc_path']}/openvpn/{$mode_id}.interface";
-                       if (file_exists($fpath)) {
-                               /* Compare the interface currently used by the VPN with the interface that should be used.
-                                  If the VPN should stay on the same interface, do not resync */
-                               if (trim(file_get_contents($fpath), " \t\n") == get_failover_interface($ovpn_settings['interface'])) {
-                                       $resync_needed = false;
-                               }
-                       }
-               }
+# /* We want to resync if called and the interface is not disabled */
+#              if (!empty($interface)) {
+#                      $mode_id = $mode . $ovpn_settings['vpnid'];
+#                      $fpath = "{$g['varetc_path']}/openvpn/{$mode_id}.interface";
+#                      if (file_exists($fpath)) {
+#                              /* Compare the interface currently used by the VPN with the interface that should be used.
+#                                 If the VPN should stay on the same interface, do not resync */
+#                              if (trim(file_get_contents($fpath), " \t\n") == get_failover_interface($ovpn_settings['interface'])) {
+#                                      $resync_needed = false;
+#                              }
+#                      }
+#              }
        }
        if ($resync_needed == true) {
                log_error("OpenVPN: Resync " . $mode_id . " " . $ovpn_settings['description']);
@@ -116,9 +117,10 @@

        if (is_array($config['openvpn']['openvpn-client'])) {
                foreach ($config['openvpn']['openvpn-client'] as &$client) {
-                       if ($client['interface'] == $interface || empty($interface) || (!empty($gwgroups) && in_array($client['interface'], $gwgroups))) {
+#                      /* We want to resync if called; in our case, $client['interface'] != $interface but we still want to resync */
+#                      if ($client['interface'] == $interface || empty($interface) || (!empty($gwgroups) && in_array($client['interface'], $gwgroups))) {
                                openvpn_resync_if_needed('client', $client, $interface);
-                       }
+#                      }
                }
        }
 }

Apply with path strip of 0 and default root dir.  And of course do so at your own risk.
I have no idea what the logic in here which twice tries to prevent the client from actually resyncing is for.
This patch disables the "only resync if VPN interface isn't what is set" logic in openvpn_resync_if_needed() and removes the various interface/gwgroups tests just for the "client" case.
Again, I don't know what they're there for, but for my use-case, this change now means that for the first time, my OpenVPN client has become reliable.
When dpinger sees a problem, it restarts the client.

This may definitely result in momentary connection loss, but that is preferable to connection being down until manual intervention.

Again, I'd appreciate commentary from those more knowledgeable on why those tests are in there.  I think I'm requesting a supported way to disable them…

Cheers