Problem with Automatic Filter Reload when OpenVPN is in a Gateway Group



  • Hello,

    I currently have a Gateway Group consisting of two gateways in failover mode:

    1. OpenVPN (tier 1)
    2. OPT1 (tier 2)

    My two physical uplinks are on WAN and OPT1. The OpenVPN client uses WAN as its uplink. The WAN link is my primary uplink but is a public network, so I tunnel traffic going out this interface through an OpenVPN server. OPT1 is over an LTE network and is bandwidth constrained.

    I have set up rules so that all LAN traffic is sent through the GW Group, along with proper NAT rules. Overall its working well.

    This is fairly new set up and I ran into an interesting problem:

    1. In a normal scenario, both the OpenVPN and OPT1 Gateways are active and LAN traffic is routed through OpenVPN (as expected)
    2. If my OpenVPN server goes down, the OpenVPN gateway also goes down and LAN traffic is shifted to OPT1 (as expected)
    3. Here's the weird part: When the OpenVPN server comes back online, the OpenVPN Gateway comes back on as well. However, LAN traffic will continue to route through OPT1
    4. If I manually initiate a Filter Reload the problem is resolved and LAN traffic correctly returns to be routed through OpenVPN

    I dug into the System Log and noticed that an automatic filter reload occurs if a physical gateway goes up or down (WAN or OPT1), and also occurs when the OpenVPN link goes down, but a reload does not occur if the OpenVPN link comes back up. Please see the images below.

    Digging into the firewall rules (using command "pfctl -sr -v"), I followed how this rule changes in each of the scenarios above (the number corresponding to the scenario above):

    1. pass in quick on mvneta0.4091 route-to (ovpnc1 172.19.17.2) inet from 172.19.26.0/24 to any flags S/SA keep state label "USER_RULE: LAN to any rule"
    2. pass in quick on mvneta0.4091 route-to (mvneta0.4092 172.19.25.1) inet from 172.19.26.0/24 to any flags S/SA keep state label "USER_RULE: LAN to any rule"
    3. pass in quick on mvneta0.4091 route-to (mvneta0.4092 172.19.25.1) inet from 172.19.26.0/24 to any flags S/SA keep state label "USER_RULE: LAN to any rule"
    4. pass in quick on mvneta0.4091 route-to (ovpnc1 172.19.17.2) inet from 172.19.26.0/24 to any flags S/SA keep state label "USER_RULE: LAN to any rule"

    Similarly, looking at /tmp/rules.debug in the various scenarios:

    1. GWGateway_Failover = " route-to { ( ovpnc1 172.19.17.2 ) } "
    2. GWGateway_Failover = " route-to { ( mvneta0.4092 172.19.25.1 ) } "
    3. GWGateway_Failover = " route-to { ( mvneta0.4092 172.19.25.1 ) } "
    4. GWGateway_Failover = " route-to { ( ovpnc1 172.19.17.2 ) } "

    As you can likely assume, my LAN subnet is 172.19.26.0/24, gateway IP for OPT1 is 172.19.25.1 and OpenVPN is 172.19.17.2.

    So it looks like pfSense not initiating a filter reloading when the OpenVPN gateway comes back online.

    Has anyone run into a similar issue? Is there a way I can configure pfSense to auto reload filters when OpenVPN comes back online?

    Note that I have also tried enabling State Killing in the advanced menu, which didn't help.

    Thanks a lot,
    AP


    System Log when a physical link goes down and up (OPT1). Notice the "Reloading filter" at 10:37:23, when the link returns online:
    opt1.png


    System Log when OpenVPN link goes down and up. Notice the lack of "Reloading filter" once the link comes back up:
    ovpn.png



  • Just as an update, I was able to create a pretty simple work-around for this issue.

    I added the following directive to the end of this file: /usr/local/sbin/ovpn-linkup (which runs once the OpenVPN client link comes online)

    (/bin/sleep 120 && /etc/rc.filter_configure)&
    

    This basically waits 2 mins after the OpenVPN link comes up (enough time for the newwanip stuff to finish) and forces a filter reload.

    Funny enough, the last line of the file is already a command to reload the filters, but I guess there's something weird going on with it.

    Marking as resolved.



  • I encountered a similar problem. A policy rule referencing the vpn gateway was not getting created at start up.
    I first tried specifying an IP rather than hostname for vpn server in the client config (in case lack of DNS delayed VPN coming up). That did not help. I then tried the fix above and thatseems to have worked.
    Since pfSense does not create policy rules if the gateway is not up, it's strange very few users seem to have experienced the issue. I imagined many would have policy routing rules that depend on a VPN gateway.
    I prefer to avoid modifying system scripts. Is there another approach to fixing this?
    Thanks.


Log in to reply