Secondary WAN interface routes through primary WAN?? & IPsec no traffic



  • Hi, I have just replaced a single PFsense firewall with dual pfsense firewalls in CARP failover.

    The basic configuration is as follows:
    2x WAN connections (Primary and backup links) on different routers on different IPs & subnets, each connected to both firewalls via switches, VIPs setup for both plus VIP for the CARP LAN address that the LAN clients use as the gateway.

    Everything is working except for 2x things:

    1. Gateway Group issues:
    When the primary WAN link (tier 1 in an gateway group) goes down, it takes the secondary (tier 2 in the same group) WAN2 down with it.  If I use the pfsense traceroute tool it seems to be routing through the primary WAN link when it should go directly out WAN2?

    2.  IPsec tunnel issue
    One of our VPN tunnels wont work, it connects and the phase2 entries all come up as well, but I cant seem to get any traffic over these tunnels.  I have another IPsec tunnel to a different client and this one works fine (after changing the gateway to the VIP of the primary WAN connection).

    I am no firewall/networking expert, and I have been through all the docs and online posts I could find and have tried a lot of things, but I am at my wits end now.  Any help would be appreciated.

    Let me know what other information I might need to share to enable you guys to help me out.  :)

    Thanks
    Matthew


  • LAYER 8 Netgate

    So we have:

    1. HA
    2. Policy Routing
    3. IPsec

    Sounds like your policy routing is probably not right.

    You should probably post your LAN rules.



  • Thanks for the reply!

    I don't think I am using policy routing, the WAN's are only for failover.  All traffic is sent to the gateway group "Failover", with all traffic going out the primary WAN link under normal circumstances.  The secondary WAN2 link is only used if the primary link goes down.

    My only LAN firewall rule is:
    IPv4 * LAN net * * * Failover none Default allow LAN to any rule

    The exception to this is the IPSec tunnels, which are secured by IP of the primary WAN connection (x.x.x0.210) so they are set to use the virtual IP of the primary WAN interface.  If the primary link goes down then so do the VPNs, but this is OK - they don't need to work when running on backup link.

    I have attached a diagram of my layout (excluding VPNs).

    Thanks
    Matthew



  • LAYER 8 Netgate

    Routing to the Failover group is policy routing.

    Traffic from LAN will always go out the failover group in that case as it is being explicitly instructed to do. It will not go out IPsec. It will not route to any of your other interfaces. It will not follow the routing table at all. It will go out Failover.

    You need to bypass policy routing for other-than-WAN traffic.

    https://doc.pfsense.org/index.php/Bypassing_Policy_Routing



  • I added the rules for the VPNs (to use WAN GW) but it made no difference, also the other VPN is working without any rules.

    Also doesn't explain why the primary link going down takes down the secondary as well…  (taking the secondary down behaves normally).

    Thanks
    Matthew


  • LAYER 8 Netgate

    Going to have to provide more information. This stuff really works if you do it right. No way to know what you've done wrong without more info.

    I added the rules for the VPNs (to use WAN GW)

    I don't know what you're trying to say there.

    Provide specifics. Instead of "the VPNs" use subnets, etc. Not a bad diagram but it does not include the VPNs you're talking about.



  • OK, I'm not sure what I did as I have been tinkering for hours, but it is now working with the bypass policy routing rules.  I dont understand why some of the VPN tunnels were working fine without these rules, but the one I was having trouble with is now working.

    Thank you so much for taking the time to help me.  I will go onsite tomorrow to see if the gateway group issue (primary link failure causing secondary link failure also), as I can't test it remotely.

    Thanks again
    Matthew



  • FYI I also fixed the failover, it turns out when importing the config from the old firewall, some of the Virtual IP's got assigned to the wrong interface, which I think is why it was failing both when the primary went down.

    Reading the manual and understanding the basic theory is nothing like being thrown in the deep end with a real-world deployment, so I have learned a lot over the last 2 days.  Thanks again for your help.


Log in to reply