Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Secondary WAN interface routes through primary WAN?? & IPsec no traffic

    Routing and Multi WAN
    2
    8
    1.3k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      SLAM-ER
      last edited by

      Hi, I have just replaced a single PFsense firewall with dual pfsense firewalls in CARP failover.

      The basic configuration is as follows:
      2x WAN connections (Primary and backup links) on different routers on different IPs & subnets, each connected to both firewalls via switches, VIPs setup for both plus VIP for the CARP LAN address that the LAN clients use as the gateway.

      Everything is working except for 2x things:

      1. Gateway Group issues:
      When the primary WAN link (tier 1 in an gateway group) goes down, it takes the secondary (tier 2 in the same group) WAN2 down with it.  If I use the pfsense traceroute tool it seems to be routing through the primary WAN link when it should go directly out WAN2?

      2.  IPsec tunnel issue
      One of our VPN tunnels wont work, it connects and the phase2 entries all come up as well, but I cant seem to get any traffic over these tunnels.  I have another IPsec tunnel to a different client and this one works fine (after changing the gateway to the VIP of the primary WAN connection).

      I am no firewall/networking expert, and I have been through all the docs and online posts I could find and have tried a lot of things, but I am at my wits end now.  Any help would be appreciated.

      Let me know what other information I might need to share to enable you guys to help me out.  :)

      Thanks
      Matthew

      1 Reply Last reply Reply Quote 0
      • DerelictD
        Derelict LAYER 8 Netgate
        last edited by

        So we have:

        1. HA
        2. Policy Routing
        3. IPsec

        Sounds like your policy routing is probably not right.

        You should probably post your LAN rules.

        Chattanooga, Tennessee, USA
        A comprehensive network diagram is worth 10,000 words and 15 conference calls.
        DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
        Do Not Chat For Help! NO_WAN_EGRESS(TM)

        1 Reply Last reply Reply Quote 0
        • S
          SLAM-ER
          last edited by

          Thanks for the reply!

          I don't think I am using policy routing, the WAN's are only for failover.  All traffic is sent to the gateway group "Failover", with all traffic going out the primary WAN link under normal circumstances.  The secondary WAN2 link is only used if the primary link goes down.

          My only LAN firewall rule is:
          IPv4 * LAN net * * * Failover none Default allow LAN to any rule

          The exception to this is the IPSec tunnels, which are secured by IP of the primary WAN connection (x.x.x0.210) so they are set to use the virtual IP of the primary WAN interface.  If the primary link goes down then so do the VPNs, but this is OK - they don't need to work when running on backup link.

          I have attached a diagram of my layout (excluding VPNs).

          Thanks
          Matthew

          firewalls.png
          firewalls.png_thumb

          1 Reply Last reply Reply Quote 0
          • DerelictD
            Derelict LAYER 8 Netgate
            last edited by

            Routing to the Failover group is policy routing.

            Traffic from LAN will always go out the failover group in that case as it is being explicitly instructed to do. It will not go out IPsec. It will not route to any of your other interfaces. It will not follow the routing table at all. It will go out Failover.

            You need to bypass policy routing for other-than-WAN traffic.

            https://doc.pfsense.org/index.php/Bypassing_Policy_Routing

            Chattanooga, Tennessee, USA
            A comprehensive network diagram is worth 10,000 words and 15 conference calls.
            DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
            Do Not Chat For Help! NO_WAN_EGRESS(TM)

            1 Reply Last reply Reply Quote 0
            • S
              SLAM-ER
              last edited by

              I added the rules for the VPNs (to use WAN GW) but it made no difference, also the other VPN is working without any rules.

              Also doesn't explain why the primary link going down takes down the secondary as well…  (taking the secondary down behaves normally).

              Thanks
              Matthew

              1 Reply Last reply Reply Quote 0
              • DerelictD
                Derelict LAYER 8 Netgate
                last edited by

                Going to have to provide more information. This stuff really works if you do it right. No way to know what you've done wrong without more info.

                I added the rules for the VPNs (to use WAN GW)

                I don't know what you're trying to say there.

                Provide specifics. Instead of "the VPNs" use subnets, etc. Not a bad diagram but it does not include the VPNs you're talking about.

                Chattanooga, Tennessee, USA
                A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                Do Not Chat For Help! NO_WAN_EGRESS(TM)

                1 Reply Last reply Reply Quote 0
                • S
                  SLAM-ER
                  last edited by

                  OK, I'm not sure what I did as I have been tinkering for hours, but it is now working with the bypass policy routing rules.  I dont understand why some of the VPN tunnels were working fine without these rules, but the one I was having trouble with is now working.

                  Thank you so much for taking the time to help me.  I will go onsite tomorrow to see if the gateway group issue (primary link failure causing secondary link failure also), as I can't test it remotely.

                  Thanks again
                  Matthew

                  1 Reply Last reply Reply Quote 0
                  • S
                    SLAM-ER
                    last edited by

                    FYI I also fixed the failover, it turns out when importing the config from the old firewall, some of the Virtual IP's got assigned to the wrong interface, which I think is why it was failing both when the primary went down.

                    Reading the manual and understanding the basic theory is nothing like being thrown in the deep end with a real-world deployment, so I have learned a lot over the last 2 days.  Thanks again for your help.

                    1 Reply Last reply Reply Quote 0
                    • First post
                      Last post
                    Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.