Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    IPsec and Tiered Gateway Groups

    IPsec
    2
    6
    2.1k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • B
      barnettd
      last edited by

      I have pfsense 2.3.4 on an SG-2440. It has 2 wan connections configured with basic tiered gateway. No load balancing, shaping, or CARP. There is an ipsec tunnel that connects to a remote Cisco ASA (ver 9.1) using the gw group as the local endpoint. The ipsec on the asa side is configed to "respond only" mode.

      Problem: The ipsec tunnel continues to use it's original local endpoint regardless of which one is "active". Stopping/starting ipsec appears to resolve issue.

      Scenario: Tier 1 ISP fails due to monitor IP not responding, pfsense marks gateway as down and traffic is routed through tier 2 ISP. Tunnel is eventually torn down due to lost service. When pfsense trys to rebuild tunnel, it continues to use the tier 1 (offline) isp. I have confirmed this in the logs.

      Workaround I'm using: Go to Status -> IPsec. Click stop service and wait until stopped. Click start service and wait until started. Click Connect VPN. VPN Successfully connects using the correct ISP. This is also needed when the tier 1 ISP comes back online. If service is not stopped/started, vpn continues to try to use incorrect gateway.

      This is a horrible workaround since it requires manual intervention when the ISP goes up/down. Can anyone give guidance on how to resolve? I'm open to any solutions.

      Thanks!

      1 Reply Last reply Reply Quote 0
      • T
        Tubal
        last edited by

        We have this same issue, only restarting IPSec doesn't solve the issue for us.

        We have to reboot the entire pfSense box in order for it to clear it's cached endpoints and try to use the new ones.

        I've even attempted create a new phase 1 connection with the new endpoints, and it still uses the end points cached on the existing phase 1 connection somehow.

        It's a real problem, particularly in HA environments where you don't want to shut down their entire network to restore a single VPN connection.

        1 Reply Last reply Reply Quote 0
        • B
          barnettd
          last edited by

          I have found that you may need to wait about 60 seconds after the service is restarted. I don't have a auto ping setup, so the tunnel doesn't try to reconnect automatically in my test environment, not sure if that plays a part.

          1 Reply Last reply Reply Quote 0
          • B
            barnettd
            last edited by

            Kind of surprised I haven't gotten more feedback on this, I thought this would have been a common configuration. Am I going about this the wrong way?

            1 Reply Last reply Reply Quote 0
            • B
              barnettd
              last edited by

              I was able to look into this a little more today. I believe the issue is related to the file /usr/local/etc/ipsec.conf not getting updated when the gateway is changed. I tested by watching the left ip address:

              Disabled the primary wan
                  Verified "left" still showed x.x.x.x. Tunnel does not work.
                  Restarted service via web page. Verified "left" now shows correct ip y.y.y.y and tunnel works.

              1 Reply Last reply Reply Quote 0
              • B
                barnettd
                last edited by

                After much troubleshooting on and off the last few weeks, I think I have this working "good enough". Our old ASA 5505s handle failing over slightly more reliably, but in the end the cost and flexibility of the pfsense devices seem to be worth it.

                I think one of the main issues was DNS resolution. The firewall was unable to update the dynamic dns service when it failed over to the backup wan. One way to resolve that is to configure "Enable default gateway switching" under advanced, but that seemed to cause issues with a static route I had setup to allow the firewall itself to reach the remote lan of the ipsec tunnel.

                My resolution was to:
                1. Make sure DNS servers are configured on all interfaces under System-> General Setup. (Why can't there be duplicates?)
                2. Under Services->DNS Resolver, check "enable forwarding mode"
                3. Set DPD to a lower retry rate to cause broken tunnel to be torn down quicker.
                4. Enabled Cisco Extensions under vpn advanced. Not sure if this helped anything.
                5. Installed cron and changed the rc.dyndns.update task to run every 3 minutes. This probably didn't fix anything, but it helped troubleshoot.

                Remaining issues:
                1. When the primary WAN comes back online, the tunnel never seems to renegotiate on that circuit automatically (maybe im not waiting long enough?). This isn't critical in my view.
                2. IPsec is currently using not very secure aggressive mode, more testing needed to check main mode.

                And maybe someday I'll be able to look into some sort of ipsec over gre connection that handles failures more gracefully, but that looks quite complex since I don't have much knowledge in the field.

                Hope this helps someone.

                1 Reply Last reply Reply Quote 0
                • First post
                  Last post
                Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.