Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    IPsec site to site dropping every 49-55 minutes

    IPsec
    4
    44
    1.0k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T
      TheStormsOfFury @TheStormsOfFury
      last edited by TheStormsOfFury

      @andrew_cb well, I have the rules as this one setup above, and it still timed out. Maybe I just need to set P2 to timeout at like 24 hours and then apply it and reboot at like 6:00 AM that way it will take it months to timeout during the working period.

      It just seems odd that I have "Make before break" enabled, yet it isn't doing that.

      Thinking about it, I have it set to WAN SUBNETS as the source as this uses CARP and VIP Ip address. Should I change it to just the "IP address" that is the VIP IP?

      T 1 Reply Last reply Reply Quote 0
      • T
        TheStormsOfFury @TheStormsOfFury
        last edited by TheStormsOfFury

        @viragomann @andrew_cb

        So i have my firewall rule to block on the WAN VIP address to see if it will stop that 45-50 second down time.

        I also performed a test and setup my tunnel times as follows:

        Phase 1:

        • Lifetime 604,800 (7 days/168 hours)
        • ReKey 604,200 (10 minutes less than lifetime)
        • ReAuth 160 (2 minutes) (This might need to be 604,640)
        • Rand Time 60 (1 minute)

        Phase 2:

        • 86,400 (1 day/24 hours)
        • 85,800 (10 minutes less than lifetime)
        • 60 (1 minute)

        The results were disastrously. The tunnels were resting every 45-50 seconds and it was creating multiple connection visible in the ipsec status.

        I reset everything back to the original mentioned above, but we're still timing out at the 55-ish minute mark, which is not good during business hours as phone calls and app connections are failing.

        I do have a question. If i am using make before break, do i still need "Phase One Child SA Close Action" to be restart? Shouldn't it be close?

        Any thoughts? I'm about ready to move on and try wireguard.

        1 Reply Last reply Reply Quote 0
        • T
          TheStormsOfFury
          last edited by

          I did a little checking and the tunnel is going down every 53 minutes 30 seconds (give or take a second).

          Then it is down for approximately 38 - 40 seconds.

          This is super strange that I have make before break enabled, but it is still doing this.

          Thanks in advance!

          TSoF

          1 Reply Last reply Reply Quote 0
          • M
            michmoor LAYER 8 Rebel Alliance @TheStormsOfFury
            last edited by michmoor

            @TheStormsOfFury said in IPsec site to site dropping every 49-55 minutes:

            We are running CARP and both sites are running identical hardware (Dell R620) with pfSense 24.11-RELEASE.

            A couple of things.

            1. Are you using the WAN CARP address to establish your IPsec tunnels?
            2. For your IPsec tunnels. Do you have gateway monitoring enabled?
            3. Have you enabled DPD ?

            Firewall: NetGate,Palo Alto-VM,Juniper SRX
            Routing: Juniper, Arista, Cisco
            Switching: Juniper, Arista, Cisco
            Wireless: Unifi, Aruba IAP
            JNCIP,CCNP Enterprise

            T 2 Replies Last reply Reply Quote 0
            • T
              TheStormsOfFury @michmoor
              last edited by

              @michmoor Thanks for the reply!

              1.) I am using the shared Virtual IP address so that way if RTR1 goes down RTR2 can provide the services.

              2.) I need to find that feature. inside Phase One there is a "Gateway Duplicate" that is disabled. Do you know where the "gateway monitoring enabled" is by any chance?

              3.) DPD is enabled in Phase One on both side of the tunnel.

              Thanks again!

              TSoF

              M 1 Reply Last reply Reply Quote 0
              • M
                michmoor LAYER 8 Rebel Alliance @TheStormsOfFury
                last edited by

                @TheStormsOfFury said in IPsec site to site dropping every 49-55 minutes:

                2.) I need to find that feature. inside Phase One there is a "Gateway Duplicate" that is disabled. Do you know where the "gateway monitoring enabled" is by any chance?

                System /Routing / Gateways

                b66c650d-7e6b-46fb-830c-2152db263611-image.png

                Firewall: NetGate,Palo Alto-VM,Juniper SRX
                Routing: Juniper, Arista, Cisco
                Switching: Juniper, Arista, Cisco
                Wireless: Unifi, Aruba IAP
                JNCIP,CCNP Enterprise

                1 Reply Last reply Reply Quote 0
                • T
                  TheStormsOfFury @michmoor
                  last edited by TheStormsOfFury

                  @michmoor I found the Gateway monitoring setting in: System > Routing > Gateways > Edit

                  Gateway Monitoring - Disable Gateway Monitoring
                  This will consider this gateway as always being up.

                  Is this what you are talking about? Reading it, without it being checked gateway monitoring is enabled.

                  I went ahead and disabled the "Gateway Monitoring" and "Gateway Action."

                  Thanks again,

                  TSoF

                  M 1 Reply Last reply Reply Quote 0
                  • M
                    michmoor LAYER 8 Rebel Alliance @TheStormsOfFury
                    last edited by

                    @TheStormsOfFury
                    I care more about Gateway Action. If that's set, lets unselect for now. Im assuming you are NOT in a multi-wan configuration

                    What that does is that if there is an issue with your gateway monitor IP (packet loss, jitter) this will bring down the IPsec connection in your case.

                    Firewall: NetGate,Palo Alto-VM,Juniper SRX
                    Routing: Juniper, Arista, Cisco
                    Switching: Juniper, Arista, Cisco
                    Wireless: Unifi, Aruba IAP
                    JNCIP,CCNP Enterprise

                    T 1 Reply Last reply Reply Quote 0
                    • T
                      TheStormsOfFury @michmoor
                      last edited by

                      @michmoor said in IPsec site to site dropping every 49-55 minutes:

                      @TheStormsOfFury
                      I care more about Gateway Action. If that's set, lets unselect for now. Im assuming you are NOT in a multi-wan configuration

                      What that does is that if there is an issue with your gateway monitor IP (packet loss, jitter) this will bring down the IPsec connection in your case.

                      Correct, we are in a single WAN configuration. They were both "enabled" or "un-checked" and i went ahead and "checked" them on both sites

                      TSoF

                      M 1 Reply Last reply Reply Quote 0
                      • M
                        michmoor LAYER 8 Rebel Alliance @TheStormsOfFury
                        last edited by michmoor

                        @TheStormsOfFury great. Let’s monitor IPsec stability.
                        How soon will you know if it dropped?

                        Edit: to be clear, disable gateway action is checked? It should be

                        Firewall: NetGate,Palo Alto-VM,Juniper SRX
                        Routing: Juniper, Arista, Cisco
                        Switching: Juniper, Arista, Cisco
                        Wireless: Unifi, Aruba IAP
                        JNCIP,CCNP Enterprise

                        T 2 Replies Last reply Reply Quote 0
                        • T
                          TheStormsOfFury @michmoor
                          last edited by TheStormsOfFury

                          @michmoor In about 5 minutes. That will be the 53 minute mark. I did make the changes on both sites as well.

                          Thanks!

                          TSoF

                          1 Reply Last reply Reply Quote 0
                          • T
                            TheStormsOfFury @michmoor
                            last edited by

                            @michmoor no dice. Still timed out at 53 minutes and 29 seconds for 39 seconds.

                            Thanks for the suggestions! Open if you have any more.

                            my logs roll over so quick, i'll have to wait as it's already pushed out.

                            Thanks again!

                            TSoF

                            M 1 Reply Last reply Reply Quote 0
                            • M
                              michmoor LAYER 8 Rebel Alliance @TheStormsOfFury
                              last edited by

                              @TheStormsOfFury

                              Have you read the suggestions here?

                              https://docs.netgate.com/pfsense/en/latest/troubleshooting/ipsec-connections.html#dpd-is-unsupported-and-one-side-drops-while-the-other-remains

                              Firewall: NetGate,Palo Alto-VM,Juniper SRX
                              Routing: Juniper, Arista, Cisco
                              Switching: Juniper, Arista, Cisco
                              Wireless: Unifi, Aruba IAP
                              JNCIP,CCNP Enterprise

                              T 1 Reply Last reply Reply Quote 0
                              • T
                                TheStormsOfFury @michmoor
                                last edited by

                                @michmoor said in IPsec site to site dropping every 49-55 minutes:

                                https://docs.netgate.com/pfsense/en/latest/troubleshooting/ipsec-connections.html#dpd-is-unsupported-and-one-side-drops-while-the-other-remains

                                DPD is enabled on both sites; however, i did not have the periodic keepalive or ping set. I did go ahead and enable that and also changed it from "child actions" of restart / reconnect to close and clear SA. This time when I reset the tunnels instead of creating multiple connections, it only created one.

                                Now we wait 53 minutes and see what happens.

                                Thanks agian!

                                TSoF

                                M A 2 Replies Last reply Reply Quote 0
                                • M
                                  michmoor LAYER 8 Rebel Alliance @TheStormsOfFury
                                  last edited by

                                  @TheStormsOfFury
                                  If this doesnt fix i strongly feel there is some mismatch between the two. If you don't mind sharing your P1/P2 settings from each side. Pictures preferred.

                                  Firewall: NetGate,Palo Alto-VM,Juniper SRX
                                  Routing: Juniper, Arista, Cisco
                                  Switching: Juniper, Arista, Cisco
                                  Wireless: Unifi, Aruba IAP
                                  JNCIP,CCNP Enterprise

                                  T 1 Reply Last reply Reply Quote 0
                                  • A
                                    andrew_cb @TheStormsOfFury
                                    last edited by

                                    @TheStormsOfFury Yes, that rule looks correct now.
                                    If you enable logging and call it something like "Block VPN subnets leaks to WAN" you can check under Status > System Logs > Firewall and see all the times that the rule is triggered.

                                    1 Reply Last reply Reply Quote 0
                                    • A
                                      andrew_cb @TheStormsOfFury
                                      last edited by

                                      @TheStormsOfFury It might help to increase all the IPsec logging by one so that you can gather more data about what is happening during the re-keying. Also increase the size of the IPsec log so that more information is visible before being overwritten.

                                      Also, does RTR2 show anything in its logs? I wonder if due to the way that IPsec is part of the kernel, maybe RTR2 responding to some of the traffic during the re-key?

                                      T 2 Replies Last reply Reply Quote 1
                                      • T
                                        TheStormsOfFury @andrew_cb
                                        last edited by

                                        @andrew_cb

                                        So, inside VPN > IPsec > Advanced Settings

                                        There is a list of 16 IPsec logging controls. Which would you recommend increasing so we can get the best results??

                                        https://imgur.com/2g4WGXh

                                        1 Reply Last reply Reply Quote 0
                                        • T
                                          TheStormsOfFury @michmoor
                                          last edited by

                                          @michmoor So it still closed and I'm going to now past copies of the P1/P2 configs per site.

                                          While I was taking these images, I was confirming that they were all identical. Let me know if you see something I missed.

                                          Site 1 Phase 1: https://imgur.com/eldBRXO

                                          • Part 1: https://imgur.com/G4kEzHl
                                          • Part 2: https://imgur.com/x44xLAj
                                          • Part 3: https://imgur.com/N70XlmN
                                            Site 1 Phase 2 ONE
                                          • Part 1: https://imgur.com/DYdObqD
                                          • Part 2: https://imgur.com/iwBOlBM
                                            Site 1 Phase 2 TWO
                                          • Part 1: https://imgur.com/m1R7THi
                                          • Part 2: https://imgur.com/Q7PiEI0

                                          Site 2 Phase 1: https://imgur.com/xgtr7Rh

                                          • Part 1: https://imgur.com/jckl5jQ
                                          • Part 2: https://imgur.com/x44xLAj
                                          • Part 3: https://imgur.com/TBvYo0b
                                            Site 2 Phase 2 ONE
                                          • Part 1: https://imgur.com/PkvVr1R
                                          • Part 2: https://imgur.com/aBcSz2n
                                            Site 2 Phase 2 TWO
                                          • Part 1: https://imgur.com/Of4Phes
                                          • Part 2: https://imgur.com/aksgmlq
                                          M 1 Reply Last reply Reply Quote 0
                                          • M
                                            michmoor LAYER 8 Rebel Alliance @TheStormsOfFury
                                            last edited by michmoor

                                            @TheStormsOfFury
                                            Thanks for this and thanks for being organized in how you presented the pictures.

                                            Curious. For picture https://imgur.com/Of4Phes

                                            I noticed that the Local Network is set to 'Network' 10.0.1.0 which is different for the other site. Is this network not directly connected to pfsense? Is it routed (another gateway/router behind pfsense)?

                                            edit: I don't think that's the problem just curious. Trying to better understand the environment.

                                            Firewall: NetGate,Palo Alto-VM,Juniper SRX
                                            Routing: Juniper, Arista, Cisco
                                            Switching: Juniper, Arista, Cisco
                                            Wireless: Unifi, Aruba IAP
                                            JNCIP,CCNP Enterprise

                                            T 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.