Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    LAN TO LAN WITH 4 VPN TUNNEL (REDUNDANT)

    Scheduled Pinned Locked Moved IPsec
    28 Posts 5 Posters 19.6k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      martinc_77
      last edited by

      HI FRIENDS, I HAVE 2 ISP FOR EACH PFSENSE AND 2 PFSENSE TO JOIN.

      MY TOPOLOGY IS:

      SUBNET 1
            |
      PFSENSE 1
      |        |
      ISP1  ISP2
      |        |
      INTERNET
      |        |
      ISP3  ISP4
      |        |
      PFSENSE 2
            |
      SUBNET 2

      I NEED CONFIGURE THIS:

      TUNNEL 1 (ISP 1 <–-> ISP 3)    TO JOIN SUBNET1 & SUBNET2
                    TUNNEL 2 (ISP 1 <---> ISP 4)    TO JOIN SUBNET1 & SUBNET2
                    TUNNEL 3 (ISP 2 <---> ISP 3)    TO JOIN SUBNET1 & SUBNET2
                    TUNNEL 4 (ISP 2 <---> ISP 4)    TO JOIN SUBNET1 & SUBNET2

      THE IDEA IS :

      IF TUNNEL 1 IS DOWN (ISP 1 OR ISP 3 CRASH) THEN REDIRECT TRAFFIC TO TUNNEL 4
                  IF TUNNEL 2 IS DOWN (ISP 1 OR ISP 4 CRASH) THEN REDIRECT TRAFFIC TO TUNNEL 3
                  IF TUNNEL 3 IS DOWN (ISP 2 OR ISP 3 CRASH) THEN REDIRECT TRAFFIC TO TUNNEL 2
                  IF TUNNEL 4 IS DOWN (ISP 2 OR ISP 4 CRASH) THEN REDIRECT TRAFFIC TO TUNNEL 1

      THIS IS POSSIBLE??? HOW?

      TKS

      1 Reply Last reply Reply Quote 0
      • H
        hoba
        last edited by

        Sorry, there is no way to make this work in this configuration that I can think of atm.

        1 Reply Last reply Reply Quote 0
        • T
          tunge2
          last edited by

          is this nog possible with openvpn?

          1 Reply Last reply Reply Quote 0
          • JeGrJ
            JeGr LAYER 8 Moderator
            last edited by

            That is less a problem with openvpn, the tunnels - so I'm pretty sure - could be established like that. But I think it's more a problem in the joint venture of openvpn an carp that would be necessary to do a failover for the tunnels!?

            Don't forget to upvote 👍 those who kindly offered their time and brainpower to help you!

            If you're interested, I'm available to discuss details of German-speaking paid support (for companies) if needed.

            1 Reply Last reply Reply Quote 0
            • H
              hoba
              last edited by

              The problem rather is having same source and destination subnets for all tunneldefinitions which is causing a conflict. And even if implementing some logic that is reconfiguring the tunnels on failure of an ISP, how should the other end know that one ISP at the remote site has failed to switch it's tunnel definition? This only could work if the both ends are exchanging information about their WAN states. You could make it one way redundant with a carp setup at one end and the other end waitung for it to tunnel in as "mobile client". However this only would provide redundancy at one end.

              LAN
                      |
                +–---+------+
                |            |    (This would need ifdepd to bring the Master down on ISP1 failure)
              pfSense      pfSense  (Master has tunnel to ISP1 at remote end)
              Master      Slave  (Slave has tunnel to ISP2 at remote end)
                |  |      |  |
                +-----------+  |
                |  |          |
                |  +----------+
                |  |         
              ISP1 ISP2
                |  |
                |  |
              (Internet)
                |  |
              ISP1 ISP2
                |  |
              pfSense (awaiting mobile clients IPSEC at ISP1 and ISP2)
                  |
                  |
                LAN

              (Don't blame me if this won't work, never tried such a setup yet  ;D )

              1 Reply Last reply Reply Quote 0
              • S
                sbyoon
                last edited by

                Dear Hoba,

                Could you give the method in more detail how to make one way redundant with a carp setup?
                And I want to know how the two pfsense should be connected for carp setup. Which one is correct in below?

                LAN
                        |
                  +–---+-------------------+
                  |                          |   
                pfSense                pfSense
                Master                  Slave 
                WAN  OPT1          WAN  OPT1
                  |        |                |        |
                  +---------------------+      |
                  |        |                          |
                  |        +--------------------+
                  |        |       
                ISP1    ISP2
                  |        |

                LAN
                        |
                  +-----+-------------------+
                  |                          |   
                pfSense                pfSense
                Master                  Slave 
                WAN  OPT1          OPT1  WAN
                  |        |                |        |
                  +---------------------+      |
                  |        |                          |
                  |        +--------------------+
                  |        |       
                ISP1    ISP2
                  |        |

                I don't think the first one can be possible because we cannot make tunnel at OPT1 with pfsense. It means we cannot make the tunnel on ISP2.
                Pls adv how to configure the carp in detail if the second one is correct. Thank you.

                1 Reply Last reply Reply Quote 0
                • H
                  hoba
                  last edited by

                  Having IPSEC tunnels at OPT interfaces needs fixing: http://cvstrac.pfsense.com/tktview?tn=1028,6 but after that it should work.

                  In case you don't want to use the both ISPs at the same time and have the second one only for failover you can set it up even easier:

                  LAN
                        |
                    +–--+----+
                    |        |
                  pfSense  pfSense  (shared CARP IP as gateway at LAN and ifdepd to briong LAN down on ISP failure)
                  Master    Slave    (Master has tunnel to ISP1 at remote end)
                    |        |      (Slave has tunnel to ISP2 at remote end)
                    |        |
                  ISP1      ISP2
                    |        |
                    |        |
                  (  Internet ) 
                      |    |
                      |    |
                    ISP1  ISP2
                      |    |
                      |    |
                      pfSense (waiting for mobile clients)
                        |
                        LAN

                  1 Reply Last reply Reply Quote 0
                  • S
                    sbyoon
                    last edited by

                    Thank you, Hoba,

                    Noted now I should use two pfsense until IPsec will be available at OPT1. And I don't need two ISP at once. I need to use just one ISP on failure of another one. So your suggestion will be helpful. I think I know how to share Carp IP as gateway at LAN but I don't know how ifdepd bring LAN down on ISP failure. I don't even know what ifdepd is. Could you give me more explanation about it? Thank you.

                    1 Reply Last reply Reply Quote 0
                    • H
                      hoba
                      last edited by

                      It's a package. You can add dependencies between interfaces with it like "if interface wan is down then down interface lan". This is needed to make the slave machine become master while ISP1 is down at the original master.

                      _ifdepd implements dependencies between network interfaces in a
                      reliable way. If any of the source interfaces fails, ifdepd sets all
                      destination interfaces to state down. If all source interfaces are active,
                      ifdepd sets all destination interfaces to state up.

                      For example, it can be used with carp(4) to provide failover functionality
                      on gateways/firewalls._

                      1 Reply Last reply Reply Quote 0
                      • M
                        martinc_77
                        last edited by

                        Hi hoba is me again, tks for the info of redundant tunnel, very cool.
                        I' am another problem , load balancing run ok but if disconnect wan interface or Wan ISP is down, then not ping OPT interface (internet ping), not internet access (from my lan) and not tunnel is establish for OPT.
                        But if i copy the OPT gateway in the Wan gateway of the WAN interface (when wan isp is down)  all run ok.

                        What's happend???

                        route default is the problem or is a bug?

                        Sorry but my english is bad.

                        Tks for all

                        1 Reply Last reply Reply Quote 0
                        • H
                          hoba
                          last edited by

                          There is still an open bug for IPSECs at OPTs: http://cvstrac.pfsense.com/tktview?tn=1028,6 which needs to be fixed.

                          1 Reply Last reply Reply Quote 0
                          • S
                            sbyoon
                            last edited by

                            Dear Hoba,

                            I tested ifdepd yesterday. And it works well with below diagram. I can use internet automatically with another pfsense when WAN at master pfsense is down.

                            LAN
                                  |
                                CARP1
                              +–--+----+
                              |            |
                            pfSense1  pfSense2  (shared CARP IP as gateway at LAN and ifdepd to briong LAN down on ISP failure)
                            Master      Slave   
                            CARP0      CARP0
                              |              |     
                              |              |
                            ISP1        ISP2
                              |              |
                              |              |
                            (  Internet )
                                  |   
                                  |   
                                  ISP3
                                  |   
                                  |   
                                pfSense3 (waiting for mobile clients)
                                  |
                                  LAN

                            But it does not work with IPSec. I think "Failover IPSEC" on IPSec configuration page would help solving this problem. I think it would be used for replacing IPSEC tunneling from pfsense1-pfsense3 to pfsense2-pfsense3 automatically. But I don't have any information how to configure it. Could you advise me which field on "Failover IPSEC" tab means?

                            "Failover IPSEC" has five fields to be filled as below.

                            1. "Enable" - I think it should be checked.

                            2. "Interface" - The hint says "Select the carp interface to use." And I can select one of r r r p l p c c. Which one should I select?

                            3. "Failover IP" - The hint says "Enter the IP address you would like to use for failover. HINT: You normally want to use a public CARP ip". Should I fill it with the Carp1 IP address? Carp1 IP address is the virtual IP address on LAN.

                            4. "Peer IP" - The hint says "Enter the peers ip address. HINT: You normally want to enter the peers REAL LAN IP here." I really don't know what IP address should be used for this field. pfSense2's LAN IP address or pfSense3' LAN address????

                            5. "Shared key" - The hint says "The shared AES key used to encrypt messages between sasyncd hosts. This configuration setting is required and must be either 16, 24 or 32 bytes long (corresponding to AES using a 128, 192 or 256 bit key)." I think it is a new shared key and it does not involve with the shared key that was used to make IPSEC tunneling.

                            And one more question. I think "Failover IPSEC" should be configured both of pfsense1 and pfsense2. Is it correct?

                            Thank you.

                            1 Reply Last reply Reply Quote 0
                            • H
                              hoba
                              last edited by

                              Failover IPSEC is for use with CARP on WAN to use the CARP IP instead of the real WAN IP. This doesn't apply to your setup and won't help you.

                              1 Reply Last reply Reply Quote 0
                              • S
                                sbyoon
                                last edited by

                                Noted. I should wait until IPSec on OPT1 will be fixed. Then, do you think IPSec failover will be possible with outgoing loadbalancing if OPT1 IPSec is available? Of course in this case we can use only one pfsense for failover.

                                Thank you.

                                1 Reply Last reply Reply Quote 0
                                • H
                                  hoba
                                  last edited by

                                  It should just work the way it's drawn without failover IPSEC settings. Just noticed you only have one WAN (ISP3) at the opposite end and not like before 2 isps.

                                  1 Reply Last reply Reply Quote 0
                                  • S
                                    sbyoon
                                    last edited by

                                    I think it does not work because of the conflict of same local subnet from pfsenes1 and pfsense2 at the remote end network. In order to succeed in IPSec failover, the first tunnelling of pfsense1-pfsense3 should be remove clearly before the second tunnelling of pfsense2-pfsense3 will be established. In case of my above diagram, the first and second tunnelling are established at once and one of those two tunnels does not work even if one of ISP disconnets and I can access to internet through another pfsense.

                                    Is there any way to remove the first tunnel automatically when another one is trying to establish tunnel? And only one IPSec tunnel should be allowed when both of ISP1 and ISP2 are connected.

                                    Thank you.

                                    1 Reply Last reply Reply Quote 0
                                    • H
                                      hoba
                                      last edited by

                                      pfSense3 is waiting for mobile clients. pfSense1's WAN is failing and pfSense2 is establishing the IPSEC as there no w is traffic for the remote subnet due to becoming the gateway. For pfSense3 this should just look like the IP of pfSense 1 has changed. I have setups like this where one pfSense sits at a dynamic IP and it works fine. Maybe http://www.pfsense.org/mirror.php?section=tutorials/mobile_ipsec/ helps how to configure it.

                                      1 Reply Last reply Reply Quote 0
                                      • S
                                        sbyoon
                                        last edited by

                                        Of course, I tested with pfsense3 wainting for mobile client. The problem is that pfsense3 cannot figure out disconneting of pfsense1 and it remember the tunnel with pfsense1 even after pfsense1 disappear. So the LAN subnet of pfsense1 and pfsense2 conflicts on pfsense3 because pfsense1 and pfsense2 has same LAN subnet and then the tunnel of pfsense2-pfsense3 does not work. Is there any solution of this problem?

                                        1 Reply Last reply Reply Quote 0
                                        • H
                                          hoba
                                          last edited by

                                          Try "prefer old IPSEC SA" setting at system>advanced at all ends and see if that makes a difference. If that doesn't help try using smaller lifetimes (like 300 seconds). This way a tunnel should expire after 5 minutes and the backupmachine then will hopefully be able to connect though this means some downtime until the tunnel will be estblished again on failover.

                                          1 Reply Last reply Reply Quote 0
                                          • S
                                            sbyoon
                                            last edited by

                                            Noted, I will try and let you know of the result. Thank you, Hoba.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.