Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Howto : CARP + VIP and outbound rules with Pfsense 2.0 release …

    Scheduled Pinned Locked Moved HA/CARP/VIPs
    19 Posts 3 Posters 18.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Z
      zeratoun
      last edited by

      Hi,

      I am having a trouble setting CARP + VIP + pfsync under pfsense 2.0 release

      My configuration :

      actual architecture : amd64 (but tested with 32bit too)

      MASTER WAN ip : XXX.XXX.XXX.52
      SLAVE WAN ip : XXX.XXX.XXX.53
      VIP WAN ip : XXX.XXX.XXX.51/29
      GW WAN (Default) : XXX.XXX.XXX.58
      Subnet : 29

      MASTER SYNC : YYY.YYY.YYY.52
      SLAVE SYNC : YYY.YYY.YYY.53
      Subnet : 24

      MASTER LAN : ZZZ.ZZZ.ZZZ.52
      SLAVE LAN : ZZZ.ZZZ.ZZZ.53
      VIP LAN : ZZZ.ZZZ.ZZZ.51/24
      GW LAN : ZZZ.ZZZ.ZZZ.254
      Subnet : 24

      Pfsync sync interface : SYNC
      Pfsync sync test : ok

      Failover test : Ok

      Outbound NAT rule set to manual :

      • Int: WAN,SRC: ZZZ.ZZZ.ZZZ.0/24, DST = *, NATADDR= XXX.XXX.XXX.51
      • Int: WAN,SRC: 127.0.0.1/8, DST = *, NATADDR= XXX.XXX.XXX.51
      • Int: LAN,SRC: *, DST = XXX.XXX.XXX.48/29, NATADDR= ZZZ.ZZZ.ZZZ.51

      My Trouble :

      When, from outside, I try to use the ViP addresses everything goes fine, the servers correctly do the failover.

      The trouble is when the comunication start from the pfsense to the outside.
      ie : If I try to go on internet it uses the WAN address of the machine and not the VIP WAN address. Or if I do a telnet from the pfsense to a machine in the LAN or in the internet it uses the LAN or WAN address set on the interfaces and not the VIP address that they must use as specified in the outbound NAT rules.

      someone knows what I am missing ?

      1 Reply Last reply Reply Quote 0
      • A
        arcel
        last edited by

        May i know where is XXX.XXX.XXX.48/29 from the last line of your OUTBOUND NAT config?

        1 Reply Last reply Reply Quote 0
        • P
          podilarius
          last edited by

          You want to use 127.0.0.1/8 as the source if you want to create a NAT to look like the firewall is communicating on the VIP for the lan, like the second line but for LAN interface. Generally, it is fine if the firewall uses the WAN and LAN ips. Is there some special circumstance here where that cannot be allowed?

          1 Reply Last reply Reply Quote 0
          • A
            arcel
            last edited by

            I have the same setup. I followed the steps on:

            http://pfsense.loquefaltaba.com/tutorials/carp/carp-cluster-new.htm

            http://doc.pfsense.org/index.php/Configuring_pfSense_Hardware_Redundancy_%28CARP%29

            but my problem is  when i check whatismyip.com, it shows me the MASTER WAN IP, or the SLAVE WAN IP when i remove the turn off the master..

            from what i understand, it should be the VIP WAN ip that will show all the time.
            using the same format, my current config for the outbound is:

            Outbound NAT rule set to automatic :

            • Int: WAN,SRC: ZZZ.ZZZ.ZZZ.0/24, DST = *, NATADDR= XXX.XXX.XXX.51
            • Int: WAN,SRC: 127.0.0.1/8, DST = *, NATADDR= XXX.XXX.XXX.51
            • Int: LAN,SRC: *, DST = 127.0.0.1/8, NATADDR= ZZZ.ZZZ.ZZZ.51

            even without the outbound rule, result is the same.

            1 Reply Last reply Reply Quote 0
            • Z
              zeratoun
              last edited by

              To Arcel :
              It's a Public IP subnet

              To podilarius :
              My second line does not seem to have any effect …..... from the server if I try to go on internet it uses .52 or .53 (master or slave) instead of .51 (VIP)

              I cannot allow it because I must avoid the most packet loss in a failover scenario ..... :

              • By using Wan or Lan ip I loose all the connexions (because the SRC of the packets are .52 or .53 and so the synchronised states are deleted).
              • The states being syncronised between master and slave, by using VIP Wan or VIP Lan I loose just some data during the failover but the connexions stay open because the SRC of the packets is the same (.51)
              1 Reply Last reply Reply Quote 0
              • A
                arcel
                last edited by

                Hi Zeratoun:

                it seems that we have the same question.
                from LAN going to internet, it uses the master or the slave WAN IP instead of WAN VIP.

                1 Reply Last reply Reply Quote 0
                • Z
                  zeratoun
                  last edited by

                  I agree !

                  I supose that the manual outbound NAT is not aplied …. but I don't know how to check this....

                  1 Reply Last reply Reply Quote 0
                  • P
                    podilarius
                    last edited by

                    I will try to duplicate that issue with my test cluster.

                    zeratoun: if you go to whatismyip.com from a computer behind the firewall, does it use WAN or VIP address? I only ask for clarification as I thought you meant traffic originating FROM the firewall. if that is the case, you don't have to worry about packet loss since pfsense normally doesn't generate the traffic, only the computer on the LAN or OPT networks its protecting.

                    1 Reply Last reply Reply Quote 0
                    • P
                      podilarius
                      last edited by

                      There must be a rule that you are not setting correctly. If you are trying with ping and other service on the pfsense, then you must set outbound NAT for the interface IP as well.
                      All traffic from behind the FW get NATed correctly on my test system.

                      Like:
                      WAN  XXX.XXX.XXX.7/32 * * * XXX.XXX.XXX.4 * NO

                      Same for the LAN side.

                      1 Reply Last reply Reply Quote 0
                      • Z
                        zeratoun
                        last edited by

                        Hi podilarius,

                        in Whatismyip it shows me WAN ip and not VIP ip ….. that's my trouble :(

                        1 Reply Last reply Reply Quote 0
                        • P
                          podilarius
                          last edited by

                          can you screen shot your outbound nat screen. you can pm me that if you don't want to post it in here.

                          1 Reply Last reply Reply Quote 0
                          • P
                            podilarius
                            last edited by

                            I can also say that having the /32 for the interface addresses is a bad thing. the secondary system does really like that.

                            1 Reply Last reply Reply Quote 0
                            • A
                              arcel
                              last edited by

                              Hi podilarius,
                              Given the settings below without the OUTBOUND NAT rule. can you config how it should be? also if there's a need on the WAN and LAN firewall rules to be added?

                              Im working on:
                              Version    2.0-RELEASE (i386)
                              built on Tue Sep 13 17:28:43 EDT 2011

                              MASTER WAN ip : XXX.XXX.XXX.121
                              SLAVE WAN ip : XXX.XXX.XXX.122
                              VIP WAN ip : XXX.XXX.XXX.123/24
                              GW WAN (Default) : XXX.XXX.XXX.254
                              Subnet : 24

                              MASTER SYNC : YYY.YYY.YYY.10
                              SLAVE SYNC : YYY.YYY.YYY.20
                              Subnet : 24

                              MASTER LAN : ZZZ.ZZZ.ZZZ.251
                              SLAVE LAN : ZZZ.ZZZ.ZZZ.252
                              (Ive noticed zeratoun has a value of GW LAN : ZZZ.ZZZ.ZZZ.254 from here. should it be the GW VIP LAN only here(in my case  ZZZ.ZZZ.ZZZ.250) that will be used by LAN network?( which work fine with me)
                              GW VIP LAN : ZZZ.ZZZ.ZZZ.250
                              Subnet : 24

                              Pfsync sync interface : SYNC
                              Pfsync sync test : ok
                              Failover test : Ok

                              again, problem is when i check whatsmyip.com, it shows the master, or the slave WAN ip when master fail.

                              Many thanks in advance!

                              1 Reply Last reply Reply Quote 0
                              • P
                                podilarius
                                last edited by

                                arcel,
                                You must use Advanced outbound NAT with failover CARP.
                                You need only 3 rules.

                                Interface Source           Source Port Destination Destination Port NAT Address NAT Port Static Port Description

                                WAN  127.0.0.0/8         *         *                  *                         *         1024:65535   NO  Auto created rule for localhost to WAN
                                WAN  ZZZ.ZZZ.ZZZ.0/24 *         *             500         XXX.XXX.XXX.254 *   YES Auto created rule for ISAKMP - LAN to WAN
                                WAN  ZZZ.ZZZ.ZZZ.0/24 *         *             *                 XXX.XXX.XXX.254 *   NO  Auto created rule for LAN to WAN

                                I hope the columns line up correctly. Traffic originating FROM the pfSense firewall should not be NATed. I would not use reflection at all, instead I would use split brain DNS utilizing the DNS services within pfSense. It works VERY well as the traffic stays within the LAN. All your servers and DHCP needs to have the default gateway of ZZZ.ZZZ.ZZZ.250.

                                1 Reply Last reply Reply Quote 0
                                • A
                                  arcel
                                  last edited by

                                  THANKS!!!! THANKS!!!! THANKS!!!! THANKS!!!! THANKS!!!! to you PODILARIUS

                                  i followed your config with slight changes:

                                  You must use Advanced outbound NAT with failover CARP.  apply this settings
                                  You need only 3 rules.

                                  Interface  Source              Source Port  Destination  Destination Port  NAT Address  NAT Port  Static Port  Description

                                  WAN      127.0.0.0/8            *            *                  *                          *          1024:65535    NO  Auto created rule for localhost to WAN 
                                  WAN      ZZZ.ZZZ.ZZZ.0/24    *            *                500          XXX.XXX.XXX.123    *    YES Auto created rule for ISAKMP - LAN to WAN 
                                  WAN      ZZZ.ZZZ.ZZZ.0/24    *            *                *                  XXX.XXX.XXX.123    *    NO  Auto created rule for LAN to WAN

                                  now it work like a charm!!!!!

                                  to zeratoun:
                                  try this one if this would work on you.

                                  1 Reply Last reply Reply Quote 0
                                  • Z
                                    zeratoun
                                    last edited by

                                    Hi,

                                    ~~I have made exactly as you (only that your .123 is .51 in my case) but whatismyip.com still shows me .52 or .53 instead of .51

                                    is there a command line way to know if the outbound rules are even applied ?~~

                                    Edit, it work too ….... however if I telnet from pfsense console (just for test) it still uses .52 or .53 ...

                                    Best regards

                                    :(

                                    1 Reply Last reply Reply Quote 0
                                    • P
                                      podilarius
                                      last edited by

                                      @zeratoun:

                                      Edit, it work too ….... however if I telnet from pfsense console (just for test) it still uses .52 or .53 ...

                                      You want that to happen and that is normal, FROM pfSense firewall itself. States from pfsync for traffic originating from the LAN will be duplicated to the backup and there will be minimal packet loss (perhaps none). You want to make sure that from within the LAN the correct IP is shown by whatismyip.com.

                                      1 Reply Last reply Reply Quote 0
                                      • Z
                                        zeratoun
                                        last edited by

                                        Exactly,

                                        i want that, from the localhost of the pfsense firewall itself it uses the VIP LAN or WAN …. it's possible ?

                                        Best regards,

                                        1 Reply Last reply Reply Quote 0
                                        • P
                                          podilarius
                                          last edited by

                                          @zeratoun:

                                          Exactly,

                                          i want that, from the localhost of the pfsense firewall itself it uses the VIP LAN or WAN …. it's possible ?

                                          Best regards,

                                          It is possible but highly NOT recommended. I got that running in my test environment and CARP was not happy as ping stopped to the gateway on the secondary firewall. I think this will have an adverse effect on the clusters ability to fail over correctly. I didn't have a chance to test fail over, but i did notice that I could not download packages or ping the gateway. There is not reason I can think of to do this. Would you mind telling us why you would like to do that?

                                          1 Reply Last reply Reply Quote 0
                                          • First post
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.