Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Multiple VIP on multi wan troubles

    Scheduled Pinned Locked Moved
    HA/CARP/VIPs
    3
    11
    7.2k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • W
      wized36
      last edited by

      I'm a noob with pfsense and bsd so looking for a bit of direction.

      Have been trying for a couple of weeks now to get a stable multi wan pfsense box to replace my 2 ipcop boxes - as follows:

      1Ghz P3, 512Mb ram 20Gb HD and 3 nic's
      3 nic's are configured as:
      LAN - 192.168.150.254/24
      WAN - 203.94.17.3/24 - Gateway 203.94.17.1
      OPT1 - 61.39.95.3/26 - Gateway 61.39.95.1

      Beta 3 installed with latest (as at 4/5) update

      The problem I have found is - install with or without load balancing works great following the tutorial. Adding one VIP to either WAN or OPT interface and adding NAT entries all still works okay although responses to pings are intermittent. The same goes if load balancing is not configured. Once another VIP is added to either WAN or OPT interfaces, pings stop, NAT seems to break and finally (after about 10 mins) the box either hangs or auto reboots. On reboot there is an error  - config.xml not found and the system has all NIC's set to 0.0.0.0 and logs are empty.

      I have changed mobo, cpu, ram, hd, and all nics with the same results everytime so I have determined its a config error.

      Initially I had the VIP's added as CARP /32 but after reading some posts I set them with the correct subnets ie. /24 and /26 - either way I get the same error sometimes after I add a couple if VIP's and sometimes after 4 or 5. I have tried with both advanced outbound nat on and off with the same result.

      I have both of these networks working correctly on the ipcop boxes but cannot get the pfsense box working with anymore than just one IP assigned to each nic. Is this a limitation with pfsense/freebsd. I need to have about 20 VIP's per nic with about 5 NAT entries for each.

      Any suggestions or advice would be appreciated.

      1 Reply Last reply Reply Quote 0
      • H
        hoba
        last edited by

        Destroying the config sounds like a configreplication loop to me. Any chance that your machine1 syncs to IP of machine2 and machine2 syncs back to machine1? You should only have machine1 syncing config to machine2. Basically at machine2 only the first 2 settings at firewall>VIP, carp settings should be configured (synchronize enabled, synchronize interface) unless there is a third machine that you want to daisy chain the configuration to (machine1->machine2->machine3…).
        Please check this setting and retest. I have some locations running CARP with 4 and more public IPs without issues.

        1 Reply Last reply Reply Quote 0
        • W
          wized36
          last edited by

          Upgraded to Beta 4 with cvs_sync.sh releng_1

          Added VIP's as proxyarp and all seems stable and working

          Just a note that when I tried to add carp using beta 4 I get the following error:

          Fatal error: Call to undefined function: return_first_three_octets() in /usr/local/www/firewall_virtual_ip_edit.php on line 117

          Thanks.

          1 Reply Last reply Reply Quote 0
          • S
            sullrich
            last edited by

            This is now fixed, thanks.

            1 Reply Last reply Reply Quote 0
            • W
              wized36
              last edited by

              Worked okay for a few hours - added another carp VIP and then the same error - auto reboot with missing xml config. For now I dont have time to work out what is wrong and will revisit pfsense with later release. Thanks and its a pity as pfsense promises alot.

              1 Reply Last reply Reply Quote 0
              • H
                hoba
                last edited by

                Did you actually read my hint about a possible configsync loop? What geekgod meant in his answer above was meant concerning the error when adding a carp vip.

                1 Reply Last reply Reply Quote 0
                • S
                  sullrich
                  last edited by

                  @wized36:

                  Worked okay for a few hours - added another carp VIP and then the same error - auto reboot with missing xml config. For now I dont have time to work out what is wrong and will revisit pfsense with later release. Thanks and its a pity as pfsense promises alot.

                  CARP sync loop.  AKA user pilot error.

                  1 Reply Last reply Reply Quote 0
                  • W
                    wized36
                    last edited by

                    I have no doubt it is a pilot error, so I have tried to fall back to the simplest configuration.

                    I am now running with no carp failover and no load balancing setup, so just WAN & OPT routing to 2 different public networks and LAN. So there is just one pfsense box, internet connectivity on both WAN and OPT network is by static IP. Could it be a problem that I have an IPCOP box on the same LAN subnet (different IP of course) which routes out via the same WAN gateway. Note that there are no duplicate IP's in use on any network.

                    Also if I dont add any VIP's and configure load balancing - that feature works fine. If I just add NAT for the primary IP's that also works fine.

                    Pfsense only falls over after adding more than one VIP and it doesn't seem to matter whether it is CARP, Proxy ARP or type other. I have a /24 subnet for my WAN and a /26 for the OPT network. Could it be a problem not have a full /24 subnet for the OPT network. Is it correct that I use the correct subnet when adding VIP's or should I use /32 for each VIP that is added.

                    Again any suggestions are appreciated.

                    1 Reply Last reply Reply Quote 0
                    • H
                      hoba
                      last edited by

                      ProxyARP IPs should be each /32.
                      CARP IPs should have the subnetmask of the real IP of the interface they are on and also be in the range of the subnet (it doesn't matter if you have a /29, /26 or /24 subnet, it just has to match the real interface settings).

                      1 Reply Last reply Reply Quote 0
                      • W
                        wized36
                        last edited by

                        I believe I found the problem - it had nothing to do with either pfsense or the config - as suggested I seems it was a loopback issue of sorts.

                        I have a HP procurve switch with 2 vlan's (the WAN & OPT networks) configured. While there were no IP conflicts I think that due to the IPCOP box also being on the WAN vlan, the Procurve's ARP table was causing a broadcast storm and I assume linux "deals" with that differently to freebsd as IPCOP was not affected while pfsense fell over.

                        Anyway since replacing the procurve with 2 simple switches everything seems to be working okay now.

                        Thanks for your help.

                        1 Reply Last reply Reply Quote 0
                        • H
                          hoba
                          last edited by

                          I still wonder what should cause the destruction of the config.xml. Keep an eye on it. There might be something else going on which is unrelated to the other error.

                          1 Reply Last reply Reply Quote 0
                          • First post
                            Last post