Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Carp issues with one interface

    Scheduled Pinned Locked Moved HA/CARP/VIPs
    11 Posts 3 Posters 5.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • E
      egarnel
      last edited by

      Here is my issue:  the wan, lan & carp interfaces on the primary become master.  The wan & carp interfaces on the 2nd become backup, the lan stays in master mode for some reason.

      Here is my setup:

      2 pfsense servers

      server A    primary  RC2h

      wan ip aaa.bbb.ccc.27 real  .25 virtual 
      lan  ip  192.168.20.14  .4 virtual
      carp ip 192.168.4.1 real  .3 virtual
      mgmt  xxx.yyy.ppp.207  no carp

      server B secondary  RC2

      wan ip aaa.bbb.ccc.26 real  .25 virtual 
      lan  ip  192.168.20.5  .4 virtual
      carp ip 192.168.4.2 real  .3 virtual
      mgmt  xxx.yyy.ppp.215  no carp

      ifconfig

      em0: flags=8943 <up,broadcast,running,promisc,simplex,multicast>mtu 1500
              options=b <rxcsum,txcsum,vlan_mtu>inet 192.168.20.5 netmask 0xffffff00 broadcast 192.168.20.255
              inet6 fe80::204:23ff:fec9:3418%em0 prefixlen 64 scopeid 0x1
              ether 00:04:23:c9:34:18
              media: Ethernet autoselect (1000baseTX <full-duplex>)
              status: active
      em1: flags=8943 <up,broadcast,running,promisc,simplex,multicast>mtu 1500
              options=b <rxcsum,txcsum,vlan_mtu>inet6 fe80::204:23ff:fec9:3419%em1 prefixlen 64 scopeid 0x2
              inet aaa.bbb.ccc.26 netmask 0xffffff00 broadcast aaa.bbb.ccc.255
              ether 00:04:23:c9:34:19
              media: Ethernet autoselect (1000baseTX <full-duplex>)
              status: active
      bge0: flags=8843 <up,broadcast,running,simplex,multicast>mtu 1500
              options=1b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging>inet xxx.yyy.ppp.207 netmask 0xffffff00 broadcast xxx.yyy.ppp.255
              inet6 fe80::216:36ff:fe13:a608%bge0 prefixlen 64 scopeid 0x3
              ether 00:16:36:13:a6:08
              media: Ethernet autoselect (1000baseTX <full-duplex>)
              status: active
      bge1: flags=8943 <up,broadcast,running,promisc,simplex,multicast>mtu 1500
              options=1b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging>inet 192.168.4.1 netmask 0xffffff00 broadcast 192.168.4.255
              inet6 fe80::216:36ff:fe13:a609%bge1 prefixlen 64 scopeid 0x4
              ether 00:16:36:13:a6:09
              media: Ethernet autoselect (1000baseTX <full-duplex>)
              status: active
      pfsync0: flags=41 <up,running>mtu 1348
              pfsync: syncdev: lo0 maxupd: 128
      lo0: flags=8049 <up,loopback,running,multicast>mtu 16384
              inet 127.0.0.1 netmask 0xff000000
              inet6 ::1 prefixlen 128
              inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
      pflog0: flags=100 <promisc>mtu 33208
      carp0: flags=49 <up,loopback,running>mtu 1500
              inet 192.168.4.3 netmask 0xffffff00
              carp: BACKUP vhid 1 advbase 1 advskew 29
      carp1: flags=49 <up,loopback,running>mtu 1500
              inet aaa.bbb.ccc.25 netmask 0xffffff00
              carp: BACKUP vhid 2 advbase 1 advskew 29
      carp2: flags=49 <up,loopback,running>mtu 1500
              inet 192.168.20.4 netmask 0xffffff00
              carp: MASTER vhid 3 advbase 1 advskew 29

      ping 192.168.20.14
      PING 192.168.20.14 (192.168.20.14): 56 data bytes
      64 bytes from 192.168.20.14: icmp_seq=0 ttl=64 time=0.293 ms
      64 bytes from 192.168.20.14: icmp_seq=1 ttl=64 time=0.407 ms
      64 bytes from 192.168.20.14: icmp_seq=2 ttl=64 time=0.283 ms
      64 bytes from 192.168.20.14: icmp_seq=3 ttl=64 time=0.292 ms

      ffff
      ifconfig
      bge0: flags=8943 <up,broadcast,running,promisc,simplex,multicast>mtu 1500
              options=1b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging>inet6 fe80::210:18ff:fe06:76b3%bge0 prefixlen 64 scopeid 0x1
              inet 192.168.20.14 netmask 0xffffff00 broadcast 192.168.20.255
              ether 00:10:18:06:76:b3
              media: Ethernet autoselect (1000baseTX <full-duplex>)
              status: active
      bge1: flags=8943 <up,broadcast,running,promisc,simplex,multicast>mtu 1500
              options=1b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging>inet aaa.bbb.ccc.27 netmask 0xffffff00 broadcast aaa.bbb.ccc.255
              inet6 fe80::211:43ff:fe5b:72d6%bge1 prefixlen 64 scopeid 0x2
              ether 00:11:43:5b:72:d6
              media: Ethernet autoselect (100baseTX <half-duplex>)
              status: active
      bge2: flags=8843 <up,broadcast,running,simplex,multicast>mtu 1500
              options=1b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging>inet xxx.yyy.ppp.215 netmask 0xffffff00 broadcast xxx.yyy.ppp.255
              inet6 fe80::211:43ff:fe5b:72d7%bge2 prefixlen 64 scopeid 0x3
              ether 00:11:43:5b:72:d7
              media: Ethernet autoselect (100baseTX <half-duplex>)
              status: active
      fxp0: flags=8943 <up,broadcast,running,promisc,simplex,multicast>mtu 1500
              options=b <rxcsum,txcsum,vlan_mtu>inet 192.168.4.2 netmask 0xffffff00 broadcast 192.168.4.255
              inet6 fe80::202:b3ff:febb:9776%fxp0 prefixlen 64 scopeid 0x4
              ether 00:02:b3:bb:97:76
              media: Ethernet autoselect (100baseTX <full-duplex>)
              status: active
      pflog0: flags=100 <promisc>mtu 33208
      enc0: flags=0<> mtu 1536
      lo0: flags=8049 <up,loopback,running,multicast>mtu 16384
              inet 127.0.0.1 netmask 0xff000000
              inet6 ::1 prefixlen 128
              inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7
      pfsync0: flags=41 <up,running>mtu 1348
              pfsync: syncdev: fxp0 maxupd: 128
      carp0: flags=49 <up,loopback,running>mtu 1500
              inet 192.168.4.3 netmask 0xffffff00
              carp: MASTER vhid 1 advbase 1 advskew 0
      carp1: flags=49 <up,loopback,running>mtu 1500
              inet aaa.bbb.ccc.25 netmask 0xffffff00
              carp: MASTER vhid 2 advbase 1 advskew 0
      carp2: flags=49 <up,loopback,running>mtu 1500
              inet 192.168.20.4 netmask 0xffffff00
              carp: MASTER vhid 3 advbase 1 advskew 0

      ping 192.168.20.5
      PING 192.168.20.5 (192.168.20.5): 56 data bytes
      64 bytes from 192.168.20.5: icmp_seq=0 ttl=64 time=0.644 ms
      64 bytes from 192.168.20.5: icmp_seq=1 ttl=64 time=0.436 ms
      64 bytes from 192.168.20.5: icmp_seq=2 ttl=64 time=0.459 ms</up,loopback,running></up,loopback,running></up,loopback,running></up,running></up,loopback,running,multicast></promisc></full-duplex></rxcsum,txcsum,vlan_mtu></up,broadcast,running,promisc,simplex,multicast></half-duplex></rxcsum,txcsum,vlan_mtu,vlan_hwtagging></up,broadcast,running,simplex,multicast></half-duplex></rxcsum,txcsum,vlan_mtu,vlan_hwtagging></up,broadcast,running,promisc,simplex,multicast></full-duplex></rxcsum,txcsum,vlan_mtu,vlan_hwtagging></up,broadcast,running,promisc,simplex,multicast></up,loopback,running></up,loopback,running></up,loopback,running></promisc></up,loopback,running,multicast></up,running></full-duplex></rxcsum,txcsum,vlan_mtu,vlan_hwtagging></up,broadcast,running,promisc,simplex,multicast></full-duplex></rxcsum,txcsum,vlan_mtu,vlan_hwtagging></up,broadcast,running,simplex,multicast></full-duplex></rxcsum,txcsum,vlan_mtu></up,broadcast,running,promisc,simplex,multicast></full-duplex></rxcsum,txcsum,vlan_mtu></up,broadcast,running,promisc,simplex,multicast>

      both servers can ping each other, the gateway & the virtual ip on the lan,wan, carp & mgmt networks. per /cf/conf/config.xml both servers have matching passwords for carp

      Also, If I select "disable carp" , why does it not stay disabled?

      1 Reply Last reply Reply Quote 0
      • S
        sullrich
        last edited by

        Check that the vhids are the same on the backup and master for the CARP'd ip's.  Also verify that pfsync is working, you can ping the dedicated interface that hosts pfsync, etc.

        Oh, I notice you don't have a dedicated pfsync interface?    Check out the tutorial, you are not doing this like we suggest.

        1 Reply Last reply Reply Quote 0
        • E
          egarnel
          last edited by

          vhids match.    I ran thru the tutorial.  I noticed on slide 18 under CARP settings that it mentions enabling preemption.  I do not have a check box for preemption or load balancing.  I am running RC2h on the primary & RC2 on the secondary.

          The interface labeled carp is the sync interface on both servers

          1 Reply Last reply Reply Quote 0
          • S
            sullrich
            last edited by

            pfsync0: flags=41 <up,running>mtu 1348
                    pfsync: syncdev: lo0 maxupd: 128

            You have not set your pfsync interface in the carp settings area.</up,running>

            1 Reply Last reply Reply Quote 0
            • E
              egarnel
              last edited by

              hmmmm.  The secondary lan interface goes into backup for a few seconds then reverts back to master. thanks for all your help in this matter

              Primary
              bge1: flags=8943 <up,broadcast,running,promisc,simplex,multicast>mtu 1500
                      options=1b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging>inet 192.168.4.1 netmask 0xffffff00 broadcast 192.168.4.255
                      inet6 fe80::216:36ff:fe13:a609%bge1 prefixlen 64 scopeid 0x4
                      ether 00:16:36:13:a6:09
                      media: Ethernet autoselect (1000baseTX <full-duplex>)
                      status: active

              pfsync0: flags=41 <up,running>mtu 1348
                      pfsync: syncdev: fxp0 maxupd: 128</up,running>

              carp0: flags=49 <up,loopback,running>mtu 1500
                      inet 192.168.4.3 netmask 0xffffff00
                      carp: MASTER vhid 1 advbase 1 advskew 0
              carp1: flags=49 <up,loopback,running>mtu 1500
                      inet bbb.yyy.xxx.25 netmask 0xffffff00
                      carp: MASTER vhid 2 advbase 1 advskew 0
              carp2: flags=49 <up,loopback,running>mtu 1500
                      inet 192.168.20.4 netmask 0xffffff00
                      carp: MASTER vhid 3 advbase 1 advskew 0

              Secondary
              bge1: flags=8943 <up,broadcast,running,promisc,simplex,multicast>mtu 1500
                      options=1b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging>inet 192.168.4.1 netmask 0xffffff00 broadcast 192.168.4.255
                      inet6 fe80::216:36ff:fe13:a609%bge1 prefixlen 64 scopeid 0x4
                      ether 00:16:36:13:a6:09
                      media: Ethernet autoselect (1000baseTX <full-duplex>)
                      status: active
              pfsync0: flags=41 <up,running>mtu 1348
                      pfsync: syncdev: bge1 maxupd: 128</up,running>

              lo0: flags=8049 <up,loopback,running,multicast>mtu 16384
                      inet 127.0.0.1 netmask 0xff000000
                      inet6 ::1 prefixlen 128
                      inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
              pflog0: flags=100 <promisc>mtu 33208
              carp0: flags=49 <up,loopback,running>mtu 1500
                      inet 192.168.4.3 netmask 0xffffff00
                      carp: BACKUP vhid 1 advbase 1 advskew 100
              carp1: flags=49 <up,loopback,running>mtu 1500
                      inet bbb.xxx.yyy.25 netmask 0xffffff00
                      carp: BACKUP vhid 2 advbase 1 advskew 100
              carp2: flags=49 <up,loopback,running>mtu 1500
                      inet 192.168.20.4 netmask 0xffffff00
                      carp: MASTER vhid 3 advbase 1 advskew 254</up,loopback,running></up,loopback,running></up,loopback,running></promisc></up,loopback,running,multicast></full-duplex></rxcsum,txcsum,vlan_mtu,vlan_hwtagging></up,broadcast,running,promisc,simplex,multicast></up,loopback,running></up,loopback,running></up,loopback,running></full-duplex></rxcsum,txcsum,vlan_mtu,vlan_hwtagging></up,broadcast,running,promisc,simplex,multicast>

              1 Reply Last reply Reply Quote 0
              • S
                sullrich
                last edited by

                on bge1, both machines need a unique ip in the  same subnet.

                1 Reply Last reply Reply Quote 0
                • E
                  egarnel
                  last edited by

                  my bad… I copied & pasted the same info twice.
                  Here is the correct info for the primary carp interface:

                  fxp0: flags=8943 <up,broadcast,running,promisc,simplex,multicast>mtu 1500
                          options=b <rxcsum,txcsum,vlan_mtu>inet 192.168.4.2 netmask 0xffffff00 broadcast 192.168.4.255
                          inet6 fe80::202:b3ff:febb:9776%fxp0 prefixlen 64 scopeid 0x4
                          ether 00:02:b3:bb:97:76
                          media: Ethernet autoselect (100baseTX <full-duplex>)
                          status: active</full-duplex></rxcsum,txcsum,vlan_mtu></up,broadcast,running,promisc,simplex,multicast>

                  1 Reply Last reply Reply Quote 0
                  • S
                    sullrich
                    last edited by

                    Then I am not sure.  It looks okay to me.  Double check your switch ports, vlans, etc.

                    Also refer to the archive, a lot of people have asked a LOT of questions about CARP.

                    1 Reply Last reply Reply Quote 0
                    • E
                      egarnel
                      last edited by

                      thanks for your help.

                      Just for grins, I am going to try to swap out the nic on the primary server. It is an intel 10/100 pro pci  plugged into a PCI-X slot. It should work fine and appears to be, but I have a PCI-X nic card available that came from another dell poweredge server that would be consistent with the other nics and also provide 1000 Mbs.

                      1 Reply Last reply Reply Quote 0
                      • N
                        Numbski
                        last edited by

                        Couple of thoughts from one of the people that has asked a LOT of questions. :)

                        1.  Have you looked at the wiki?
                        2.  VHID's need to be unique not just for that set of carp interfaces, but for anything else that might be on that same network segment, as CARP is broadcast, not a peer-to-peer technology.
                        3.  Triple check that you are using the correct subnet mask for your CARP IP's, and not /32.
                        4.  Make sure that your CARP interface has an allow any any statement on it, just to be safe, and that the systems are connected by a crossover cable for additional security.
                        5.  If you must do carp without a dedicated interface, make sure you have an allow statement for CARP and pfSync from the opposing system, and block traffic from any other hosts.

                        That's about all I can think of.

                        1 Reply Last reply Reply Quote 0
                        • E
                          egarnel
                          last edited by

                          I have done all that you mention.  I am using a dedicated interface for carp.  Both carp interfaces are connected via the same vlan and xmlrpc updates are successful.  I have not had the chance to swap out the nic for a pci-x nic yet, but I will start with a fresh install when I do. I will have to wait until the next maintenance window

                          1 Reply Last reply Reply Quote 0
                          • First post
                            Last post
                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.