Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Pfsense CARP switch from MASTER/BACKUP randomly

    Scheduled Pinned Locked Moved
    HA/CARP/VIPs
    4
    9
    2.7k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • P
      pfsense7515
      last edited by

      Hello,

      We have two firewalls pfsense cluster High Availability. We noticed that several times, primary pfsense switch automatically CARP from master to backup and secondary pfsense switch automatically CARP from backup to master without reason. Few seconds after reswicth on normal state. We had recently upgraded to latest stable release pfsense 2.6.0-RELEASE but problem persist. Do you have any ideas about this problem and help to us ?

      Thanks a lot by advance for your support

      Regards

      DerelictD 1 Reply Last reply Reply Quote 0
      • DerelictD
        Derelict LAYER 8 Netgate @pfsense7515
        last edited by

        @pfsense7515 Pretty much the only way for that to happen are issues at Layer 2. But that almost always means you end up with MASTER/MASTER not an actual failover.

        Is an interface with CARP addresses on it losing link? If so, it's doing what it's supposed to do.

        Chattanooga, Tennessee, USA
        A comprehensive network diagram is worth 10,000 words and 15 conference calls.
        DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
        Do Not Chat For Help! NO_WAN_EGRESS(TM)

        P 1 Reply Last reply Reply Quote 0
        • P
          pfsense7515 @Derelict
          last edited by

          @derelict

          Hello, Thank you for your feedback.
          How can I debug this problem with a monitoring tool. What do you advise me to do? ping one or more physical interfaces? or one or more VIPs?

          Thank you for your precious help

          S DerelictD 2 Replies Last reply Reply Quote 0
          • S
            SteveITS Rebel Alliance @pfsense7515
            last edited by

            @pfsense7515 Take a look at:
            https://docs.netgate.com/pfsense/en/latest/troubleshooting/high-availability.html#other-switch-and-layer-2-issues

            Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
            When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
            Upvote 👍 helpful posts!

            1 Reply Last reply Reply Quote 0
            • DerelictD
              Derelict LAYER 8 Netgate @pfsense7515
              last edited by

              @pfsense7515 Look at the logs. System log has interface down/up and CARP events.

              Chattanooga, Tennessee, USA
              A comprehensive network diagram is worth 10,000 words and 15 conference calls.
              DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
              Do Not Chat For Help! NO_WAN_EGRESS(TM)

              P 1 Reply Last reply Reply Quote 0
              • P
                pfsense7515 @Derelict
                last edited by

                @derelict

                Hello Thank you for answer. For the carp events, I received message like this below
                only by email not on system logs

                Notifications in this message: 5

                10:57:14 HA cluster member "(X.X.X.X@igb1.XXX): (VLAN1)" has resumed CARP state "MASTER" for vhid 8
                10:57:14 HA cluster member "(X.X.X.X@igb1.XXX): (VLAN2)" has resumed CARP state "MASTER" for vhid 2
                10:57:14 HA cluster member "(X.X.X.X@igb1.XXX): (VLAN3)" has resumed CARP state "MASTER" for vhid 4
                10:57:14 HA cluster member "(X.X.X.X@igb1.XXX): (VLAN4)" has resumed CARP state "MASTER" for vhid 1
                10:57:14 HA cluster member "(X.X.X.X@igb1.XXX): (VLAN5)" has resumed CARP state "MASTER" for vhid 3

                For see if there are potentially problem with physically link. What kind of message must be find in system logs please ?

                Thank You for your help at this topic

                DerelictD 1 Reply Last reply Reply Quote 1
                • DerelictD
                  Derelict LAYER 8 Netgate @pfsense7515
                  last edited by

                  @pfsense7515 Need to look at what caused that. That is only part of the event. There are also logs like links going down and up, etc.

                  Chattanooga, Tennessee, USA
                  A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                  DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                  Do Not Chat For Help! NO_WAN_EGRESS(TM)

                  E 1 Reply Last reply Reply Quote 1
                  • M marcolefo referenced this topic on
                  • M marcolefo referenced this topic on
                  • M marcolefo referenced this topic on
                  • E
                    edgarquadros @Derelict
                    last edited by

                    @Derelict I'm having a similar problem here, but curiously it happens just in only one VLAN interface.
                    I have 2 "whiteboxes" running pfSense+ 24.03 as MASTER and BACKUP and have CARP VIPs configured for some VLANs for WANs and LANs, and the problem is happening in just one of our internal VLANs.
                    Looking at the pfSense System Logs, I can't see anything related to CARP, but as the 2 pfsense boxes are configured to send me e-mails for anything, I'm receiving these notifications, every 2 minutes, sent only for the BACKUP server:

                    11:35:22 HA cluster member "(10.48.4.254@lagg0.104): (VLAN104)" has resumed CARP state "MASTER" for vhid 104
                    11:35:26 HA cluster member "(10.48.4.254@lagg0.104): (VLAN104)" has resumed CARP state "BACKUP" for vhid 104

                    Talking about the topology here, I have the 2 boxes with 4 ethernet ports each, configured as LAGG, in 2 Extreme Network switches model X440t.
                    The MASTER server has the ports igc0 and igc1 connected in ports 1 and 2 of switch 1, and the ports igc2 and igc3 connected in ports 1 and 2 of switch 2, and from the BACKUP server we have the ports igc0 and igc1 connected in ports 3 and 4 of switch 1, and the ports igc2 and igc3 connected in ports 3 and 4 of switch 2. Also, we have configured LAGG LACP on the switches side. All VLANs are configured (tagged) in these ports from both switches.
                    Looking at the switches logs, I can't find any message related to LAGG problems.

                    Attached here is the print screen from CARP VIPs config, and also the packet capture from both MASTER and BACKUP servers, filtered by CARP protocol, where I can see the advertisements normally and can't identify any reason for the problem.
                    May if you can see something that I'm not seeing...

                    CARP_VIPs.jpeg backup-server_carp-logs.txt master-server_car-logs.txt CARP_cfg_vlan104_BACKUP.jpeg CARP_cfg_vlan104_MASTER.jpeg

                    DerelictD 1 Reply Last reply Reply Quote 0
                    • DerelictD
                      Derelict LAYER 8 Netgate @edgarquadros
                      last edited by

                      @edgarquadros If it's only on one VLAN, be sure everything is tagged through your switches like the rest.

                      Chattanooga, Tennessee, USA
                      A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                      DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                      Do Not Chat For Help! NO_WAN_EGRESS(TM)

                      1 Reply Last reply Reply Quote 0
                      • First post
                        Last post