Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    carp preempt problem - only the fault interface vip switches

    HA/CARP/VIPs
    3
    13
    1.4k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      spark5gmbh
      last edited by

      hi guys, great to be here.
      actually, i'm testing pfsense with carp and pfsync.
      so i setup a couple of proxmox kvm machines with bridges, not connected to any physical interface.
      if i reboot the master, the backup take over ... so everything is fine.

      what i did not find, if i set an interface down:

      • carp event MASTER -> INIT (interface down)
      • carp demoted by 240 to 240
      • link changed to down
      • vhid for this interface goes BACKUP
      • other site vhid for this interface goes MASTER
      • on MASTER the vhid stuck in INIT

      so i have a split situation, one machine WAN MASTER, other LAN MASTER.

      i found a lot of old stuff in the forum, but none of the solutions worked for me.
      i tried virtio vs e1000 and played with the tunables.

      do you have any ideas?

      thanks a lot and kind regards,
      ronny

      1 Reply Last reply Reply Quote 0
      • JeGrJ
        JeGr LAYER 8 Moderator
        last edited by

        @spark5gmbh said in carp preempt problem - only the fault interface vip switches:

        vhid for this interface goes BACKUP
        other site vhid for this interface goes MASTER
        on MASTER the vhid stuck in INIT

        so i have a split situation, one machine WAN MASTER, other LAN MASTER.

        Can you show what you mean with a screenshot? I don't understand it. First you write the Master is stuck in Init (not Master) and below you say it's Master/Master`?

        Don't forget to upvote 👍 those who kindly offered their time and brainpower to help you!

        If you're interested, I'm available to discuss details of German-speaking paid support (for companies) if needed.

        1 Reply Last reply Reply Quote 0
        • S
          spark5gmbh
          last edited by

          hi ... here they are
          one machine is wan master and the other is lan master, after i unplugged the wan link.

          pfsense_master.png
          pfsense_backup.png

          1 Reply Last reply Reply Quote 0
          • JeGrJ
            JeGr LAYER 8 Moderator
            last edited by

            If you temp. disable CARP on the secondary machine and re-enable it -> does it still go to master mode? Can both nodes ping each other on the LAN or is there a communication problem?

            Don't forget to upvote 👍 those who kindly offered their time and brainpower to help you!

            If you're interested, I'm available to discuss details of German-speaking paid support (for companies) if needed.

            1 Reply Last reply Reply Quote 0
            • S
              spark5gmbh
              last edited by

              hi ... if i do so, state will be like before.
              init and only the fault ip switch.

              i do not see any communication problems.

              ronny

              1 Reply Last reply Reply Quote 0
              • JeGrJ
                JeGr LAYER 8 Moderator
                last edited by

                Did you check the documentation about Proxmox setups and kinks? That looks like a VM issue. WAN shouldn't be backup on the secondary as the primary is in init. Also Vodafone should failover, too. And LAN looks like it has a comm problem, as it is in split-brain.

                Don't forget to upvote 👍 those who kindly offered their time and brainpower to help you!

                If you're interested, I'm available to discuss details of German-speaking paid support (for companies) if needed.

                1 Reply Last reply Reply Quote 0
                • S
                  spark5gmbh
                  last edited by

                  hi ... yes i checked up documentation and did all the thinks.
                  i also thought, there is something with the communication, but i did not find anything.

                  now i setup opnsense with the same steps and klicks ... and everything is working.
                  after that, i went back to pfsense and the problem is still there.

                  do you know the differences between pfsense and opnsense ... i think there is something with the network/kvm/proxmox.

                  i am running out of ideas ... thank you,
                  ronny

                  1 Reply Last reply Reply Quote 0
                  • DerelictD
                    Derelict LAYER 8 Netgate
                    last edited by Derelict

                    This is what I get when I mark LAN as disconnected under proxmox:

                    Sep 23 19:33:52 	kernel 		carp: 236@vtnet0: MASTER -> INIT (hardware interface down)
Sep 23 19:33:52 	kernel 		carp: demoted by 240 to 240 (interface down)
Sep 23 19:33:52 	kernel 		carp: 239@vtnet0: MASTER -> INIT (hardware interface down)
Sep 23 19:33:52 	kernel 		carp: demoted by 240 to 480 (interface down)
Sep 23 19:33:52 	kernel 		vtnet0: link state changed to DOWN
Sep 23 19:33:52 	kernel 		carp: 241@vtnet4: MASTER -> BACKUP (more frequent advertisement received)
Sep 23 19:33:52 	kernel 		vtnet4: deletion failed: 3
Sep 23 19:33:52 	kernel 		carp: 238@vtnet1: MASTER -> BACKUP (more frequent advertisement received)
Sep 23 19:33:52 	kernel 		carp: 228@vtnet1: MASTER -> BACKUP (more frequent advertisement received)
Sep 23 19:33:52 	kernel 		vtnet1: deletion failed: 3
Sep 23 19:33:52 	kernel 		vtnet1: deletion failed: 3
Sep 23 19:33:52 	kernel 		vtnet1: deletion failed: 3
Sep 23 19:33:52 	kernel 		vtnet1: deletion failed: 3
Sep 23 19:33:52 	kernel 		vtnet1: deletion failed: 3
Sep 23 19:33:52 	kernel 		carp: 240@vtnet2: MASTER -> BACKUP (more frequent advertisement received)
Sep 23 19:33:52 	kernel 		carp: 237@vtnet2: MASTER -> BACKUP (more frequent advertisement received)
Sep 23 19:33:52 	kernel 		vtnet2: deletion failed: 3
Sep 23 19:33:52 	check_reload_status 	62776 	Carp backup event
Sep 23 19:33:52 	check_reload_status 	62776 	Carp backup event
Sep 23 19:33:52 	check_reload_status 	62776 	Linkup starting $vtnet0
Sep 23 19:33:53 	check_reload_status 	62776 	Carp backup event
Sep 23 19:33:53 	check_reload_status 	62776 	Carp backup event
Sep 23 19:33:53 	check_reload_status 	62776 	Carp backup event
Sep 23 19:33:53 	check_reload_status 	62776 	Carp backup event
Sep 23 19:33:53 	check_reload_status 	62776 	Carp backup event
Sep 23 19:33:54 	check_reload_status 	62776 	Reloading filter
Sep 23 19:33:54 	php-fpm 	56075 	/rc.carpbackup: HA cluster member "(172.25.228.140@vtnet1): (WAN)" has resumed CARP state "BACKUP" for vhid $238
Sep 23 19:33:54 	php-fpm 	60328 	/rc.carpbackup: HA cluster member "(172.25.228.140@vtnet1): (WAN)" has resumed CARP state "BACKUP" for vhid $228
Sep 23 19:33:55 	php-fpm 	56075 	/rc.carpbackup: HA cluster member "(172.25.228.65@vtnet1): (WAN)" has resumed CARP state "BACKUP" for vhid $238
Sep 23 19:33:55 	php-fpm 	60328 	/rc.carpbackup: HA cluster member "(172.25.228.65@vtnet1): (WAN)" has resumed CARP state "BACKUP" for vhid $228
Sep 23 19:33:55 	php-fpm 	56075 	/rc.carpbackup: HA cluster member "(172.25.228.66@vtnet1): (WAN)" has resumed CARP state "BACKUP" for vhid $238
Sep 23 19:33:55 	php-fpm 	60328 	/rc.carpbackup: HA cluster member "(172.25.228.66@vtnet1): (WAN)" has resumed CARP state "BACKUP" for vhid $228
Sep 23 19:33:55 	php-fpm 	56075 	/rc.carpbackup: HA cluster member "(172.25.228.67@vtnet1): (WAN)" has resumed CARP state "BACKUP" for vhid $238
Sep 23 19:33:56 	php-fpm 	60328 	/rc.carpbackup: HA cluster member "(172.25.228.67@vtnet1): (WAN)" has resumed CARP state "BACKUP" for vhid $228

                    Everything works as expected and the secondary takes over. Everything on the Primary is BACKUP or INIT and everything on the secondary is MASTER. Did you change the advbase and advskew values? What is in syslog when you disconnect an interface? What happens when you enter CARP maintenance mode on the primary?

                    Chattanooga, Tennessee, USA
                    A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                    DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                    Do Not Chat For Help! NO_WAN_EGRESS(TM)

                    1 Reply Last reply Reply Quote 0
                    • S
                      spark5gmbh
                      last edited by

                      hi and thanks,

                      that is, what i expect.
                      i see in the logs, that the unplugged interface is marked as down and the carp vhid is changing, nothing else.

                      i will try to start from an new, fresh installation an test again.
                      tell you later ...

                      ronny

                      1 Reply Last reply Reply Quote 0
                      • S
                        spark5gmbh
                        last edited by

                        hi ... i setup a fresh installation and it works now ... dont know why.
                        i put all the interfaces in one group and allow all traffic from any to any.

                        so maybe there was an firewall problem, but i did not see any dropped packages.

                        could you, please, tell me, what ports are necessary for carp and xml sync?

                        thanks,
                        ronny

                        1 Reply Last reply Reply Quote 0
                        • DerelictD
                          Derelict LAYER 8 Netgate
                          last edited by

                          What firewall where? pfSense will pass everything you need for CARP on CARP interfaces. XMLRPC sync goes over the same port as the webgui.

                          Chattanooga, Tennessee, USA
                          A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                          DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                          Do Not Chat For Help! NO_WAN_EGRESS(TM)

                          1 Reply Last reply Reply Quote 0
                          • S
                            spark5gmbh
                            last edited by

                            hi ... so, there is no need to configure any firewall rules ... ok, i will test and tell you later

                            thanks,
                            ronny

                            1 Reply Last reply Reply Quote 0
                            • DerelictD
                              Derelict LAYER 8 Netgate
                              last edited by

                              https://docs.netgate.com/pfsense/en/latest/book/highavailability/example-redundant-configuration.html#setup-sync-interface

                              Chattanooga, Tennessee, USA
                              A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                              DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                              Do Not Chat For Help! NO_WAN_EGRESS(TM)

                              1 Reply Last reply Reply Quote 0
                              • First post
                                Last post
                              Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.