4. carp preempt problem - only the fault interface vip switches

carp preempt problem - only the fault interface vip switches

  • S

    hi guys, great to be here.
    actually, i'm testing pfsense with carp and pfsync.
    so i setup a couple of proxmox kvm machines with bridges, not connected to any physical interface.
    if i reboot the master, the backup take over ... so everything is fine.

    what i did not find, if i set an interface down:

    • carp event MASTER -> INIT (interface down)
    • carp demoted by 240 to 240
    • link changed to down
    • vhid for this interface goes BACKUP
    • other site vhid for this interface goes MASTER
    • on MASTER the vhid stuck in INIT

    so i have a split situation, one machine WAN MASTER, other LAN MASTER.

    i found a lot of old stuff in the forum, but none of the solutions worked for me.
    i tried virtio vs e1000 and played with the tunables.

    do you have any ideas?

    thanks a lot and kind regards,
    ronny

    0
  • LAYER 8 Moderator

    @spark5gmbh said in carp preempt problem - only the fault interface vip switches:

    vhid for this interface goes BACKUP
    other site vhid for this interface goes MASTER
    on MASTER the vhid stuck in INIT

    so i have a split situation, one machine WAN MASTER, other LAN MASTER.

    Can you show what you mean with a screenshot? I don't understand it. First you write the Master is stuck in Init (not Master) and below you say it's Master/Master`?

    0
  • S

    hi ... here they are
    one machine is wan master and the other is lan master, after i unplugged the wan link.

    pfsense_master.png
    pfsense_backup.png

    0
  • LAYER 8 Moderator

    If you temp. disable CARP on the secondary machine and re-enable it -> does it still go to master mode? Can both nodes ping each other on the LAN or is there a communication problem?

    0
  • S

    hi ... if i do so, state will be like before.
    init and only the fault ip switch.

    i do not see any communication problems.

    ronny

    0
  • LAYER 8 Moderator

    Did you check the documentation about Proxmox setups and kinks? That looks like a VM issue. WAN shouldn't be backup on the secondary as the primary is in init. Also Vodafone should failover, too. And LAN looks like it has a comm problem, as it is in split-brain.

    0
  • S

    hi ... yes i checked up documentation and did all the thinks.
    i also thought, there is something with the communication, but i did not find anything.

    now i setup opnsense with the same steps and klicks ... and everything is working.
    after that, i went back to pfsense and the problem is still there.

    do you know the differences between pfsense and opnsense ... i think there is something with the network/kvm/proxmox.

    i am running out of ideas ... thank you,
    ronny

    0
  • LAYER 8 Netgate

    This is what I get when I mark LAN as disconnected under proxmox:

    Sep 23 19:33:52 	kernel 		carp: 236@vtnet0: MASTER -> INIT (hardware interface down)
Sep 23 19:33:52 	kernel 		carp: demoted by 240 to 240 (interface down)
Sep 23 19:33:52 	kernel 		carp: 239@vtnet0: MASTER -> INIT (hardware interface down)
Sep 23 19:33:52 	kernel 		carp: demoted by 240 to 480 (interface down)
Sep 23 19:33:52 	kernel 		vtnet0: link state changed to DOWN
Sep 23 19:33:52 	kernel 		carp: 241@vtnet4: MASTER -> BACKUP (more frequent advertisement received)
Sep 23 19:33:52 	kernel 		vtnet4: deletion failed: 3
Sep 23 19:33:52 	kernel 		carp: 238@vtnet1: MASTER -> BACKUP (more frequent advertisement received)
Sep 23 19:33:52 	kernel 		carp: 228@vtnet1: MASTER -> BACKUP (more frequent advertisement received)
Sep 23 19:33:52 	kernel 		vtnet1: deletion failed: 3
Sep 23 19:33:52 	kernel 		vtnet1: deletion failed: 3
Sep 23 19:33:52 	kernel 		vtnet1: deletion failed: 3
Sep 23 19:33:52 	kernel 		vtnet1: deletion failed: 3
Sep 23 19:33:52 	kernel 		vtnet1: deletion failed: 3
Sep 23 19:33:52 	kernel 		carp: 240@vtnet2: MASTER -> BACKUP (more frequent advertisement received)
Sep 23 19:33:52 	kernel 		carp: 237@vtnet2: MASTER -> BACKUP (more frequent advertisement received)
Sep 23 19:33:52 	kernel 		vtnet2: deletion failed: 3
Sep 23 19:33:52 	check_reload_status 	62776 	Carp backup event
Sep 23 19:33:52 	check_reload_status 	62776 	Carp backup event
Sep 23 19:33:52 	check_reload_status 	62776 	Linkup starting $vtnet0
Sep 23 19:33:53 	check_reload_status 	62776 	Carp backup event
Sep 23 19:33:53 	check_reload_status 	62776 	Carp backup event
Sep 23 19:33:53 	check_reload_status 	62776 	Carp backup event
Sep 23 19:33:53 	check_reload_status 	62776 	Carp backup event
Sep 23 19:33:53 	check_reload_status 	62776 	Carp backup event
Sep 23 19:33:54 	check_reload_status 	62776 	Reloading filter
Sep 23 19:33:54 	php-fpm 	56075 	/rc.carpbackup: HA cluster member "(172.25.228.140@vtnet1): (WAN)" has resumed CARP state "BACKUP" for vhid $238
Sep 23 19:33:54 	php-fpm 	60328 	/rc.carpbackup: HA cluster member "(172.25.228.140@vtnet1): (WAN)" has resumed CARP state "BACKUP" for vhid $228
Sep 23 19:33:55 	php-fpm 	56075 	/rc.carpbackup: HA cluster member "(172.25.228.65@vtnet1): (WAN)" has resumed CARP state "BACKUP" for vhid $238
Sep 23 19:33:55 	php-fpm 	60328 	/rc.carpbackup: HA cluster member "(172.25.228.65@vtnet1): (WAN)" has resumed CARP state "BACKUP" for vhid $228
Sep 23 19:33:55 	php-fpm 	56075 	/rc.carpbackup: HA cluster member "(172.25.228.66@vtnet1): (WAN)" has resumed CARP state "BACKUP" for vhid $238
Sep 23 19:33:55 	php-fpm 	60328 	/rc.carpbackup: HA cluster member "(172.25.228.66@vtnet1): (WAN)" has resumed CARP state "BACKUP" for vhid $228
Sep 23 19:33:55 	php-fpm 	56075 	/rc.carpbackup: HA cluster member "(172.25.228.67@vtnet1): (WAN)" has resumed CARP state "BACKUP" for vhid $238
Sep 23 19:33:56 	php-fpm 	60328 	/rc.carpbackup: HA cluster member "(172.25.228.67@vtnet1): (WAN)" has resumed CARP state "BACKUP" for vhid $228

    Everything works as expected and the secondary takes over. Everything on the Primary is BACKUP or INIT and everything on the secondary is MASTER. Did you change the advbase and advskew values? What is in syslog when you disconnect an interface? What happens when you enter CARP maintenance mode on the primary?

    0
  • S

    hi and thanks,

    that is, what i expect.
    i see in the logs, that the unplugged interface is marked as down and the carp vhid is changing, nothing else.

    i will try to start from an new, fresh installation an test again.
    tell you later ...

    ronny

    0
  • S

    hi ... i setup a fresh installation and it works now ... dont know why.
    i put all the interfaces in one group and allow all traffic from any to any.

    so maybe there was an firewall problem, but i did not see any dropped packages.

    could you, please, tell me, what ports are necessary for carp and xml sync?

    thanks,
    ronny

    0
  • LAYER 8 Netgate

    What firewall where? pfSense will pass everything you need for CARP on CARP interfaces. XMLRPC sync goes over the same port as the webgui.

    0
  • S

    hi ... so, there is no need to configure any firewall rules ... ok, i will test and tell you later

    thanks,
    ronny

    0
  • LAYER 8 Netgate

    https://docs.netgate.com/pfsense/en/latest/book/highavailability/example-redundant-configuration.html#setup-sync-interface

    0
Log in to reply
 

Products

Services

Support

News

Resources

Company

Our Mission

We provide leading-edge network security at a fair price - regardless of organizational size or network sophistication. We believe that an open-source security model offers disruptive pricing along with the agility required to quickly address emerging threats.

Subscribe to our Newsletter

Product information, software announcements, and special offers. See our newsletter archive for past announcements.

© Copyright 2002 - 2019 Rubicon Communications, LLC | Privacy Policy