CARP corruption



  • CARP for my LAN interface was / has been running fine. Today, I setup a new vlan to test CARP for wifi.
    Somehow, after the CARP setup, I noticed that master and backup both showed the new VIP as master.
    So, in troubleshooting, I ended up removing everything related to the VIP, even the interface for the
    new vlan and recreate the vlan interface.

    But, as of this writing, after recreating the vlans, I still cannot ping the interface IP on the pfsense2 (backup).
    the FW rules are in-sync for the interface, so, should be no issue there.

    Is there a master config file for CARP that I need to delete the settings from on pfsense2?

    pfsense1 (master): vlan interface ip 172.31.99.21
    ^ pings fine
    pfsense2 (backup): vlan interface ip 172.31.99.22
    ^ cannot ping since CARP configuration
    VIP for vlan: 172.31.99.1

    The captures below are from after the VIP has been removed.

    PACKET CAPTURE on pfsense1 (master when the VIP was configured)
    00:39:23.187831 ARP, Request who-has 172.31.99.22 tell 172.31.99.21, length 28
    00:39:28.187479 ARP, Request who-has 172.31.99.22 tell 172.31.99.21, length 28

    PACKET CAPTURE on pfsense2 (should have been backup when the VIP was configured)
    00:40:17.093910 ARP, Request who-has 172.31.99.21 tell 172.31.99.101, length 46
    00:40:18.094080 ARP, Request who-has 172.31.99.21 tell 172.31.99.101, length 46
    00:40:18.109910 ARP, Request who-has 172.31.99.1 tell 172.31.99.101, length 46
    00:40:19.094036 ARP, Request who-has 172.31.99.1 tell 172.31.99.101, length 46

    ^ 172.31.99.101 is my workstation on the same vlan pinging

    00:40:19.104109 ARP, Request who-has 172.31.99.21 tell 172.31.99.101, length 46
    00:40:20.093943 ARP, Request who-has 172.31.99.1 tell 172.31.99.101, length 46
    00:40:20.094069 ARP, Request who-has 172.31.99.21 tell 172.31.99.101, length 46
    00:40:21.093823 ARP, Request who-has 172.31.99.21 tell 172.31.99.101, length 46
    00:40:21.454727 ARP, Request who-has 172.31.99.1 tell 172.31.99.101, length 46
    00:40:22.093863 ARP, Request who-has 172.31.99.1 tell 172.31.99.101, length 46
    00:40:22.103879 ARP, Request who-has 172.31.99.21 tell 172.31.99.101, length 46
    00:40:22.621053 IP 172.31.99.101.49985 > 239.255.255.250.1900: UDP, length 173
    00:40:23.093904 ARP, Request who-has 172.31.99.1 tell 172.31.99.101, length 46
    00:40:23.094017 ARP, Request who-has 172.31.99.21 tell 172.31.99.101, length 46
    00:40:23.625441 IP 172.31.99.101.49985 > 239.255.255.250.1900: UDP, length 173
    00:40:24.093854 ARP, Request who-has 172.31.99.21 tell 172.31.99.101, length 46
    00:40:24.641005 IP 172.31.99.101.49985 > 239.255.255.250.1900: UDP, length 173
    00:40:25.104748 ARP, Request who-has 172.31.99.21 tell 172.31.99.101, length 46
    00:40:25.501295 ARP, Request who-has 172.31.99.1 tell 172.31.99.101, length 46
    00:40:25.656857 IP 172.31.99.101.49985 > 239.255.255.250.1900: UDP, length 173
    00:40:26.093777 ARP, Request who-has 172.31.99.21 tell 172.31.99.101, length 46
    00:40:26.093949 ARP, Request who-has 172.31.99.1 tell 172.31.99.101, length 46
    00:40:27.093832 ARP, Request who-has 172.31.99.21 tell 172.31.99.101, length 46
    00:40:27.094017 ARP, Request who-has 172.31.99.1 tell 172.31.99.101, length 46
    00:40:28.105223 ARP, Request who-has 172.31.99.21 tell 172.31.99.101, length 46
    00:40:28.109504 ARP, Request who-has 172.31.99.1 tell 172.31.99.101, length 46
    00:40:29.093823 ARP, Request who-has 172.31.99.21 tell 172.31.99.101, length 46
    00:40:29.093925 ARP, Request who-has 172.31.99.1 tell 172.31.99.101, length 46
    00:40:30.093828 ARP, Request who-has 172.31.99.21 tell 172.31.99.101, length 46
    00:40:30.094024 ARP, Request who-has 172.31.99.1 tell 172.31.99.101, length 46
    00:40:31.103989 ARP, Request who-has 172.31.99.21 tell 172.31.99.101, length 46

    System log on pfsense2 (backup) when the VIP was created
    Jan 11 23:59:05 check_reload_status Carp backup event
    Jan 11 23:53:51 php-fpm 342 /interfaces.php: Configuring CARP settings finalize...
    Jan 11 23:53:23 php-fpm 93187 /rc.carpmaster: HA cluster member "(172.31.99.1@vmx3.25): (INT_TESTWIF1)" has resumed CARP state "MASTER" for vhid 25
    Jan 11 23:53:22 check_reload_status Carp master event
    Jan 11 23:53:22 kernel carp: 25@vmx3.25: BACKUP -> MASTER (master timed out)
    Jan 11 23:53:20 php-fpm 93187 /rc.carpbackup: HA cluster member "(172.31.99.1@vmx3.25): (INT_TESTWIF1)" has resumed CARP state "BACKUP" for vhid 25
    Jan 11 23:53:20 php-fpm 93187 /rc.carpbackup: HA cluster member "(172.31.99.1@vmx3.25): (INT_TESTWIF1)" has resumed CARP state "BACKUP" for vhid 25
    Jan 11 23:53:19 php-fpm 341 /interfaces.php: Configuring CARP settings finalize...
    Jan 11 23:53:19 kernel carp: 25@vmx3.25: INIT -> BACKUP (initialization complete)
    Jan 11 23:53:19 check_reload_status Carp backup event
    Jan 11 23:53:19 kernel carp: 25@vmx3.25: MASTER -> INIT (hardware interface up)
    Jan 11 23:53:19 check_reload_status Carp backup event
    Jan 11 23:52:56 php-fpm 93187 /rc.carpmaster: HA cluster member "(172.31.99.1@vmx3.25): (INT_TESTWIF1)" has resumed CARP state "MASTER" for vhid 25


Log in to reply