States not syncing (2.2.6 & 2.3.2_1)

aventrax

Hello,

I have 2 different pfsense CARP installations, both virtual. Le pfSense VMs are running one-per-node, vmware-vmware (2.2.6) and the vmware-xen (2.3.2_1)
All the VMs have e1000 virtual NICs added with the same order (em0->LAN, em1->WAN, em2->WAN2, em3->SYNC), and the SYNC interface is on a /30 network.

My states are not syncronized: this is because shutting down the master drops all the connections, but I can also see a different state count on the dashboard (2/3000 on the master and only 30/50 on the backup node).

First question: On the dashboard of a working system, is the master's states counter supposed to be very close to the one on the backup node? Looking at this is the right thing to do in case of issue on the states syncronization? Should I look somewhere else?

Second question: On the logs I can see the syncronization were successfully, is this enough?

Dec 1 17:55:47 192.168.16.254 php-fpm[17980]: /rc.filter_synchronize: Beginning XMLRPC sync to https://172.29.97.110:8081.
Dec 1 17:55:47 192.168.16.254 php-fpm[17980]: /rc.filter_synchronize: XMLRPC sync successfully completed with https://172.29.97.110:8081.
Dec 1 17:55:52 192.168.16.254 php-fpm[17980]: /rc.filter_synchronize: Filter sync successfully completed with https://172.29.97.110:8081

Can you help me? What can it be? The CARP setup is working perfectly expecting this issue.
Obviously my vSwitch are configured to accept "promiscuous/forged/etc packets"..

Thanks a lot!

dotdash

Due to underlying OS changes, both nodes of a cluster must be running 2.2.x or 2.3.x to sync states. You cannot sync states between 2.2 and 2.3

aventrax

Thanks for your help, but I didn't mixed any version.
I have 2 couple of CARP, the first with 2.2.6 and the second one with 2.3.2_1, both have the same issue.

dotdash

Looking at two of mine, the master has 20+k and the backup has 19+k, so not exact, but close. HA settings? Should have sync states checked on both, sync int selected, peer ip the other box on the sync net.

Derelict

Are you sure it's the states not syncing and not something like not using CARP VIPs as client default gateways and for outbound NAT?

Those log entries you see are for XMLRPC (config sync), not pfsync (state sync).

What are your filter rules on the sync interfaces?

As dotdash suggested, looking at the state table size is a good way to generally validate that states are syncing.

aventrax

Sorry, my mistake.

I missed one thing clearly written on the ufficial guide: the states syncronization MUST be enabled on the slave node too!
After enabling this everything workey, now my OpenVPN/SSH connections remain up&running even if I shutdown the primary node, pretty impressive :)

Thanks all for your help!