they are aleady absolutely the same for both servers (Master and Backup) bge0 –- WAN1 bge1 --- WAN2 em0 ---- LAN em1 ---- HA

It will use however much bandwidth it needs to communicate all of the state change information required (inserts, updates, deletes). The more traffic and states you have, the higher the sync traffic bandwidth will be.

Hello, sorry for old post, but same question here.  Is there any solution for this?

Well go figure, re-configuring the sync interface to use igb4 instead of igb5, and then swapping the firewall rules assigned to the interface and hey presto, a working XMLRPC setup,  so devs…bug here hey?! tcpdump -i igb4 results: 08:19:47.313327 IP 172.16.0.3 > 172.16.0.2: PFSYNCv5 len 280     update compressed count 3     eof count 1 08:19:47.758196 IP 172.16.0.2 > 172.16.0.3: PFSYNCv5 len 280     update compressed count 3     eof count 1 08:19:48.377325 IP 172.16.0.3 > 172.16.0.2: PFSYNCv5 len 196     update compressed count 2     eof count 1

Same VHID on the CARP VIP and the VRRP? Though that should blow up with only one link due to the identical MAC addresses. I would pcap on both nodes for CARP and connect both and see what's really happening.

I found this solution here and will try it out for the next days. The symptom is not exactly the same, but it deals with Limiters and HA and is not solved. Btw. I had also a crash of the master node after those flooding messages. https://redmine.pfsense.org/issues/4310#note-44 After a few days operating in production, the solution above is working with pfsync and limiters… perfect. Tag a VLAN on the LAGG and that will support altq. OK, thanks for your advice! At the moment we do not use any VLANs…

Yeah, I came to that conclusion as well. The customer needed some persuation though…

@Topski: And I am using VMware 5.x. Can I use HA without vDS (no enterprise licenses here)? Does it work across ESXi boxes, when creating dedicated port groups for the promiscuous mode? If not using vDS, then the switch is 'per hyper visor'. AFAIK RARP advertisements appear only on the switch it is connected to. Just tested, this works fine  8) :)

Well, I know what it's not… pfSense. It's always the switching layer, bro.

In Diagnostics > Ping you can set the CARP VIP as the source address. See if you can ping the ISP gateway or things out on the internet like 8.8.8.8 when doing that. You can also use Diagnostics > Test Port to do the same thing. See if you can connect to something like www.google.com on port 443 sourcing from the CARP VIP. If either of these fail, outbound NAT using that address will very likely fail too and more investigation will be necessary. Probably packet captures to see what's really going on out on WAN where the ISP device and the CARP VIPs are concerned.

Thank you guys for the clarification!

So just one client on the network was impacted? Probably that client has something hardcoded pointed at the master and not a CARP VIP (e.g. its gateway or DNS servers)

Yes, I guess I should have done that to begin with. When things are working vs not,``` ifconfig igb1 working: inet 108.245.XXX.XXX netmask 0xfffffc00 broadcast 108.245.XXX.255 inet 192.168.10.2 netmask 0xffffff00 broadcast 192.168.10.255 The obfuscated 108.245 address is the modem's address on the internet. 192.168.10.2 is the VIP, used for modem administration as the modem is at 192.168.10.1 In this scenario, 108.245.xxx.xxx is what will be used by "WAN" for firewall rules, OpenVPN, etc "Status->Interfaces" shows this address for WAN When things go wrong, I see: inet 192.168.10.2 netmask 0xffffff00 broadcast 192.168.10.255 inet 108.245.XXX.XXX netmask 0xfffffc00 broadcast 108.245.XXX.255 "Status->Interfaces" shows 192.168.10.2 address for WAN, and things are broken.

@Derelict: All of those should be changed to the CARP VIP. Thank you! It is working now. Latest upgrade to 2.3.4-RELEASE-p1 worked fine as well. Again, thanks for your help!  :D

Hooray! Confirmed working on 2.4.0-RC

Thanks jimp for the answer. I am gonna investigate haproxy ;-)

Is this a client or a server? OpenVPN servers bound to CARP VIPs are kept shut down on whichever unit has the VIP in a BACKUP state. So it's normal for them not to be running on the secondary. When the VIP state transitions to MASTER they are started automatically.