That was the problem. However, it was actually a virtual IP address on the second pfSense box. CARP was not configured yet. Thank you.

@dark.fibre: The sync-NICs are connected with a bridge cable, they can ping each other, IP are 192.168.0.1 and 192.168.0.2. Second FW has a rule at Sync for TCP/UDP Port 443, Destination: WAN-ADRESS What is my mistake? If you have separated sync-NICs, why do you allow traffic to WAN address for syncing? Just add a rule on both boxes on sync interface to allow traffic from any to any and it will be done. The sync packets uses pfsync protocol, not TCP nor UDP!

You are right - it was my own issue. I hadn't taken an old test system offline. It was not working properly and only connected to the network intermittently causing the issue. I hate it when people don't post their answers to problems so even though I'm "late" I'm hoping that's better than "never". Thanks again! m

I believe the issue may be due to interrupts. From watching top (top -P CC), I see that the interrupts are only hitting one CPU. Is there a way to balance that load across both CPU's? last pid: 79130;  load averages:  0.33,  0.22,  0.15                                                            up 85+19:36:44  13:05:01 38 processes:  1 running, 37 sleeping CPU 0:  0.0% user,  0.0% nice,  0.0% system, 54.3% interrupt, 45.7% idle CPU 1:  0.0% user,  0.0% nice,  1.1% system,  0.0% interrupt, 98.9% idle Mem: 84M Active, 32M Inact, 281M Wired, 1336K Cache, 91M Buf, 3524M Free

If your status is correct, all master on the primary, all backup on the secondary, and you're hitting the secondary when going to the CARP IP, that's a problem with your switch(es) on that interface. They're sending the CARP MAC to the wrong device. The advertisements from the master system update switch CAM tables so things go to the correct port, and that's not being handled properly on your switch for some reason.

yes, that's what I meant. Thanks,

Thanks again to jimp for putting me on the right path.  :D

that did it!  Thank you very much for the help!

Thus when the "master" is down, ether don't do any change to slave, or if there is change to slave, we have to manually update the master when it is up I guess? I tried to do the "bold" but it gave error on the "slave" immediately saying something like it does find the target as a valid sync device.

No suggestions? It cant be network traffic on Sync interface since its a direct connect. Not sure where to look and how to fix this. What the correct procedure to swing  the interface back to primary? Last time I just rebooted the secondary but it seems a bit crude.

I created another vlan (12) today, and have the exact same issue. I can ping the backup-carp ip, but not that master nor the virtual. The firewall rules are empty, so nothing should be allowed. Any help or ideas are very welcome!

That's still a bit vague as to what they actually meant. They might have meant that you should have multiple ISPs not just that one, so that you could have some redundancy in case it fails. To say more you'll need to have them clarify the original statement.

No.