Answered here https://forum.pfsense.org/index.php?topic=87813.msg483500#msg483500

This really should be in the upgrade notes.

@Guldil:

https://doc.pfsense.org/index.php/UpgradeGuide#Upgrading_High_Availability_Deployments

Nothing about 2.1.5 to 2.2

I'll try the procedure like i did before (slave first, carp off on master, then master).

I've just finished upgrading my CARP set based on 2 DELL R220II as recommended there and at the end it works great.
However, upgrading the backup box at first, messed my outbound NAT settings. Automatic mode was activated and therefore it used the WAN address instead of CARP VIP. In result, connections which were made from inside to web services secured by IP authorization were rejected.

Upgrade of master worked as expected.
I am happy now.  :)

It still works the same in regards to multicast. The fact it doesn't have an additional interface isn't really of any functional consequence from a user's perspective.

No difference on 2.2

Split DNS is the better fix.

Ok, I see uniqids got introduced in https://github.com/pfsense/pfsense/commit/89f171b052fbe72aed654d2a1c3d5a24e9bf9902

hmmm… Need to stop tinkering with this since its beyond my understanding. For sure I thought uniqid should show up in config.xml for CARP VIPs but maybe some sort of magic is going on behind the scenes someplace.

-Shahid

That was the problem.
However, it was actually a virtual IP address on the second pfSense box.
CARP was not configured yet.

Thank you.

@dark.fibre:

The sync-NICs are connected with a bridge cable, they can ping each other, IP are 192.168.0.1 and 192.168.0.2.

Second FW has a rule at Sync for TCP/UDP Port 443, Destination: WAN-ADRESS

What is my mistake?

If you have separated sync-NICs, why do you allow traffic to WAN address for syncing?

Just add a rule on both boxes on sync interface to allow traffic from any to any and it will be done.
The sync packets uses pfsync protocol, not TCP nor UDP!

You are right - it was my own issue. I hadn't taken an old test system offline. It was not working properly and only connected to the network intermittently causing the issue. I hate it when people don't post their answers to problems so even though I'm "late" I'm hoping that's better than "never".
Thanks again!
m

I believe the issue may be due to interrupts. From watching top (top -P CC), I see that the interrupts are only hitting one CPU. Is there a way to balance that load across both CPU's?

last pid: 79130;  load averages:  0.33,  0.22,  0.15                                                            up 85+19:36:44  13:05:01 38 processes:  1 running, 37 sleeping CPU 0:  0.0% user,  0.0% nice,  0.0% system, 54.3% interrupt, 45.7% idle CPU 1:  0.0% user,  0.0% nice,  1.1% system,  0.0% interrupt, 98.9% idle Mem: 84M Active, 32M Inact, 281M Wired, 1336K Cache, 91M Buf, 3524M Free

If your status is correct, all master on the primary, all backup on the secondary, and you're hitting the secondary when going to the CARP IP, that's a problem with your switch(es) on that interface. They're sending the CARP MAC to the wrong device. The advertisements from the master system update switch CAM tables so things go to the correct port, and that's not being handled properly on your switch for some reason.

yes, that's what I meant.

Thanks,

Thanks again to jimp for putting me on the right path.  :D

that did it!  Thank you very much for the help!