Transitioning to CARP on live system

clarknova

2.2.6-RELEASE (amd64) on VM

I want to change two things about this pfSense VM:

Update it to the latest pfSense
Create a second pfSense VM and configure them for HA/CARP, using the existing IP addresses as CARP VIP

I've read up on HA and it looks straightforward enough. I have a plan, but I'm trying to determine whether I can accomplish this without interruption to the network. The high-level view of the plan looks like this:

Configure sync on live VM without enabling it
Clone existing VM (A) and power on new VM (B) with disconnected interfaces
Configure interfaces on B including CARP VIPs, sync and AON
Connect interfaces on B, enable sync
Enable sync on A

At this point sync is enabled on both firewalls but B is using CARP VIPs and A is using the VIPs as primary. Does this break something or can I put A in HA maintenance mode now to have B promoted to primary and continue configuring A?

Am I on the right track or is there a better way to do this? Can it be done without interruption to the attached networks or is down time inevitable?

SteveITS

Putting an HA router in CARP maintenance just means that it starts advertising itself at a higher number and thus is not the active node. Since you'll have to switch IP addresses to make the current WAN/LAN IPs the CARP IPs, there's not much way around downtime. Since it's a VM can you create two new ones and configure, then turn the live one off and connect those? Technically you can create or reconfigure the first one, and then set up the backup.

Also read the recent threads on 2.4.5 in a VM with more than 1 CPU core. Might want to go to 2.4.4 instead.

clarknova

Is there no way to run pfsync without first configuring CARP on both systems then?

Maybe an alternative approach would be to reconfigure all of the interfaces' primary and CARP VIPs at once on firewall A, then hit the Apply button after all the changes are queued. I don't know if it's possible to queue primary and CARP IP assignments on multiple interfaces and then apply them in one shot, or if the state table would survive such a change, but I would expect the network interruption to be on the order of a few seconds while the IP address changes are applied. That might be the least-disruptive approach if it can be done that way, then turn on HA sync.

I could create two new VMs and cut over as you suggested, but there's still an interruption, even if brief, and state table would be wiped out. Downtime is ok as long as we plan for it. No downtime is better when it's an option.

Thanks for the heads-up on the recent threads about multi-core. I was just reading that.

SteveITS

What if you made a backup of A, then changed the IP addresses on A and set up the CARP IPs (only on A). At that point it should be working correct? With devices using the now-CARP LAN IP? Then set up B. If something doesn't go right, revert to the backup...

clarknova

Yes, I think that makes the most sense. Change outbound NAT address from interface to CARP IP, then replace all primary IPs and create old primary as CARP VIP; apply all at once. Outage should be brief to none, then firewall B can be configured as backup.