Upgrade from 2.4.4_3 to 21.02_1 sanity check

mahomed

Hi

We are looking to upgrade our pair of XG-7100 routers that are running as a failover pair. I've read through the release notes, blog posts and forum posts, but would appreciate any feedback on my intended upgrade plan. My main concern is about this warning in the upgrade notes:

The built-in relayd load balancer has been deprecated and removed as it does not compile or run on pfSense 2.5.0. A copy of the load balancer configuration will be left in /conf/deprecated_load_balancer.xml for reference when converting to an alternate solution, such as HAProxy (HAProxy package).

How do I check if we are using relayd and if I definitely need to migrate to HAProxy before attempting to upgrade?

We are running:

CARP
OpenVPN
IPSec tunnels to AWS
Firewall
NAT
Config/HA sync

The upgrade page is showing:

Current Base System: 2.4.4_3
Latest Base System:  21.02_1

My plan is:

Put secondary router in maintenance mode (just to ensure it doesn't accidentally become master for whatever reason)
Reboot the router
Upgrade to 2.4.5 (Other posts in this forum suggest this is the best path, but not sure if that's because of limited resources on the smaller routers.)
After upgrade is complete, reboot once more just to make sure.
Disable maintenance mode on router 2
Check status of CARP/IPSec
Enable maintenance mode on router 1
Verify connectivity and IPSec failover. Wait 15 minutes to make sure our monitoring doesn't detect any issues.
Reboot Router 1
Wait 30 minutes to ensure still no further issues
Upgrade router 1, reboot
Upgrade plugins on router 1 then reboot
Disable maintenance mode on router 1 and check that CARP fails over correctly.
Wait another 30 minutes to ensure everything is working before repeating the above steps until I'm running the latest version available/detected.

Unfortunately we don't have a spare pair of routers for testing.

mahomed

Although having checked a bit further, I might be confusing Load Balancing and CARP/Failover.

I have nothing configured under System > Routing > Gateways > Gateway Groups and I don't have anything configured under Services > Load Balancer

If anyone more knowledgeable could confirm, that would be awesome. Thanks.

SteveITS

Load Balancing is separate from HA/CARP.

I would say there is probably not a reason to upgrade to 2.4.5 first, but there are notes to read.

It does seem odd that you only show 21.02, your router should see 21.05 which is current. Is System/Update set to Previous Stable Version?

Do you have any packages installed? Netgate's advice is to uninstall those, then upgrade pfSense, then reinstall the packages. Otherwise the upgrade will try to update them for you. Also there is a bugfix for pfBlocker-devel sync on HA if you have that one.

On slower routers I allow 10-15 minutes or so for the upgrade before I start worrying. A 7100 is probably a bit faster.

mahomed

@steveits Thanks very much Steve. I have gone through the release notes and that's where I picked up the passage about the load balancer.

System/Update is set to latest stable version.

I did read the warning about removing the packages, but I have two concerns about that.

If it removes all related configs used/setup by the plugin, then it is going to cause downtime.
Re-installing the packages will cause config changes/resets that could cause issues, especially as we have the Config Sync setup.

Below are all the packages we have installed.

SteveITS

There is a doc page on troubleshooting why the latest version isn't offered.

re: packages, most have an option to preserve settings upon uninstall, if they have their own settings, for this (upgrades) reason. We mostly use Suricata/Snort and pfBlocker(-devel), and apcupsd. The latter rarely changes so I haven't bothered removing it. The others, more often. You can try upgrading without removing any and see. Offhand I don't think any of the ones you listed will cause downtime if removed? (unlike, removing pfBlocker will break aliases). Coreboot I think gets removed in 21.x, at least it was on the models I've updated so far.

The config sync doesn't generally work across versions anyway.

mahomed

Thanks for that link. According to it, I think it's normal that it's not offering 21.05 yet as need to upgrade to 21.02 first.

If that procedure results in an error, or the upgrade is still not offered, then attempt to update to an intermediate version. For example, to get from Plus 2.4.5-p1 to Plus 21.05 or later, upgrade to Plus 21.02.x before proceeding to later versions if a direct upgrade does not succeed. In these cases, the appropriate version will be visible as a branch to select.

Noted on the packages advice. I'll take a backup and then remove the packages and see what it does. If anything breaks, I can either restore the config, or hopefully will have the settings required to reconfigure.

SteveITS

@mahomed said in Upgrade from 2.4.4_3 to 21.02_1 sanity check:

if a direct upgrade does not succeed

I took one from 2.4.5 to 21.05 recently and have skipped in the past. I guess I would read "does not succeed" means "if it fails then try 21.02." Should not be a problem doing it in two steps.

mahomed

For anyone finding this in future, I am glad to report that my router upgrades were successful (thank you pfSense team and contributors).

I did have to go from 2.4.4_3 to 2.4.5_1 first (on both the 7100 and the 3100). After the upgrade, I did not touch any of the plugins, changed the update branch to latest and rebooted again (just to be sure).

Then I was able to go from 2.4.5_1 straight to 21.05 and it even then upgraded all the plugins to the latest for me.

It did take me several hours because I had two routers to do, I had CARP failover and IPSec tunnels and I waited to make sure none of my alerting systems went crazy. But the actual upgrade process was suprisingly and refreshingly straight forward without any drama.