Best practice upgrading critical systems?

pete.s.

We have a HA pfSense installation running on two bare metal servers.

We follow a procedure for upgrading the firewalls based on the redundant firewalls upgrade guide. Basically that means that after some verification, reboots and backups we upgrade the secondary first (normally BACKUP) then some more checks and then we upgrade the primary (normally MASTER).

Now I've noticed that we had actually have some issues with the packages, even though everything is working. And there is something not right with OpenVPN failover as well.

So I wonder:

Is it better practice for us to upgrade by taking a backup, then reinstall from ISO and restore backup config? That should make it impossible for some things to have the wrong version or not being properly updated.
If number 1 is correct, is there something special that we need to keep in mind since this is a HA config?
We would have to do reinstalls remotely over ipmi / idrac / ilo. Is it safe to install pfSense ISO while the ethernet interface are connected to the internet? Or is there any window during the installation that some pfSense default configuration is running, perhaps designating our WAN interfaces as LAN? If I understand correctly placing the config file in the root of the ISO will make it automatically restore that config.