Upgrade HA Pfsense 2.1.5 (i386)



  • Hi,

    I have two VM Pfsense 2.1.5 with HA (IPSEC and a lot of CARP IP).

    I know that PFsense 2.4.4 is not anymore available for i386, so i have to restore my configuration on a new fresh installation.

    Will this work :

    1. Create two new Pfsense 2.4 VM
    2. Stop Slave VM 2.1.5
    3. Restore Configuration of Slave VM 2.1.5 to a new VM 2.4.4
    4. Disable Master CARP VM 2.1.5 to failover to new Slave VM 2.4.4 (?)
    5. Shutdown Master CARP VM 2.1.5
    6. Restore Confguration of Master VM 2.1 to second VM 2.4.4
    7. Enable CARP on new Master 2.4 ?

    I wan to limit as short as possible the downtime...

    Thanks

    Yathus



  • 2.1 is pretty old...it might be worth trying to upgrade those to 2.2 or 2.3 first?

    Note page https://www.netgate.com/docs/pfsense/highavailability/redundant-firewalls-upgrade-guide.html that "The underlying pfsync protocol often changes between FreeBSD versions, for example FreeBSD 7.2 (1.2.3) and 8.3 (2.1) and 10.1 (2.2) and 10.3 (2.3), so these versions cannot sync their states between each other. Failover will still function, but not stateful failover so all existing connections will be dropped."

    Also the next section, "On pfSense 2.2-RELEASE and later, states contain information about the interface to which they are bound. If the interfaces do not line up on both nodes then the states will not properly sync, for example if WAN is igb0 on one unit and em0 on the other.
    Adding interfaces to LAGGs can work around this..."

    So there are a few reasons states may not sync from the primary physical router to a VM backup running 2.4.x.



  • @teamits said in Upgrade HA Pfsense 2.1.5 (i386):

    2.1 is pretty old...it might be worth trying to upgrade those to 2.2 or 2.3 first?

    Yes, may be i could upgrade to 2.3.5 first and wait some days.

    So there are a few reasons states may not sync from the primary physical router to a VM backup running 2.4.x.

    So it's better to stop both nodes of current HA Pfsense 2.1, then restore config of master on a new 2.4.4 and rebuild HA after ? Downtime will me around 2 - 3 min, just time to reload config no (or less if i restore my config on a unplugged Pfsense 2.4 then start it.)


  • Netgate Administrator

    You should be able to do this with minimal downtime. Even if the states don't sync the connections should just open new states when it fails over which would take seconds not minutes for most TCP connections. You might find UDP stuff times out.

    You should disable config sync on the primary before bringing up the secondary. Those earlier versions did not have code to prevent syncing invalid config onto a newer version.

    The states may well sync fine though what interfaces types do you see in 2.1.5?

    Steve


  • LAYER 8 Netgate

    I would go straight to 2.4.4-p2. It will either work or it won't. If anything any issues in the config update code were corrected so you want to be running the latest when you import a config. It generally doesn't get worse, especially at the X.4-p2 stage of a development cycle.

    If you have any additional packages installed I would remove them then take a backup of the configuration to be restored, then reinstall the packages after you restore. A LOT has changed in the package handling since 2.1.5.

    Core functionality like CARP should be fine. Since you waited so long, yes, state sync won't work. Should be just a speedbump though if you're in a maintenance window. All they have to do is reload the page, etc.

    You can always import the config to a VM that is "on the bench" and see how it goes.

    I don't believe persistent CARP maintenance mode existed in 2.1.5 either. When you bring the primary back up I would make the new VM "on the bench" , restore the configuration, enable persistent CARP maintenance mode, then shut it down and "move" it into place. You should see all the VIPs go to BACKUP. Then just reinstall any packages, leave maintenance mode and you're whole.



  • Thanks Derelict and Stephen for your responses.

    I confirm, there is no persistent CARP maintenance mode in 2.1.5, after a reboot of Master/Slave, main role are back.

    For packages, i have only Openvpn Client Export, IPerf and OpenVM Tools, i can remove them i think.

    So here is my plan :

    • Create two new Pfsense 2.4.4 VM
    • Snapshot Vmware of Pfsense 2.1.5
    • Stop Slave VM 2.1.5 (after removing packages and backup config.xml)
    • Disable Sync on Master Pfsense 2.1.5
    • Restore Configuration of Slave VM 2.1.5 to a new VM 2.4.4
    • Reinstall Packages on 2.4.4
    • Disable Master CARP VM 2.1.5 to failover to new Slave VM 2.4.4 (no state sync)
    • Shutdown Master CARP VM 2.1.5
    • Restore Confguration of Master VM 2.1 to second VM 2.4.4
    • Enable CARP on new Master 2.4

    But first i'll test Pfsense 2.4 CARP and IP Alias, i dont want to discover WebGUI in my maintenance Window...



  • @yathus said in Upgrade HA Pfsense 2.1.5 (i386):

    But first i'll test Pfsense 2.4 CARP and IP Alias, i dont want to discover WebGUI in my maintenance Window...

    i restore configuration from 2.1.5 slave to a new Pfsense 2.4.4-p2 it's OK for CARP IP and Alias.

    Next test : disable Sync, shutdown current slave (2.1.5) and put in live new slave (2.4.4).


  • Rebel Alliance Developer Netgate

    @derelict said in Upgrade HA Pfsense 2.1.5 (i386):

    I would go straight to 2.4.4-p2. It will either work or it won't.

    This x1000. Rip off the band-aid and there will be less pain, and the whole process will be faster.

    If you have any additional packages installed I would remove them then take a backup of the configuration to be restored, then reinstall the packages after you restore. A LOT has changed in the package handling since 2.1.5.

    Definitely remove all packages.

    @yathus said in Upgrade HA Pfsense 2.1.5 (i386):

    Create two new Pfsense 2.4 VM
    Stop Slave VM 2.1.5
    Restore Configuration of Slave VM 2.1.5 to a new VM 2.4.4
    Disable Master CARP VM 2.1.5 to failover to new Slave VM 2.4.4 (?)
    Shutdown Master CARP VM 2.1.5
    Restore Confguration of Master VM 2.1 to second VM 2.4.4
    Enable CARP on new Master 2.4 ?

    You have VMs. Do this:

    • Backups!
    • Snapshot both VMs
    • Disable config sync on the primary
    • Remove all packages from secondary
    • Shutdown & adjust VM settings for the new OS and install 2.4.4-p1 on top of the secondary using the Recover config.xml option
    • Reinstall packages on secondary and test (check logs, ensure services are running, etc)
    • Disable CARP on the primary
    • Remove all packages from the primary
    • Shutdown & adjust VM settings for the new OS and install 2.4.4-p1 on top of the primary using the Recover config.xml option
    • When the primary boots, it will take over since 2.1.x didn't have CARP maintenance mode. If you do not have any packages required for service this may be OK, otherwise, put it in maintenance mode here.
    • Reinstall all packages on the primary and test
    • Re-enable config sync on the primary
    • Take the primary out of maintenance mode and test

    If at any point you have a problem, roll back to an earlier VM snapshot.

    Then you don't have to fuss with a second pair of VMs and making sure things get turned off/on at the right times.

    You still can use new VMs, it's just probably not necessary.



  • @jimp said in Upgrade HA Pfsense 2.1.5 (i386):

    • Backups!
    • Snapshot both VMs
    • Disable config sync on the primary
    • Remove all packages from secondary
    • Shutdown & adjust VM settings for the new OS and install 2.4.4-p1 on top of the secondary using the Recover config.xml option
    • Reinstall packages on secondary and test (check logs, ensure services are running, etc)
    • Disable CARP on the primary

    this is the moment where I cross my fingers ? 🤞

    • Remove all packages from the primary
    • Shutdown & adjust VM settings for the new OS and install 2.4.4-p1 on top of the primary using the Recover config.xml option
    • When the primary boots, it will take over since 2.1.x didn't have CARP maintenance mode. If you do not have any packages required for service this may be OK, otherwise, put it in maintenance mode here.
    • Reinstall all packages on the primary and test
    • Re-enable config sync on the primary
    • Take the primary out of maintenance mode and test

    If at any point you have a problem, roll back to an earlier VM snapshot.

    Sounds good to me, i think i'll test with your plan.

    Then you don't have to fuss with a second pair of VMs and making sure things get turned off/on at the right times.

    You still can use new VMs, it's just probably not necessary.

    i'm using E1000 NIC, it's usefull to move to VMXNET3 ?


  • Rebel Alliance Developer Netgate

    @yathus said in Upgrade HA Pfsense 2.1.5 (i386):

    i'm using E1000 NIC, it's usefull to move to VMXNET3 ?

    In general, yes, but if the e1000 NICs are performing well for you then there is compelling need to switch.



  • it's done !

    and it's working :)

    thanks @jimp your plan worked as expected, upgrade VM "in place" was a good advice.

    I only have 30 seconds of timeout in my RDP Session when disable / enabling CARP between nodes.