Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Upgrading - following the pfSense docs 'Installing and Upgrading' includes having a fall back plan

    Scheduled Pinned Locked Moved Problems Installing or Upgrading pfSense Software
    36 Posts 12 Posters 3.1k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • ?
      A Former User @fabrizior
      last edited by

      @fabrizior Agreed. It's an odd policy at best that on day one the "older" versions become inaccessible.

      You can open a ticket with them and they will get you a 2.4.5_p1 image. No support contract required. That's a pain for both users and Netgate.

      As to the other drama I'll not say anything more than I've said in other threads. I, speaking only for myself, will move on to another product when the time is convenient. I have exactly zero interest in upgrading to 2.5. I'm done...

      1 Reply Last reply Reply Quote 1
      • G
        guardian Rebel Alliance
        last edited by

        What about all the plugins. IIUC, even if you have kept the old installer (which I have done), the installer wants to reinstall the plugins from the net. Do the old repos disappear?

        It won't work for sealed appliances, but for anyone using a generic PC that can boot from a USB drive, what about a bootable Clonezilla or Foxclone disk. They are linux based, but does that matter if you image the whole disk? I am in the process of building a bootable USB hard drive which I hope will be able to store the store the image.

        An ideal situation would be to have something FreeBSD based. For some reason this solution seemed to be met with hostility, but I'm not sure why.

        My proposed plan:

        1. Save the config file from the running install
        2. Have a USB of the installer for the running version available
        3. Download a copy of the new version and make a USB
        4. Take pfSense offline, and boot from a USB image program. Make an image/verify it.
        5. Restart the system and once it is up again, run the built in install
          ----- If successful done:
          ----- Else gather info for a bug report? Suggestions on this?
        6. Run the installer made in #3 and reload the saved config.
          ----- If successful done:
          ----- Else gather info for a bug report? Suggestions on this?
        7. Use the USB from Step #2 and reload the saved config. I'm not sure if this is an option if you have plugings - Comments?
          ----- If successful done:
          ----- Else gather info for a bug report? Suggestions on this?
        8. Boot into USB recovery disk and restore the disk image. Reboot pfSense and you should be back to where you started.

        Another possible solution would be to have a script that runs from single user mode that does a file level backup of the directories which contain all the plugins/config files. I don't know enough about how hard/easy this would be, and most importantly how reliable (would ongoing maintenance be required?).

        I've been using pfSense for about 5-6 years, but the last time, I had to go to step #6, and fortunately it worked so I didn't have to worry about what to do next. It did make me think I need to be much better prepared.

        I'd love to hear what the gurus here think/how they have solved this problem.

        If you find my post useful, please give it a thumbs up!
        pfSense 2.7.2-RELEASE

        T F 2 Replies Last reply Reply Quote 1
        • T
          Tzvia @guardian
          last edited by

          @guardian Good questions. I would first verify that when I do a backup of settings, that UPDATE/SYSTEM UPDATE is set to PREVIOUS STABLE. You can then also verify that the software versions of anything specific to 2.45p1 are still available in PACKAGE MANAGER. Provided they are listed, applying your backup to a fresh install of 2..45p1 should set it this way and it should be able to install all the packages when it reboots...

          Tzvia

          Current build:
          Hunsn/CWWK Pentium Gold 8505, 6x i226v 'micro firewall'
          16 gigs ram
          500gig WD Blue nvme
          Using modded BIOS (enabled CSTATES)
          PFSense 2.72-RELEASE
          Enabled Intel SpeedShift
          Snort
          PFBlockerNG
          LAN and 5 VLANS

          1 Reply Last reply Reply Quote 0
          • F
            fabrizior @guardian
            last edited by

            @guardian
            Have any recommendations on a usb-bootable tool for step 4, preferably to make a bootable backup copy. not well-versed in FreeBSD SysAdmin.

            G 1 Reply Last reply Reply Quote 0
            • G
              guardian Rebel Alliance @fabrizior
              last edited by

              @fabrizior said in Upgrading - following the pfSense docs 'Installing and Upgrading' includes having a fall back plan:

              @guardian
              Have any recommendations on a usb-bootable tool for step 4, preferably to make a bootable backup copy. not well-versed in FreeBSD SysAdmin.

              I was hoping someone who is BSD experienced would comment.

              IIUC as long as I am going to image the whole disk and the backup and restore are done from another running system (i.e. boot from a USB disk or second drive) almost any imaging method will do All we are doing is blindly copying bytes. If it is necessary to restore, then the whole image must be restored to the same disk, or one with identical geometry.

              My plan was to use Foxclone. I installed a copy of Linux Mint and Foxclone on an old 320GB USB hard drive. I have about 300GB free to copy drive images. That should be more than enough. I was thinking that I could even do something as simple as:
              Backup:

              dd if=/dev/sdx bs=4096 of=backupdir/pfsense.img status=progress
              

              Restore:

              dd if=backupdir/pfsense.img bs=4096 of=/dev/sdx status=progress
              

              Can someone confirm if my understanding is correct or not and/or a better tool to use.

              If you find my post useful, please give it a thumbs up!
              pfSense 2.7.2-RELEASE

              F 1 Reply Last reply Reply Quote 0
              • F
                fabrizior @guardian
                last edited by

                @guardian
                How’s this sound as a rough starting point?
                I don’t have a dev system to try this on right now... VM infra is busy with other projects ATM.

                For the purposes of switch-over / fail-back with upgrades, and to maintain RTO/RPO SLA objectives against drive faults (not catastrophic system-wide HW/electrical faults) I’m considering:

                1. Add a second internal drive (same size)
                  Could try this with an approp-sized removable flash drive for off-line / off-site backup safety vs ease of switchover/fail-back, depending on SLAs, I suppose.

                2. partitioning new drive appropriately and make it bootable (values may vary, only an pasted example)
                  Q: can anyone help validate details with a pfsense env?

                gpart destroy -F adaX
                gpart create -s GPT adaX
                gpart add -b 40 -s 128 -t freebsd-boot adaX
                gpart add -s 1880 -t efi adaX
                gpart add -s 4G -t freebsd-swap -l SWAP adaX
                gpart add -t freebsd-ufs -l RECOVER adaX
                
                gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 adaX
                gpart bootcode -p /boot/boot1.efifat -i 2 adaX
                gpart set -a bootme -i 4 adaX
                
                newfs -ntEU /dev/adaXp4
                tunefs -a enable /dev/adaXp4
                
                1. periodically cloning the current boot drive to the other. (with additional script logic to set source/dest parameters and manage the mount point, etc...)

                use sysutils/clone to make a perfect clone of your system’s startup disk as follows:

                mount -o noatime /dev/gpt/RECOVER /mnt
                clone -c rwoff / /mnt
                

                After the file system cloning has finished, you only would need to edit /mnt/etc/fstab:

                # Device            Mountpoint      FStype  Options Dump    Pass#
                /dev/gpt/RECOVER    /               ufs     rw      1       1
                /dev/gpt/SWAP       none            swap    sw      0       0
                proc                /proc           procfs  rw      0       0
                

                clone(1) can be used to keep file systems synchronized, and you would use the very clone command together with the -s flag for keeping your recovery disk updated. In this mode only changed files would be copied and synchronization needs much less time than the initial full cloning process.

                Alternatives? Is there a better way, still w/o 3rd-party software? dd doesn’t have an incremental sync mode like clone(1) does... while block-leve copies might be technically faster, dd is an all-or-nothing operation. clone seems like it would have some advantages after the first time.

                Any other (perhaps pfSense-specific) partitioning/boot details to account for?

                credit:
                found the idea and example code from an 6-Aug-20 reply by @obsigna on the freebsd forum.

                1 Reply Last reply Reply Quote 0
                • P
                  Patch
                  last edited by

                  Given Negate policy to remove access to old binaries as soon as a new versions is released the only safe strategy I can see is for all users to individually maintain their own archive of any version they use. So

                  • When wanting to upgrade pfSense by any method, first download and archive an iso in case you need it at a later date (note you can not reliably do this later as Netgate will pull the binary before you try to upgrade the next time).

                  • Upgrade pfSense by whatever method you prefer

                  • If the upgrade fails for you, use the archived version (the one you archived during a prior upgrade

                  Not efficient for a community but at least it ensures a reliable upgrade fall back from a known good source.

                  G F 2 Replies Last reply Reply Quote 0
                  • G
                    guardian Rebel Alliance @Patch
                    last edited by

                    @patch said in Upgrading - following the pfSense docs 'Installing and Upgrading' includes having a fall back plan:

                    Given Negate policy to remove access to old binaries as soon as a new versions is released the only safe strategy I can see is for all users to individually maintain their own archive of any version they use. So

                    • When wanting to upgrade pfSense by any method, first download and archive an iso in case you need it at a later date (note you can not reliably do this later as Netgate will pull the binary before you try to upgrade the next time).

                    • Upgrade pfSense by whatever method you prefer

                    • If the upgrade fails for you, use the archived version (the one you archived during a prior upgrade

                    Not efficient for a community but at least it ensures a reliable upgrade fall back from a known good source.

                    Good for the base version, but what about the plug-ins etc.?

                    If you find my post useful, please give it a thumbs up!
                    pfSense 2.7.2-RELEASE

                    1 Reply Last reply Reply Quote 0
                    • F
                      fabrizior @Patch
                      last edited by

                      @patch Any thoughts on an appropriate process to make a bootable clone of the boot disk?

                      Thought being: if an upgrade fails, change the boot device to the backup disk and reboot to the clone... back online almost immediately. then clone the booted backup back over the failed upgrade image to restore DR capability before attempting upgrade again at next maintenance window...

                      Anyone have practical experience with the partitioning/cloning a pfSense drive?

                      Would this need to be done offline?
                      If attempted online, any pid/lock/runtime state files that need to be excluded or reset on the clone?

                      1 Reply Last reply Reply Quote 0
                      • P
                        Patch
                        last edited by

                        First line upgrade strategy is for the upgrade to work as expected.

                        A fall back plan for me needs to be much more reliable than the upgrade path, hence the desire to return to a reference state (original install media & backed up configuration file).

                        Dynamic snapshots are useful if you have been using them regularly so have validated their functionality. Relying on them for a once off fall back plan is too risky for me, or if validated, too time consuming for something I intend not to use.

                        Dynamic snapshots do make far more sense if you run pfSense on a virtual machine (but then you are in the "using them regularly" group).

                        F 1 Reply Last reply Reply Quote 0
                        • F
                          fabrizior @Patch
                          last edited by fabrizior

                          @patch Thanks. That standard process is probably a good fit for many. [edit] Though, one might think the most “reliable” fallback plan would be a direct clone of the (presumably stable) system immediately prior to upgrade. Having to reinstall and reapply a config reliant on specific prior versions of packages that may or may not be available...

                          From my point of view, we could come up with a quicker, less maintenance-intensive process from an RTO metric / SLA perspective (when relevant) without resorting to clustered hardware.

                          The recovery time in case of upgrade failure when the process is to re-install the base image, restore the config, validate all the packages (hopefully at same version as pre-upgrade), etc... isn’t the quickest process and somewhat manually labor intensive (in comparison.)

                          Executing a scripted incremental update of a cloned disk (especially if done regularly, and if could be done online (?)) just-in-time prior to the upgrade maintenance window would allow for a faulted upgrade recovery time no longer than what it takes to reboot.
                          Much reduced recovery effort seems valuable enough to continue investigating how this could be done.

                          Only downsides seems to be the cloning process discovery/analysis and the cost of a second disk.

                          I 1 Reply Last reply Reply Quote 0
                          • I
                            itpp21 @fabrizior
                            last edited by

                            Reading this with interest, I'm looking at an usb stick recover method and the possibilities of adding packages from a running environment to this stick. As recovery(installation) by stick is already supported we might expand this by incorporating packages.
                            One question is will the usb stick auto install or do you still need console access to get it running? if so can you force auto install so it becomes a matter of insert stick and boot?
                            (yes you can yell at me) but under windows you can extract the wim file and modify it to your liking, repackage the wim and at boottime push out additional packages without a key pressed to add on a fresh installation.

                            GertjanG 1 Reply Last reply Reply Quote 0
                            • GertjanG
                              Gertjan @itpp21
                              last edited by

                              @itpp21 said in Upgrading - following the pfSense docs 'Installing and Upgrading' includes having a fall back plan:

                              (yes you can yell at me)

                              Noop.
                              Just 'gime' 1 % of 1 % of the Microsoft develop budget and you'll have that option ready for tomorrow.
                              Btw : with MS conditions of course : no documentation what so ever, and a real "lets test it in the field and see what telemetry brings home".

                              No "help me" PM's please. Use the forum, the community will thank you.
                              Edit : and where are the logs ??

                              I 1 Reply Last reply Reply Quote 0
                              • I
                                itpp21 @Gertjan
                                last edited by

                                I don't get the reply, if the stick allows pfSense installation (which it does) then why can't it be modified to auto install and why can't you add stuff? Its just an 'image' bits and bytes. I remember the ubuntu iso's which when you don't do anything it times out of some default options and then deploys.

                                GertjanG 1 Reply Last reply Reply Quote 0
                                • GertjanG
                                  Gertjan @itpp21
                                  last edited by

                                  You can modify the ISO before you use it.
                                  What's stopping you ?

                                  ( ok, you need to know how, what to place where, how things work, have plenty of time to test etc )

                                  No "help me" PM's please. Use the forum, the community will thank you.
                                  Edit : and where are the logs ??

                                  I 1 Reply Last reply Reply Quote 0
                                  • I
                                    itpp21 @Gertjan
                                    last edited by

                                    @gertjan said in Upgrading - following the pfSense docs 'Installing and Upgrading' includes having a fall back plan:

                                    ( ok, you need to know how, what to place where, how things work, have plenty of time to test etc )

                                    Exactly, I've ordered some sticks for this, will get back when I get anywhere, maybe others find this idea interesting to assist in the how, where and what.

                                    I 1 Reply Last reply Reply Quote 1
                                    • I
                                      itpp21 @itpp21
                                      last edited by

                                      The easy solution:
                                      Create a pfSense VM with virtualbox
                                      Load packages and settings from your bare metal device
                                      Verify the config works as bare metal.
                                      Shut the VM down

                                      Restore:
                                      Use vboxmanage to convert vdi to vhd
                                      Use rufus to load the vhd on the ram device (taken out of the bare device mounted via usb)
                                      Place storage back, done. (and tested)

                                      I'm gonna test the reverse (take bare metal in to a VM) but have no doubts this will work.

                                      nb. cloning the storage device as backup also works.

                                      1 Reply Last reply Reply Quote 0
                                      • First post
                                        Last post
                                      Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.