• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

Upgrade to ZFS

Scheduled Pinned Locked Moved Problems Installing or Upgrading pfSense Software
58 Posts 16 Posters 19.3k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • R
    RyanM
    last edited by Jun 9, 2021, 4:07 AM

    Is there a way to tell if I am running ZFS? If I am not, what is the path for upgrading to ZFS? Is it straightforward?

    G J 2 Replies Last reply Jun 9, 2021, 5:53 AM Reply Quote 0
    • G
      Gertjan @RyanM
      last edited by Jun 9, 2021, 5:53 AM

      @ryanm said in Upgrade to ZFS:

      Is it straightforward?

      Maybe.
      Before using it, of even think you have a choice, you should read about it.
      By default, you stick - without maybe even knowing what it is or means - with what was installed when you installed a/the OS on a disk.

      To use ZFS, there are some conditions.

      To go from UFS to ZFS : re install pfSEnse from scratch, which is actually an easy thing to do.

      @ryanm said in Upgrade to ZFS:

      Is there a way to tell if I am running ZFS

      For example :

      zpool list
      

      or : ask the guy who installed pfSense, as the choice was there during initial disk partitioning and formatting ;)

      No "help me" PM's please. Use the forum, the community will thank you.
      Edit : and where are the logs ??

      J 1 Reply Last reply Jun 9, 2021, 10:20 AM Reply Quote 1
      • J
        johnpoz LAYER 8 Global Moderator @Gertjan
        last edited by Jun 9, 2021, 10:20 AM

        Or just look at the gui system widget

        zfs.png

        An intelligent man is sometimes forced to be drunk to spend time with his fools
        If you get confused: Listen to the Music Play
        Please don't Chat/PM me for help, unless mod related
        SG-4860 24.11 | Lab VMs 2.8, 24.11

        1 Reply Last reply Reply Quote 2
        • J
          JeGr LAYER 8 Moderator @RyanM
          last edited by Jun 9, 2021, 10:25 AM

          @ryanm said in Upgrade to ZFS:

          Is there a way to tell if I am running ZFS? If I am not, what is the path for upgrading to ZFS? Is it straightforward?

          As @johnpoz linked above: check your dashboard widget. The disc space section tells you if your / (root) is running on either UFS or ZFS.

          If I am not, what is the path for upgrading to ZFS? Is it straightforward?

          Re-installation with "recover config.xml" and "Auto (ZFS)" will bring you up to speed and having ZFS in ~10min. :)

          Cheers

          Don't forget to upvote 👍 those who kindly offered their time and brainpower to help you!

          If you're interested, I'm available to discuss details of German-speaking paid support (for companies) if needed.

          J C 2 Replies Last reply Jun 9, 2021, 10:44 AM Reply Quote 2
          • J
            johnpoz LAYER 8 Global Moderator @JeGr
            last edited by Jun 9, 2021, 10:44 AM

            @jegr is correct the only way move to zfs if your not is a clean install. If your on an netgate appliance just email, or open a ticket with netgate support and they will send you a + image for install. You do not need to have a support contract for this.

            I moved to zfs with the release of 21.02.2 - the 21.02 had a bit of an issue with installing to zfs. There was a work around for it.. but I just waited til 21.02.2 release to make the move.

            You can do a clean and then restore of your backup, or just clean and reconfigure by hand - really depends on how much stuff you have configured.. If your a pretty vanilla install - only take a few minutes to do a clean install and be up and running just out of the box.

            An intelligent man is sometimes forced to be drunk to spend time with his fools
            If you get confused: Listen to the Music Play
            Please don't Chat/PM me for help, unless mod related
            SG-4860 24.11 | Lab VMs 2.8, 24.11

            1 Reply Last reply Reply Quote 0
            • J
              jimp Rebel Alliance Developer Netgate
              last edited by Jun 9, 2021, 12:57 PM

              If you are on CE, wait for 2.5.2 to reinstall with ZFS. It has a much better starting dataset layout.

              If you are on Plus, 21.05 is good for amd64 devices (e.g. SG-5100, XG-7100, etc), but ARM devices don't yet support ZFS. We're working on bringing support for ZFS to the 64-bit ARM devices (SG-1100, SG-2100), though. If you have one of those, stay tuned.

              Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

              Need help fast? Netgate Global Support!

              Do not Chat/PM for help!

              J B 2 Replies Last reply Jun 9, 2021, 1:00 PM Reply Quote 2
              • J
                johnpoz LAYER 8 Global Moderator @jimp
                last edited by Jun 9, 2021, 1:00 PM

                @jimp said in Upgrade to ZFS:

                It has a much better starting dataset layout.

                Is there any change in the layout from 21.02.2 and 21.05? If did clean install with 21.02.2 to zfs, is there any reason that I should redo that with 21.05?

                An intelligent man is sometimes forced to be drunk to spend time with his fools
                If you get confused: Listen to the Music Play
                Please don't Chat/PM me for help, unless mod related
                SG-4860 24.11 | Lab VMs 2.8, 24.11

                1 Reply Last reply Reply Quote 0
                • J
                  jimp Rebel Alliance Developer Netgate
                  last edited by Jun 9, 2021, 1:12 PM

                  The layout is new to 21.05 and 2.5.2. If you have it, you'd know, since there are a lot more datasets and the pool is named pfSense instead of zroot

                  Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                  Need help fast? Netgate Global Support!

                  Do not Chat/PM for help!

                  J 1 Reply Last reply Jun 9, 2021, 3:30 PM Reply Quote 1
                  • J
                    johnpoz LAYER 8 Global Moderator @jimp
                    last edited by johnpoz Jun 9, 2021, 3:34 PM Jun 9, 2021, 3:30 PM

                    So should prob do a clean install then.. Since would assume the pfsense name will be what is used going forward. I do recall seeing something about recovery/restore related to the zfs name..

                    edit: Prob just wait til 21.09 comes out and do clean install then.. From reading the redmine about the recovery.

                    An intelligent man is sometimes forced to be drunk to spend time with his fools
                    If you get confused: Listen to the Music Play
                    Please don't Chat/PM me for help, unless mod related
                    SG-4860 24.11 | Lab VMs 2.8, 24.11

                    B 1 Reply Last reply Nov 26, 2021, 12:35 PM Reply Quote 0
                    • R
                      RyanM
                      last edited by Jun 9, 2021, 6:30 PM

                      Thank you all for the information.

                      I am running UFS (thanks @johnpoz for pointing out the widget).

                      @Gertjan I am somewhat familiar with ZFS, less so UFS. I am running a Free/TrueNAS server with ZFS and am familiar with the underlying concept of how it is a journal filesystem and such.

                      From what I recall reading on these forums, there are advantages of moving from UFS to ZFS regarding performance and reliability. Maybe I misread it, or the advantage is minor, idk. Even if minor, it sounds like there may be benefits of re-installing and using ZFS. I think my router originally had something in the 2.3.x or maybe early 2.4.x and I have been upgrading it pretty regularly since then.

                      @jimp thank you for pointing out to wait for 2.5.2. Since I am not on ZFS, I will wait until 2.5.2 before I mess with this. This only came up because I decided to upgrade to 2.5.1 yesterday and remembered that I had wanted to look into ZFS a few months ago but got distracted.

                      Ok, so I understand that the right path forward is a clean install. If I have been backing up my configuration to Netgate, is there anything special I need to check or ensure before I do the clean install? Would it be better to upgrade to 2.5.2, let the configuration backup, then clean install 2.5.2 and restore the config?

                      I assume after the clean install I will need to get connected to the internet and then use the restore from either the UI or the console menu? Is there an option to restore during installation? Does this require my Netgate login or something else?

                      Is there anything that may not be included in my configuration backup? Like package configuration?

                      1 Reply Last reply Reply Quote 0
                      • A
                        AndyRH
                        last edited by Jun 9, 2021, 6:46 PM

                        You can practice with a spare PC or a VM. It does not need to replace the existing FW, just allow you to install and restore.
                        I install enough to get on the LAN then restore a backup through the web UI. I expect you will find it easy since you made TrueNAS work.

                        o||||o
                        7100-1u

                        1 Reply Last reply Reply Quote 0
                        • J
                          jimp Rebel Alliance Developer Netgate
                          last edited by jimp Jun 9, 2021, 7:09 PM Jun 9, 2021, 7:09 PM

                          The main advantages of ZFS are stability and features such as native compression, mirroring, snapshots, etc. Some things we don't take full advantage of quite yet in pfSense.

                          ZFS isn't really known for its performance, but filesystem speed on a firewall isn't critical compared to stability. It also has increased memory requirements compared to UFS, which is why it's not ideal for certain platforms (e.g. 32-bit ARM, systems with low RAM).

                          When the time comes, take a fresh local backup (just in case), then boot the install media and choose "recover config.xml", then install, pick ZFS, select the drive(s), and go on from there. It will boot back up with your existing configuration in place, reinstall your packages, and be back up and running as it was before.

                          Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                          Need help fast? Netgate Global Support!

                          Do not Chat/PM for help!

                          1 Reply Last reply Reply Quote 0
                          • M
                            mer
                            last edited by Jun 15, 2021, 8:12 PM

                            Memory requirements. Lots of information around on that, most of the recommendations are 64bit system, at least 4GB RAM, more is way better.
                            From a design standpoint, ZFS was created to handle large disks.

                            From a usability POV, I think the best part of ZFS is Boot Environments (BEs). They are a very good way to upgrade a system safely and rollback if needed. UFS has had work recently to mimic/mirror that (think separate partitions for the root/boot filesystem, then flags used to tell which one to actually use).

                            pfSense with ZFS? On a big enough system, it's probably a good thing. I'd be interested in seeing how some of the things like mirrors and snapshots would be used.

                            J J 2 Replies Last reply Jun 15, 2021, 9:36 PM Reply Quote 0
                            • J
                              johnpoz LAYER 8 Global Moderator @mer
                              last edited by Jun 15, 2021, 9:36 PM

                              The only reason I moved to zfs vs ufs is better handling of power loss. While I have never had my netgate loose power out of the blue. It is on a ups, etc. And power here is pretty stable and rarely have extended outages..

                              But since pfsense allows for choice.. And my box is never close to using its ram why not.. It wasn't something I was going to switch to out of the blue since it requires clean install - but with the move to new 2.5/21 versions - figured it was a good time to make the move..

                              Where they go with it in leveraging its other functions - can only wait and see.. I would love the ability to rollback to previous version on failed upgrade, etc. That would be slick!!

                              An intelligent man is sometimes forced to be drunk to spend time with his fools
                              If you get confused: Listen to the Music Play
                              Please don't Chat/PM me for help, unless mod related
                              SG-4860 24.11 | Lab VMs 2.8, 24.11

                              M 1 Reply Last reply Jun 15, 2021, 9:49 PM Reply Quote 2
                              • M
                                mer @johnpoz
                                last edited by Jun 15, 2021, 9:49 PM

                                @johnpoz said in Upgrade to ZFS:

                                Where they go with it in leveraging its other functions - can only wait and see.. I would love the ability to rollback to previous version on failed upgrade, etc. That would be slick!!

                                Yes yes it would. Having used ZFS for a while on the desktop, switching is trivial. The tricky parts are the logic of failback. If you make it at least to init you can start checking flags and such. But if the kernel crashes on the way up, manual intervention required.

                                1 Reply Last reply Reply Quote 0
                                • J
                                  JeGr LAYER 8 Moderator @mer
                                  last edited by Jun 15, 2021, 10:30 PM

                                  @mer said in Upgrade to ZFS:

                                  pfSense with ZFS? On a big enough system, it's probably a good thing. I'd be interested in seeing how some of the things like mirrors and snapshots would be used.

                                  I read that through the whole internet. ZFS... memory... blah. When last speaking with folks from TruNAS and iXsystems they all replied to me that most things are just BS nowadays. Yes in theory ZFS needs more RAM and more CPU. But the features they are used in are of no particular interest in a pfSense installation setting. Namely things like deduplication for large storages and other advanced features.

                                  We are running pfSense on ZFS on hardware boxes for years now and I still have to find that "largely increased memory and CPU footprint" everyone is so quick to post when it comes to ZFS.
                                  It all comes down to reliability and ZFS safed our customer's hides more then once when there was "a sudden power loss"... again. Since pfSense introduced ZFS in the installer we are using it and don't lost a single box since then.

                                  Of course, I'd love to see snapshots and BEs popping up being used as that would bring back those early days when you could click the box besides the update button to "make a full backup prior to updateing the box" and you could then roll back easily. Having that with a combo of BEs and snapshots would be incredibly helpful!

                                  But if it's just "should I run ZFS... " my answer would almost always* be a "YES".

                                  • Not counting virtual/VM setups. You can/should use & rely on your hypervisor for that.

                                  Don't forget to upvote 👍 those who kindly offered their time and brainpower to help you!

                                  If you're interested, I'm available to discuss details of German-speaking paid support (for companies) if needed.

                                  J 1 Reply Last reply Jun 15, 2021, 10:34 PM Reply Quote 0
                                  • J
                                    johnpoz LAYER 8 Global Moderator @JeGr
                                    last edited by Jun 15, 2021, 10:34 PM

                                    @jegr said in Upgrade to ZFS:

                                    Namely things like deduplication for large storages and other advanced features.

                                    Yeah came to the same conclusion - not running it on a 100TB array ;) My pfsense is like a 24GB disk.. And it has 8GB of ram - so what if using ZFS uses a bit more than ufs..

                                    An intelligent man is sometimes forced to be drunk to spend time with his fools
                                    If you get confused: Listen to the Music Play
                                    Please don't Chat/PM me for help, unless mod related
                                    SG-4860 24.11 | Lab VMs 2.8, 24.11

                                    M 1 Reply Last reply Jun 15, 2021, 10:45 PM Reply Quote 1
                                    • M
                                      mer @johnpoz
                                      last edited by Jun 15, 2021, 10:45 PM

                                      @johnpoz @JeGr Well, as you both point out, it's all about the use case, I was speaking in generalities, not as it would relate specifically to pfSense devices. Apologies if that was not clear.

                                      Take the SG2100/SG3100, could you run ZFS on them? With a bit of work, probably. Would it be worth it? Up to you.

                                      8GB RAM and 24GB disk? Of course.

                                      The issue is perception. Remember when embedded devices had MB of RAM and KB of storage?
                                      Heck how much does a low end phone have now?
                                      4GB of RAM is a fairly low bar now so yes, of course, it makes sense.

                                      J 1 Reply Last reply Jun 18, 2021, 1:26 PM Reply Quote 1
                                      • J
                                        JeGr LAYER 8 Moderator @mer
                                        last edited by Jun 18, 2021, 1:26 PM

                                        @mer said in Upgrade to ZFS:

                                        @johnpoz @JeGr Well, as you both point out, it's all about the use case, I was speaking in generalities, not as it would relate specifically to pfSense devices. Apologies if that was not clear.

                                        No problem :)

                                        4GB of RAM is a fairly low bar now so yes, of course, it makes sense.

                                        But that's what I was talking about. The "perception" - and in many heads that half-knowledge is still pretty current and active - is that you need veeeery much RAM to run ZFS because it is a big storage filesystem. That's simply not true nowadays according to those that deal with it on a day by day basis. Yeah sure, that was the case quite a bit ago, but today?

                                        I run and have installed devices down to 1-2GB of RAM on a 32GB mSATA SSD - that is really small for a device today. And I didn't really see a jump in idle CPU power or RAM usage on those systems. As install size is around 1-2GB for a core installation with a few logs, that's nothing in ZFS terms. And compared with the ease of mind, that ZFS is far more robust than the default UFS installation without journaling etc against power loss - that was a really easy decision for us to roll out anything we can with ZFS. Really awaiting first contact with ARM-SGs and ZFS but also I don't expect them to be hit very hard by a switch to ZFS either.

                                        Cheers :)

                                        Don't forget to upvote 👍 those who kindly offered their time and brainpower to help you!

                                        If you're interested, I'm available to discuss details of German-speaking paid support (for companies) if needed.

                                        M 1 Reply Last reply Jun 18, 2021, 2:00 PM Reply Quote 1
                                        • M
                                          mer @JeGr
                                          last edited by Jun 18, 2021, 2:00 PM

                                          @jegr Yep perception.

                                          My problem is I still think of 1GB RAM and 8GB storage as huge. And yes the workload and features used are what kills it. ARC and other caching are primarily "read" operations (yes I know writes go through cache at some levels too), so if the system is primarily write (like pfSense would be) they are not going to try to use memory.
                                          For a system like pfSense, 2GB RAM should be enough to let you use the good features of things like Boot Environments. The physical storage device I think is a bit more important than how much RAM in a small system.

                                          Upgrading my home desktops, upgrades/updates are:
                                          create a new BE
                                          chroot into it
                                          install updates/upgrades of base system and update all applications
                                          set new BE active
                                          reboot

                                          If it fails, I haven't mucked up a running system, stopping in the boot loader lets you easily roll back. I can imagine a "run once" rc script that if you haven't been able to run (however that is defined) you simply activate the previous working BE and reboot and you're back up and running.

                                          Anyway, it's Friday.

                                          J 1 Reply Last reply Jun 18, 2021, 2:39 PM Reply Quote 1
                                          20 out of 58
                                          • First post
                                            20/58
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
                                            This community forum collects and processes your personal information.
                                            consent.not_received