Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Another Netgate with storage failure, 6 in total so far

    Scheduled Pinned Locked Moved Official Netgate® Hardware
    301 Posts 37 Posters 54.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • valnarV
      valnar
      last edited by

      As someone about to buy a 4200, does anyone know if Netgate has implemented any of these strategies in their latest pfSense Plus? If not, can somebody summarize the recommended tweaks?

      M S A fireodoF 4 Replies Last reply Reply Quote 0
      • M
        Mission-Ghost @valnar
        last edited by

        @valnar buy a plus, set the vfs.zfs.txg.timeout to a value of between 15 and 30 seconds, connect it to an apc ups and use apcupsd package to initiate low power safe shutdown.

        A bonus would be to buy a fan so it runs cool. Not necessary by any means but would probably help with overall service life.

        My feeble understanding is that Netgate is working on zfs adjustments to address the poor ssd life but has released nothing yet.

        valnarV 1 Reply Last reply Reply Quote 1
        • valnarV
          valnar @Mission-Ghost
          last edited by

          @Mission-Ghost Thank you.

          1 Reply Last reply Reply Quote 0
          • S
            SteveITS Galactic Empire @valnar
            last edited by

            @valnar said in Another Netgate with storage failure, 6 in total so far:

            summarize the recommended tweaks

            See this post:
            https://forum.netgate.com/topic/195990/another-netgate-with-storage-failure-6-in-total-so-far/21

            Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
            When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
            Upvote 👍 helpful posts!

            1 Reply Last reply Reply Quote 1
            • S SteveITS referenced this topic
            • A
              andrew_cb @valnar
              last edited by

              @valnar For now, I recommend these two changes:

              Navigate to System > Advanced > System Tunables and add a new tunable:

              • Tunable vfs.zfs.tgx.timeout
              • Value 30

              Navigate to Diagnostics > Command Prompt and run
              zfs set sync=always pfSense/ROOT/default/cf

              A M JonathanLeeJ 3 Replies Last reply Reply Quote 1
              • A
                azdeltawye @andrew_cb
                last edited by

                @andrew_cb

                Thanks for the easy step-by-step instruction!

                Quick question - Do you have to run the zfs set command on each subsequent reboot or just the first time after adding the tunable?

                M 1 Reply Last reply Reply Quote 0
                • M
                  Mission-Ghost @andrew_cb
                  last edited by

                  @andrew_cb said in Another Netgate with storage failure, 6 in total so far:

                  Navigate to Diagnostics > Command Prompt and run
                  zfs set sync=always pfSense/ROOT/default/cf

                  Note: the command should read:

                  zfs set sync=always pfSense/ROOT/Default/cf

                  Default

                  starts with a capital D.

                  1 Reply Last reply Reply Quote 0
                  • M
                    Mission-Ghost @azdeltawye
                    last edited by

                    @azdeltawye said in Another Netgate with storage failure, 6 in total so far:

                    Quick question - Do you have to run the zfs set command on each subsequent reboot or just the first time after adding the tunable?

                    I'd be interested in the answer as well. The freeBSD help and Google Gemini (which is often wrong but never in doubt) suggests it's a one-time setting.

                    I missed this setting in earlier discussions, so I just executed it and it's set in the zfs config now. I haven't rebooted. Yet.

                    fireodoF 1 Reply Last reply Reply Quote 0
                    • fireodoF
                      fireodo @Mission-Ghost
                      last edited by fireodo

                      @Mission-Ghost said in Another Netgate with storage failure, 6 in total so far:

                      I missed this setting in earlier discussions, so I just executed it and it's set in the zfs config now. I haven't rebooted. Yet.

                      If you make a entry in the "system tuning" the setting will survive even a system upgrade (if still necessary)...

                      Kettop Mi4300YL CPU: i5-4300Y @ 1.60GHz RAM: 8GB Ethernet Ports: 4
                      SSD: SanDisk pSSD-S2 16GB (ZFS) WiFi: WLE200NX
                      pfsense 2.8.0 CE
                      Packages: Apcupsd, Cron, Iftop, Iperf, LCDproc, Nmap, pfBlockerNG, RRD_Summary, Shellcmd, Snort, Speedtest, System_Patches.

                      M 1 Reply Last reply Reply Quote 0
                      • E
                        elvisimprsntr
                        last edited by elvisimprsntr

                        Even though I have a Kingston mSATA SDD with 2.1 years power on hours, life is down to 90%

                        
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  9 Power_On_Hours          0x0000   100   100   000    Old_age   Offline      -       18867
169 Remaining_Lifetime_Perc 0x0000   090   090   000    Old_age   Offline      -       90


                        sysctl vfs.zfs.txg.timeout=30

                        Lower case "default" on 2.7.2 CE

                        zfs list

NAME                                 USED  AVAIL  REFER  MOUNTPOINT
pfSense                             24.1G   205G    96K  /pfSense
pfSense/ROOT                        1.11G   205G    96K  none
pfSense/ROOT/default                1.11G   205G  1.10G  /
pfSense/ROOT/default/cf             6.88M   205G  6.88M  /cf
pfSense/ROOT/default/var_cache_pkg   336K   205G   336K  /var/cache/pkg
pfSense/ROOT/default/var_db_pkg     5.42M   205G  5.42M  /var/db/pkg
pfSense/home                         108K   205G   108K  /home
pfSense/reservation                   96K   227G    96K  /pfSense/reservation
pfSense/tmp                         1.37M   205G  1.37M  /tmp
pfSense/var                         35.9M   205G  3.45M  /var
pfSense/var/cache                     96K   205G    96K  /var/cache
pfSense/var/db                      2.38M   205G  2.38M  /var/db
pfSense/var/empty                     96K   205G    96K  /var/empty
pfSense/var/log                     29.8M   205G  29.8M  /var/log
pfSense/var/tmp                      136K   205G   136K  /var/tmp

zfs set sync=always pfSense/ROOT/default/cf
                        M 1 Reply Last reply Reply Quote 0
                        • fireodoF
                          fireodo @valnar
                          last edited by

                          @valnar said in Another Netgate with storage failure, 6 in total so far:

                          If not, can somebody summarize the recommended tweaks?

                          These are my settings:

                          vfs.zfs.txg.timeout=120 (you can increase this value as you wish - I was gone until 2400 in experiments with no problems)
zfs set sync=disabled zroot/tmp (pfSense/tmp)
zfs set sync=disabled zroot/var (pfSense/var)

                          Regards,
                          fireodo

                          Kettop Mi4300YL CPU: i5-4300Y @ 1.60GHz RAM: 8GB Ethernet Ports: 4
                          SSD: SanDisk pSSD-S2 16GB (ZFS) WiFi: WLE200NX
                          pfsense 2.8.0 CE
                          Packages: Apcupsd, Cron, Iftop, Iperf, LCDproc, Nmap, pfBlockerNG, RRD_Summary, Shellcmd, Snort, Speedtest, System_Patches.

                          JonathanLeeJ 1 Reply Last reply Reply Quote 2
                          • M
                            Mission-Ghost @elvisimprsntr
                            last edited by

                            @elvisimprsntr Interesting. Why the inconsistency, I wonder?

                            1 Reply Last reply Reply Quote 0
                            • M
                              Mission-Ghost @fireodo
                              last edited by

                              @fireodo said in Another Netgate with storage failure, 6 in total so far:

                              If you make a entry in the "system tuning" the setting will survive even a system upgrade (if still necessary)...

                              Sweet. Thanks.

                              fireodoF 1 Reply Last reply Reply Quote 0
                              • fireodoF
                                fireodo @Mission-Ghost
                                last edited by

                                @Mission-Ghost said in Another Netgate with storage failure, 6 in total so far:

                                Sweet.

                                Nice that it tastes good 😁 😁

                                Kettop Mi4300YL CPU: i5-4300Y @ 1.60GHz RAM: 8GB Ethernet Ports: 4
                                SSD: SanDisk pSSD-S2 16GB (ZFS) WiFi: WLE200NX
                                pfsense 2.8.0 CE
                                Packages: Apcupsd, Cron, Iftop, Iperf, LCDproc, Nmap, pfBlockerNG, RRD_Summary, Shellcmd, Snort, Speedtest, System_Patches.

                                1 Reply Last reply Reply Quote 0
                                • JonathanLeeJ
                                  JonathanLee @fireodo
                                  last edited by JonathanLee

                                  @fireodo You should set this in cron right? Because this will default every reboot...

                                  zfs set sync=disabled pfSense/var
                                  zfs set sync=disabled pfSense/tmp

                                  should they no longer show if you run "zfs list" also?

                                  Make sure to upvote

                                  fireodoF 2 Replies Last reply Reply Quote 0
                                  • fireodoF
                                    fireodo @JonathanLee
                                    last edited by

                                    @JonathanLee said in Another Netgate with storage failure, 6 in total so far:

                                    You use the zfs set sync=disabled pfSense/var in cron right?

                                    No.

                                    Because this will default every reboot...

                                    Not here - once set it remains that way even after reboot. (btw. on 23.04.2025 the system was rebooted, after an long power outage that exhausted the UPS, caused by work on the power line by the provider)

                                    Kettop Mi4300YL CPU: i5-4300Y @ 1.60GHz RAM: 8GB Ethernet Ports: 4
                                    SSD: SanDisk pSSD-S2 16GB (ZFS) WiFi: WLE200NX
                                    pfsense 2.8.0 CE
                                    Packages: Apcupsd, Cron, Iftop, Iperf, LCDproc, Nmap, pfBlockerNG, RRD_Summary, Shellcmd, Snort, Speedtest, System_Patches.

                                    cmcdonaldC 1 Reply Last reply Reply Quote 0
                                    • fireodoF
                                      fireodo @JonathanLee
                                      last edited by

                                      @JonathanLee said in Another Netgate with storage failure, 6 in total so far:

                                      should they no longer show if you run "zfs list" also?

                                      You should see the status if you run:

                                      zfs get -r sync pfSense

                                      or

                                      zfs get -r sync zroot

                                      depending on what pfsense system you are.

                                      Kettop Mi4300YL CPU: i5-4300Y @ 1.60GHz RAM: 8GB Ethernet Ports: 4
                                      SSD: SanDisk pSSD-S2 16GB (ZFS) WiFi: WLE200NX
                                      pfsense 2.8.0 CE
                                      Packages: Apcupsd, Cron, Iftop, Iperf, LCDproc, Nmap, pfBlockerNG, RRD_Summary, Shellcmd, Snort, Speedtest, System_Patches.

                                      JonathanLeeJ 1 Reply Last reply Reply Quote 2
                                      • JonathanLeeJ
                                        JonathanLee @fireodo
                                        last edited by JonathanLee

                                        @fireodo Thank you for the reply,

                                        Shell Output - zfs get -r sync pfSense
NAME                                                                                PROPERTY  VALUE     SOURCE
pfSense                                                                             sync      standard  default
pfSense/ROOT                                                                        sync      standard  default
pfSense/ROOT/23_05_01_clone                                                         sync      standard  default
pfSense/ROOT/23_05_01_clone/cf                                                      sync      standard  default
pfSense/ROOT/23_05_01_clone/var_db_pkg                                              sync      standard  default
pfSense/ROOT/23_05_01_ipv4                                                          sync      standard  default
pfSense/ROOT/23_05_01_ipv4@2024-01-12-11:46:05-0                                    sync      -         -
pfSense/ROOT/23_05_01_ipv4@2024-02-29-08:52:57-0                                    sync      -         -
pfSense/ROOT/23_05_01_ipv4@2024-04-01-12:32:27-0                                    sync      -         -
pfSense/ROOT/23_05_01_ipv4@2024-06-27-11:52:26-0                                    sync      -         -
pfSense/ROOT/23_05_01_ipv4@2024-07-03-09:09:28-0                                    sync      -         -
pfSense/ROOT/23_05_01_ipv4@2025-01-20-10:11:49-0                                    sync      -         -
pfSense/ROOT/23_05_01_ipv4/cf                                                       sync      standard  default
pfSense/ROOT/23_05_01_ipv4/cf@2025-01-20-10:11:49-0                                 sync      -         -
pfSense/ROOT/23_05_01_ipv4/var_cache_pkg                                            sync      standard  default
pfSense/ROOT/23_05_01_ipv4/var_cache_pkg@2025-01-20-10:11:49-0                      sync      -         -
pfSense/ROOT/23_05_01_ipv4/var_db_pkg                                               sync      standard  default
pfSense/ROOT/23_05_01_ipv4/var_db_pkg@2025-01-20-10:11:49-0                         sync      -         -
pfSense/ROOT/23_05_01_ipv4_Backup                                                   sync      standard  default
pfSense/ROOT/23_05_01_ipv4_Backup/cf                                                sync      standard  default
pfSense/ROOT/23_05_01_ipv4_Backup/var_cache_pkg                                     sync      standard  default
pfSense/ROOT/23_05_01_ipv4_Backup/var_db_pkg                                        sync      standard  default
pfSense/ROOT/23_05_01_ipv6                                                          sync      standard  default
pfSense/ROOT/23_05_01_ipv6@2025-04-30-11:18:04-0                                    sync      -         -
pfSense/ROOT/23_05_01_ipv6/cf                                                       sync      standard  default
pfSense/ROOT/23_05_01_ipv6/cf@2025-04-30-11:18:04-0                                 sync      -         -
pfSense/ROOT/23_05_01_ipv6/var_db_pkg                                               sync      standard  default
pfSense/ROOT/23_05_01_ipv6/var_db_pkg@2025-04-30-11:18:04-0                         sync      -         -
pfSense/ROOT/23_05_01_ipv6_non_website_test_proxy                                   sync      standard  default
pfSense/ROOT/23_05_01_ipv6_non_website_test_proxy/cf                                sync      standard  default
pfSense/ROOT/23_05_01_ipv6_non_website_test_proxy/cf@2024-01-12-11:46:05-0          sync      -         -
pfSense/ROOT/23_05_01_ipv6_non_website_test_proxy/cf@2024-02-29-08:52:57-0          sync      -         -
pfSense/ROOT/23_05_01_ipv6_non_website_test_proxy/cf@2024-04-01-12:32:27-0          sync      -         -
pfSense/ROOT/23_05_01_ipv6_non_website_test_proxy/cf@2024-06-27-11:52:26-0          sync      -         -
pfSense/ROOT/23_05_01_ipv6_non_website_test_proxy/cf@2024-07-25-15:54:45-0          sync      -         -
pfSense/ROOT/23_05_01_ipv6_non_website_test_proxy/cf@2025-04-30-12:10:00-0          sync      -         -
pfSense/ROOT/23_05_01_ipv6_non_website_test_proxy/var_db_pkg                        sync      standard  default
pfSense/ROOT/23_05_01_ipv6_non_website_test_proxy/var_db_pkg@2024-01-12-11:46:05-0  sync      -         -
pfSense/ROOT/23_05_01_ipv6_non_website_test_proxy/var_db_pkg@2024-02-29-08:52:57-0  sync      -         -
pfSense/ROOT/23_05_01_ipv6_non_website_test_proxy/var_db_pkg@2024-04-01-12:32:27-0  sync      -         -
pfSense/ROOT/23_05_01_ipv6_non_website_test_proxy/var_db_pkg@2024-06-27-11:52:26-0  sync      -         -
pfSense/ROOT/23_05_01_ipv6_non_website_test_proxy/var_db_pkg@2024-07-25-15:54:45-0  sync      -         -
pfSense/ROOT/23_05_01_ipv6_non_website_test_proxy/var_db_pkg@2025-04-30-12:10:00-0  sync      -         -
pfSense/ROOT/23_05_01_ipv6_website_test_proxy_clone                                 sync      standard  default
pfSense/ROOT/23_05_01_ipv6_website_test_proxy_clone@2024-07-25-15:54:45-0           sync      -         -
pfSense/ROOT/23_05_01_ipv6_website_test_proxy_clone@2025-04-30-12:10:00-0           sync      -         -
pfSense/ROOT/23_05_01_ipv6_website_test_proxy_clone/cf                              sync      standard  default
pfSense/ROOT/23_05_01_ipv6_website_test_proxy_clone/var_db_pkg                      sync      standard  default
pfSense/ROOT/23_09_01_ipv4_20240703094025                                           sync      standard  default
pfSense/ROOT/23_09_01_ipv4_20240703094025/cf                                        sync      standard  default
pfSense/ROOT/23_09_01_ipv4_20240703094025/var_cache_pkg                             sync      standard  default
pfSense/ROOT/23_09_01_ipv4_20240703094025/var_db_pkg                                sync      standard  default
pfSense/ROOT/24_03_01_ipv4                                                          sync      standard  default
pfSense/ROOT/24_03_01_ipv4/cf                                                       sync      standard  default
pfSense/ROOT/24_03_01_ipv4/var_cache_pkg                                            sync      standard  default
pfSense/ROOT/24_03_01_ipv4/var_db_pkg                                               sync      standard  default
pfSense/ROOT/24_03_01_ipv6_20250113135850                                           sync      standard  default
pfSense/ROOT/24_03_01_ipv6_20250113135850@2024-07-03-09:40:36-0                     sync      -         -
pfSense/ROOT/24_03_01_ipv6_20250113135850@2024-07-23-10:05:22-0                     sync      -         -
pfSense/ROOT/24_03_01_ipv6_20250113135850@2025-01-13-13:59:02-0                     sync      -         -
pfSense/ROOT/24_03_01_ipv6_20250113135850/cf                                        sync      standard  default
pfSense/ROOT/24_03_01_ipv6_20250113135850/cf@2024-07-03-09:09:28-0                  sync      -         -
pfSense/ROOT/24_03_01_ipv6_20250113135850/cf@2024-07-03-09:40:36-0                  sync      -         -
pfSense/ROOT/24_03_01_ipv6_20250113135850/cf@2024-07-23-10:05:22-0                  sync      -         -
pfSense/ROOT/24_03_01_ipv6_20250113135850/cf@2025-01-13-13:59:02-0                  sync      -         -
pfSense/ROOT/24_03_01_ipv6_20250113135850/var_cache_pkg                             sync      standard  default
pfSense/ROOT/24_03_01_ipv6_20250113135850/var_cache_pkg@2024-07-03-09:09:28-0       sync      -         -
pfSense/ROOT/24_03_01_ipv6_20250113135850/var_cache_pkg@2024-07-03-09:40:36-0       sync      -         -
pfSense/ROOT/24_03_01_ipv6_20250113135850/var_cache_pkg@2024-07-23-10:05:22-0       sync      -         -
pfSense/ROOT/24_03_01_ipv6_20250113135850/var_cache_pkg@2025-01-13-13:59:02-0       sync      -         -
pfSense/ROOT/24_03_01_ipv6_20250113135850/var_db_pkg                                sync      standard  default
pfSense/ROOT/24_03_01_ipv6_20250113135850/var_db_pkg@2024-07-03-09:09:28-0          sync      -         -
pfSense/ROOT/24_03_01_ipv6_20250113135850/var_db_pkg@2024-07-03-09:40:36-0          sync      -         -
pfSense/ROOT/24_03_01_ipv6_20250113135850/var_db_pkg@2024-07-23-10:05:22-0          sync      -         -
pfSense/ROOT/24_03_01_ipv6_20250113135850/var_db_pkg@2025-01-13-13:59:02-0          sync      -         -
pfSense/ROOT/auto-default-20240112115753                                            sync      standard  default
pfSense/ROOT/auto-default-20240112115753@2024-01-12-11:57:53-0                      sync      -         -
pfSense/ROOT/auto-default-20240112115753/cf                                         sync      standard  default
pfSense/ROOT/auto-default-20240112115753/cf@2024-01-12-11:57:53-0                   sync      -         -
pfSense/ROOT/auto-default-20240112115753/var_cache_pkg                              sync      standard  default
pfSense/ROOT/auto-default-20240112115753/var_cache_pkg@2024-01-12-11:57:53-0        sync      -         -
pfSense/ROOT/auto-default-20240112115753/var_db_pkg                                 sync      standard  default
pfSense/ROOT/auto-default-20240112115753/var_db_pkg@2024-01-12-11:57:53-0           sync      -         -
pfSense/ROOT/quick-20240401123227                                                   sync      standard  default
pfSense/ROOT/quick-20240401123227/cf                                                sync      standard  default
pfSense/ROOT/quick-20240401123227/var_db_pkg                                        sync      standard  default
pfSense/home                                                                        sync      standard  default
pfSense/reservation                                                                 sync      standard  default
pfSense/tmp                                                                         sync      disabled  local
pfSense/var                                                                         sync      disabled  local
pfSense/var/cache                                                                   sync      disabled  inherited from pfSense/var
pfSense/var/db                                                                      sync      disabled  inherited from pfSense/var
pfSense/var/log                                                                     sync      disabled  inherited from pfSense/var
pfSense/var/tmp                                                                     sync      disabled  inherited from pfSense/var

                                        logs and temp are now excluded from sync

                                        and I also set

                                        vfs.zfs.txg.timeout=120

                                        I also use a secondary nvme drive just for package logs I have a mount point and created linker files that direct everything that is write intensive over to it. That helps alot also, and or you can use a usb thumb drive for package logs if anyone is really worried.

                                        691b97ae-7284-4f19-af85-e2f8ac1c14a8-image.png

                                        https://forum.netgate.com/topic/195843/unofficial-guide-have-package-logs-record-to-a-secondary-ssd-drive-snort-syslog-squid-and-or-squid-cache-system/

                                        Make sure to upvote

                                        1 Reply Last reply Reply Quote 2
                                        • cmcdonaldC
                                          cmcdonald Netgate Developer @fireodo
                                          last edited by

                                          @fireodo said in Another Netgate with storage failure, 6 in total so far:

                                          Not here - once set it remains that way even after reboot.

                                          Yes. This is correct. ZFS properties persist across reboots (unless they are changed by something during startup, say in pfSense-rc, though I don't think we do any of that)

                                          Need help fast? https://www.netgate.com/support

                                          1 Reply Last reply Reply Quote 1
                                          • w0wW w0w referenced this topic
                                          • D
                                            dugeem
                                            last edited by

                                            One additional issue to be aware of is that up until pfSense+ 24.03 the Netgate SG-1100 (& SG-2100) installation images resulted in eMMC having non aligned flash partitions. This can result in file system activity causing sub optimal block writing (due to sectors crossing erase boundaries) resulting in increased flash wear.

                                            pfSense Redmine reference -> https://redmine.pfsense.org/issues/15126

                                            Easy to check - from shell use the command:

                                            gpart show mmcsd0

                                            and check numbers in first column for either freebsd / freebsd-zfs partitions are divisible by 8 (or a higher power of 2).

                                            Generally if your SG-1100 (also SG-2100) was originally commissioned prior to pfSense+ 24.03 then you should consider reinstalling and restoring config.

                                            Relevant Netgate SG-1100 documentation -> https://docs.netgate.com/pfsense/en/latest/solutions/sg-1100/reinstall-pfsense.html

                                            1 Reply Last reply Reply Quote 3
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.