Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Average writes to Disk

    Scheduled Pinned Locked Moved General pfSense Questions
    71 Posts 13 Posters 12.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • fireodoF
      fireodo @johnpoz
      last edited by fireodo

      @johnpoz

      For my system I took these settings (considering also jimps opinion)

      zfs set sync=disabled zroot/tmp (pfSense/tmp on new 2.5.2 install)
      zfs set sync=standard zroot/var (pfSense/var)

      and fine tuning: vfs.zfs.txg.timeout=120

      I hope to get a balance between security and the write intensity to disk. I will report in the next 24 hours.

      Thank you all for your Patience and help!
      fireodo

      Kettop Mi4300YL CPU: i5-4300Y @ 1.60GHz RAM: 8GB Ethernet Ports: 4
      SSD: SanDisk pSSD-S2 16GB (ZFS) WiFi: WLE200NX
      pfsense 2.7.2 CE
      Packages: Apcupsd Cron Iftop Iperf LCDproc Nmap pfBlockerNG RRD_Summary Shellcmd Snort Speedtest System_Patches.

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator @johnpoz
        last edited by

        @johnpoz said in Average writes to Disk:

        edit: @stephenw10 that looks interesting.. Must be something new for 21.09, I do not see any sort of graph available in 21.05.1... Or is that something outside of pfsense, like vm host where you have pfsense running?

        Yeah, sorry, that's a VM in Proxmox. Waaaay easier to see the actual disk use there IMO.

        Steve

        johnpozJ 1 Reply Last reply Reply Quote 0
        • johnpozJ
          johnpoz LAYER 8 Global Moderator @stephenw10
          last edited by

          @stephenw10 said in Average writes to Disk:

          Yeah, sorry, that's a VM in Proxmox.

          For a second there - thought we were getting some new information graphs ;) In light of this discussion, such info in the monitor graphs would be slick.. Especially if could show total amount of data written lifetime, etc.

          An intelligent man is sometimes forced to be drunk to spend time with his fools
          If you get confused: Listen to the Music Play
          Please don't Chat/PM me for help, unless mod related
          SG-4860 24.11 | Lab VMs 2.7.2, 24.11

          fireodoF 1 Reply Last reply Reply Quote 1
          • fireodoF
            fireodo @johnpoz
            last edited by fireodo

            As I promised:

            after 24 hours the main system (with the settings above) reduced the writings from about 20GB/day to about 1,8GB/day :-)
            edit1: after another 24h the writes have settled at 0,89 GB/day ;-)

            edit2: after about 2 weeks the writes come to a middle of 0,70GB/day 👍 (for those who are interested)

            Kettop Mi4300YL CPU: i5-4300Y @ 1.60GHz RAM: 8GB Ethernet Ports: 4
            SSD: SanDisk pSSD-S2 16GB (ZFS) WiFi: WLE200NX
            pfsense 2.7.2 CE
            Packages: Apcupsd Cron Iftop Iperf LCDproc Nmap pfBlockerNG RRD_Summary Shellcmd Snort Speedtest System_Patches.

            johnpozJ 1 Reply Last reply Reply Quote 1
            • johnpozJ
              johnpoz LAYER 8 Global Moderator @fireodo
              last edited by johnpoz

              That is a signification change to be sure..

              How much does UFS write?

              An intelligent man is sometimes forced to be drunk to spend time with his fools
              If you get confused: Listen to the Music Play
              Please don't Chat/PM me for help, unless mod related
              SG-4860 24.11 | Lab VMs 2.7.2, 24.11

              fireodoF 1 Reply Last reply Reply Quote 0
              • fireodoF
                fireodo @bmeeks
                last edited by

                @bmeeks said in Average writes to Disk:

                This may be a classic case of you can't "have your cake and eat it, too" ... .

                You're right, its always a balance and taking the decision goes hand in in hand with support the consequences :-)

                Kettop Mi4300YL CPU: i5-4300Y @ 1.60GHz RAM: 8GB Ethernet Ports: 4
                SSD: SanDisk pSSD-S2 16GB (ZFS) WiFi: WLE200NX
                pfsense 2.7.2 CE
                Packages: Apcupsd Cron Iftop Iperf LCDproc Nmap pfBlockerNG RRD_Summary Shellcmd Snort Speedtest System_Patches.

                1 Reply Last reply Reply Quote 0
                • fireodoF
                  fireodo @johnpoz
                  last edited by

                  @johnpoz said in Average writes to Disk:

                  That is a signification change to be sure..

                  Indeed.

                  How much does UFS write?

                  I have no unit with UFS so I cannot say. The unit I had setup with UFS is in the meantime on ZFS. But I remember the writings where much lower ...

                  Kettop Mi4300YL CPU: i5-4300Y @ 1.60GHz RAM: 8GB Ethernet Ports: 4
                  SSD: SanDisk pSSD-S2 16GB (ZFS) WiFi: WLE200NX
                  pfsense 2.7.2 CE
                  Packages: Apcupsd Cron Iftop Iperf LCDproc Nmap pfBlockerNG RRD_Summary Shellcmd Snort Speedtest System_Patches.

                  johnpozJ 1 Reply Last reply Reply Quote 0
                  • johnpozJ
                    johnpoz LAYER 8 Global Moderator @fireodo
                    last edited by

                    Well my question is then - if your going to drastically reduce the sync to disk which is one of the advantages of zfs.

                    Until such time that some of the other things are available for easy use, like snapshots and say rollback of updates that zfs could bring to the table.

                    If your concerned about the amount of writes because of ssd or emmc, wouldn't it make more sense to just use ufs?

                    An intelligent man is sometimes forced to be drunk to spend time with his fools
                    If you get confused: Listen to the Music Play
                    Please don't Chat/PM me for help, unless mod related
                    SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                    fireodoF 1 Reply Last reply Reply Quote 0
                    • fireodoF
                      fireodo @johnpoz
                      last edited by fireodo

                      @johnpoz said in Average writes to Disk:

                      If your concerned about the amount of writes because of ssd or emmc, wouldn't it make more sense to just use ufs?

                      Thats indeed an argument. But ... even with those "adjustments" ZFS is still much more robust and those disabled syncs (/tmp /var) are not compromising the consistency of the FS pool (zroot resp. pfSense). (I read that in a documentation about ZFS) (https://docs.oracle.com/cd/E26217_01/E35769/html/performance-storage.html)
                      BTW. This are tweaks for me and my environment - a pfsense in a enterprise environment I would leave untouched in this matter.

                      Kettop Mi4300YL CPU: i5-4300Y @ 1.60GHz RAM: 8GB Ethernet Ports: 4
                      SSD: SanDisk pSSD-S2 16GB (ZFS) WiFi: WLE200NX
                      pfsense 2.7.2 CE
                      Packages: Apcupsd Cron Iftop Iperf LCDproc Nmap pfBlockerNG RRD_Summary Shellcmd Snort Speedtest System_Patches.

                      1 Reply Last reply Reply Quote 1
                      • fireodoF fireodo referenced this topic on
                      • J
                        jacol @tomashk
                        last edited by

                        @tomashk Thanks for this, on my pfsense 2.7.2 I used this commands

                        on CLI
                        zfs set sync=disabled pfSense
                        System -> Advanced -> System Tunables - set vfs.zfs.txg.timeout to 180

                        dropped from 28% IO delay to 5 %
                        26dfeae6-4122-40bc-a1d3-2ad978396090-image.png

                        1 Reply Last reply Reply Quote 2
                        • A andrew_cb referenced this topic on
                        • fireodoF fireodo referenced this topic on
                        • First post
                          Last post
                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.