• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

Average writes to Disk

General pfSense Questions
13
71
11.8k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • F
    fireodo
    last edited by fireodo Aug 11, 2021, 2:51 PM Aug 11, 2021, 8:23 AM

    Hi,

    as I had, a wile a go, a sudden death of a SSD i took a look into my pfsense and I was astonished to see that my pfsense is writing about 20GB (TWENTY) per Day to Disk. A Lab Installation with almost no auxiliary packages comes to aprox. 14-18GB/Day.
    Could you take a look at your machines and write down how much writes you have?
    Just for curiosity!

    Thank you all,
    fireodo
    (My config is visible in my signature)

    Kettop Mi4300YL CPU: i5-4300Y @ 1.60GHz RAM: 8GB Ethernet Ports: 4
    SSD: SanDisk pSSD-S2 16GB (ZFS) WiFi: WLE200NX
    pfsense 2.7.2 CE
    Packages: Apcupsd Cron Iftop Iperf LCDproc Nmap pfBlockerNG RRD_Summary Shellcmd Snort Speedtest System_Patches.

    G 1 Reply Last reply Aug 11, 2021, 12:14 PM Reply Quote 1
    • G
      Gertjan @fireodo
      last edited by Aug 11, 2021, 12:14 PM

      @fireodo said in Average writes to Disk:

      (My config is visible in my signature)

      These :
      Iftop,
      Iperf,
      pfBlockerNG-devel,
      Snort

      operate on every packet that comes in, to make logs, stats, etc.

      You were asking for details, and you are served details. These packages are created for that very reason.
      Details have to be stored somewhere.

      Easy to test - proof the contrary : remove the WAN cable and check that writing comes to a "stand still".

      RRD_Summary : I'm not sure.

      No "help me" PM's please. Use the forum, the community will thank you.
      Edit : and where are the logs ??

      1 Reply Last reply Reply Quote 0
      • A
        AndyRH
        last edited by Aug 11, 2021, 2:49 PM

        It would be useful if the install allowed the user to set an amount of space to be left unpartitioned. Some modern SSDs can use un-partitioned space to map out failed blocks. Some SSDs have this space already hidden. This option would help prevent a sudden failure from one of the 2 probable causes, wear. SMART would have time to warn and be noticed. A controller failure remains sudden and unpredictable.

        o||||o
        7100-1u

        1 Reply Last reply Reply Quote 1
        • S
          stephenw10 Netgate Administrator
          last edited by Aug 12, 2021, 1:34 PM

          Are either of those old installs that have been upgraded?

          There was a bug at one time that was not mounting root as noatime. That increased drive writes in normal use. Check how it's mounted with mount -p

          Steve

          F S 2 Replies Last reply Aug 12, 2021, 1:59 PM Reply Quote 0
          • F
            fireodo @stephenw10
            last edited by stephenw10 Aug 12, 2021, 2:10 PM Aug 12, 2021, 1:59 PM

            @stephenw10 said in Average writes to Disk:

            Are either of those old installs that have been upgraded?

            One is fesh install with 2.5.1 and upgradet to 2.5.2 and the Lab machine is a fresh install with 2.5.2.

            mount -p
            zroot/ROOT/default      /                       zfs     rw,noatime,nfsv4acls    0 0
            devfs                   /dev                    devfs   rw              0 0
            zroot/tmp               /tmp                    zfs     rw,nosuid,noatime,nfsv4acls     0 0
            zroot/var               /var                    zfs     rw,noatime,nfsv4acls    0 0
            zroot                   /zroot                  zfs     rw,noatime,nfsv4acls    0 0
            /dev/md0                /var/run                ufs     rw              2 2
            devfs                   /var/dhcpd/dev          devfs   rw              0 0
            /usr/local/bin          /var/unbound/usr/local/bin nullfs       ro,noatime,nfsv4acls    0 0
            /usr/local/lib          /var/unbound/usr/local/lib nullfs       ro,noatime,nfsv4acls    0 0
            /lib                    /var/unbound/lib        nullfs  ro,noatime,nfsv4acls    0 0
            devfs                   /var/unbound/dev        devfs   rw              0 0
            /var/log/pfblockerng    /var/unbound/var/log/pfblockerng nullfs rw,noatime,nfsv4acls    0 0
            /usr/local/share/GeoIP  /var/unbound/usr/local/share/GeoIP nullfs       ro,noatime,nfsv4acls    0 0
            

            Kettop Mi4300YL CPU: i5-4300Y @ 1.60GHz RAM: 8GB Ethernet Ports: 4
            SSD: SanDisk pSSD-S2 16GB (ZFS) WiFi: WLE200NX
            pfsense 2.7.2 CE
            Packages: Apcupsd Cron Iftop Iperf LCDproc Nmap pfBlockerNG RRD_Summary Shellcmd Snort Speedtest System_Patches.

            1 Reply Last reply Reply Quote 0
            • S
              stephenw10 Netgate Administrator
              last edited by Aug 12, 2021, 2:12 PM

              What are you using to check the drive writes? What's the actual data it's giving you?

              F 1 Reply Last reply Aug 12, 2021, 2:19 PM Reply Quote 0
              • F
                fireodo @stephenw10
                last edited by Aug 12, 2021, 2:19 PM

                @stephenw10 said in Average writes to Disk:

                What are you using to check the drive writes? What's the actual data it's giving you?

                On the one hand the smart value of LBA_written and on the other hand iostat -d 5 6.

                Kettop Mi4300YL CPU: i5-4300Y @ 1.60GHz RAM: 8GB Ethernet Ports: 4
                SSD: SanDisk pSSD-S2 16GB (ZFS) WiFi: WLE200NX
                pfsense 2.7.2 CE
                Packages: Apcupsd Cron Iftop Iperf LCDproc Nmap pfBlockerNG RRD_Summary Shellcmd Snort Speedtest System_Patches.

                1 Reply Last reply Reply Quote 0
                • S
                  stephenw10 Netgate Administrator
                  last edited by Aug 12, 2021, 2:25 PM

                  And what values are you actually seeing there?

                  F 1 Reply Last reply Aug 12, 2021, 2:28 PM Reply Quote 0
                  • F
                    fireodo @stephenw10
                    last edited by stephenw10 Aug 12, 2021, 2:56 PM Aug 12, 2021, 2:28 PM

                    @stephenw10 said in Average writes to Disk:

                    And what values are you actually seeing there?

                    iostat -d 5 6
                                 md0             ada0            pass0
                      KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s
                      0.00   0  0.00  15.55  14  0.21   0.38   0  0.00
                      0.00   0  0.00  12.80  22  0.27   0.00   0  0.00
                      0.00   0  0.00  14.67  15  0.21   0.00   0  0.00
                      0.00   0  0.00  12.59  14  0.17   0.00   0  0.00
                      0.00   0  0.00  14.67  14  0.20   0.00   0  0.00
                      0.00   0  0.00  13.56  14  0.19   0.00   0  0.00
                    

                    Using mathematics it comes to about 20GB/day

                    PS. On the LAB machine (the fresh 2.5.2 install) there are no other packages running. Basic pfsense.

                    Kettop Mi4300YL CPU: i5-4300Y @ 1.60GHz RAM: 8GB Ethernet Ports: 4
                    SSD: SanDisk pSSD-S2 16GB (ZFS) WiFi: WLE200NX
                    pfsense 2.7.2 CE
                    Packages: Apcupsd Cron Iftop Iperf LCDproc Nmap pfBlockerNG RRD_Summary Shellcmd Snort Speedtest System_Patches.

                    1 Reply Last reply Reply Quote 0
                    • S
                      SteveITS Galactic Empire @stephenw10
                      last edited by Aug 12, 2021, 3:23 PM

                      @stephenw10 said in Average writes to Disk:

                      There was a bug at one time that was not mounting root as noatime

                      I looked, as we have some older routers, and found a few.

                      /dev/gptid/4eca01a4-9c0d-11e7-bbd9-0008a20cdf70 / ufs rw 1 1
                      devfs /dev devfs rw 0 0
                      /dev/md0 /var/run ufs rw 2 2
                      devfs /var/dhcpd/dev devfs rw 0 0

                      Just change rw to rw,noatime in /etc/fstab, and reboot?

                      Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
                      When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
                      Upvote 👍 helpful posts!

                      F 1 Reply Last reply Aug 12, 2021, 3:35 PM Reply Quote 0
                      • F
                        fireodo @SteveITS
                        last edited by fireodo Aug 12, 2021, 3:38 PM Aug 12, 2021, 3:35 PM

                        @steveits said in Average writes to Disk:

                        Just change rw to rw,noatime in /etc/fstab, and reboot?

                        In /etc/fstab there is only that:

                        "# Device Mountpoint FStype Options Dump Pass#
                        /dev/ada0p2 none swap sw 0 0"

                        And if you look in the beginning of the thread there is the output of mount -p - most every FS is mounted noatime.

                        Kettop Mi4300YL CPU: i5-4300Y @ 1.60GHz RAM: 8GB Ethernet Ports: 4
                        SSD: SanDisk pSSD-S2 16GB (ZFS) WiFi: WLE200NX
                        pfsense 2.7.2 CE
                        Packages: Apcupsd Cron Iftop Iperf LCDproc Nmap pfBlockerNG RRD_Summary Shellcmd Snort Speedtest System_Patches.

                        S 1 Reply Last reply Aug 12, 2021, 3:39 PM Reply Quote 0
                        • S
                          stephenw10 Netgate Administrator
                          last edited by Aug 12, 2021, 3:38 PM

                          Yup, exactly, just edit the fstab if you have some that were created at that time.

                          ZFS does write to the drive more than UFS by design. I'm not sure what might be done there.

                          Enabling RAM disks reduces it significantly but you wouldn't want to do that with Snort and pfBlocker running.

                          Steve

                          F 2 Replies Last reply Aug 12, 2021, 3:40 PM Reply Quote 0
                          • S
                            stephenw10 Netgate Administrator @fireodo
                            last edited by Aug 12, 2021, 3:39 PM

                            @fireodo said in Average writes to Disk:

                            In /etc/fstab there is only that:

                            Because you're running ZFS which doesn't use the fstab.

                            1 Reply Last reply Reply Quote 0
                            • F
                              fireodo @stephenw10
                              last edited by Aug 12, 2021, 3:40 PM

                              @stephenw10 said in Average writes to Disk:

                              Enabling RAM disks reduces it significantly but you wouldn't want to do that with Snort and pfBlocker running.

                              That was my idea too but as you say - no way with snort&pfblockerNG ...

                              Kettop Mi4300YL CPU: i5-4300Y @ 1.60GHz RAM: 8GB Ethernet Ports: 4
                              SSD: SanDisk pSSD-S2 16GB (ZFS) WiFi: WLE200NX
                              pfsense 2.7.2 CE
                              Packages: Apcupsd Cron Iftop Iperf LCDproc Nmap pfBlockerNG RRD_Summary Shellcmd Snort Speedtest System_Patches.

                              1 Reply Last reply Reply Quote 0
                              • F
                                fireodo @stephenw10
                                last edited by fireodo Aug 12, 2021, 3:48 PM Aug 12, 2021, 3:42 PM

                                @stephenw10 said in Average writes to Disk:

                                ZFS does write to the drive more than UFS by design.

                                Do you think that could be from ZFS? Because I have stopped most every service that can be stopped but the writings go on ...

                                On the other hand - is my installation so particular?

                                Kettop Mi4300YL CPU: i5-4300Y @ 1.60GHz RAM: 8GB Ethernet Ports: 4
                                SSD: SanDisk pSSD-S2 16GB (ZFS) WiFi: WLE200NX
                                pfsense 2.7.2 CE
                                Packages: Apcupsd Cron Iftop Iperf LCDproc Nmap pfBlockerNG RRD_Summary Shellcmd Snort Speedtest System_Patches.

                                J 1 Reply Last reply Aug 12, 2021, 3:48 PM Reply Quote 0
                                • J
                                  johnpoz LAYER 8 Global Moderator @fireodo
                                  last edited by johnpoz Aug 12, 2021, 3:51 PM Aug 12, 2021, 3:48 PM

                                  @fireodo what ssd are you using? Modern SSD even with 20GB a day you should still be like a 10 year lifespan..

                                  I just looked up a shit $20 128GB ssd from lexar the ns100, and it lists 60TB write life.. at 20GB a day your still looking at 8 some years..

                                  Off the cuff 20GB a day does seem like a lot... But even at that you should get decent life out of the SSD..

                                  An intelligent man is sometimes forced to be drunk to spend time with his fools
                                  If you get confused: Listen to the Music Play
                                  Please don't Chat/PM me for help, unless mod related
                                  SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                                  F 1 Reply Last reply Aug 12, 2021, 3:55 PM Reply Quote 0
                                  • F
                                    fireodo @johnpoz
                                    last edited by fireodo Aug 12, 2021, 3:59 PM Aug 12, 2021, 3:55 PM

                                    @johnpoz said in Average writes to Disk:

                                    what ssd are you using?

                                    See my signature :-)

                                    I already ordered some spare for ... just in case ;-)

                                    Kettop Mi4300YL CPU: i5-4300Y @ 1.60GHz RAM: 8GB Ethernet Ports: 4
                                    SSD: SanDisk pSSD-S2 16GB (ZFS) WiFi: WLE200NX
                                    pfsense 2.7.2 CE
                                    Packages: Apcupsd Cron Iftop Iperf LCDproc Nmap pfBlockerNG RRD_Summary Shellcmd Snort Speedtest System_Patches.

                                    J 1 Reply Last reply Aug 12, 2021, 4:05 PM Reply Quote 0
                                    • J
                                      johnpoz LAYER 8 Global Moderator @fireodo
                                      last edited by Aug 12, 2021, 4:05 PM

                                      @fireodo I don't see anything listed for 16GB size.. But the 64 is listed as

                                      36.5TBW (64GB)

                                      So you have to assume lower than that.. So yeah, prob not the best drives for lots of writes..

                                      Lets call it half that.. Your looking at like 2.5 years only..

                                      An intelligent man is sometimes forced to be drunk to spend time with his fools
                                      If you get confused: Listen to the Music Play
                                      Please don't Chat/PM me for help, unless mod related
                                      SG-4860 24.11 | Lab VMs 2.7.2, 24.11

                                      F 2 Replies Last reply Aug 12, 2021, 4:17 PM Reply Quote 0
                                      • F
                                        fireodo @johnpoz
                                        last edited by Aug 12, 2021, 4:17 PM

                                        @johnpoz said in Average writes to Disk:

                                        @fireodo I don't see anything listed for 16GB size.. But the 64 is listed as

                                        36.5TBW (64GB)

                                        So you have to assume lower than that.. So yeah, prob not the best drives for lots of writes..

                                        Lets call it half that.. Your looking at like 2.5 years only..

                                        Of course this all sounds reasonable but for me its strange that a older APU (that I have as quick replacement in case of hardware failure - i do much HomeOffice) with 2.4.5 shows insignificant writes to disk (also ZFS) far far away from 20GB/day.

                                        Kettop Mi4300YL CPU: i5-4300Y @ 1.60GHz RAM: 8GB Ethernet Ports: 4
                                        SSD: SanDisk pSSD-S2 16GB (ZFS) WiFi: WLE200NX
                                        pfsense 2.7.2 CE
                                        Packages: Apcupsd Cron Iftop Iperf LCDproc Nmap pfBlockerNG RRD_Summary Shellcmd Snort Speedtest System_Patches.

                                        1 Reply Last reply Reply Quote 0
                                        • A
                                          AndyRH
                                          last edited by Aug 12, 2021, 4:19 PM

                                          Extra space will get longer life, more room for TRIM to work with and at todays prices it is worth the investment IMHO.

                                          o||||o
                                          7100-1u

                                          F 1 Reply Last reply Aug 12, 2021, 4:29 PM Reply Quote 0
                                          8 out of 71
                                          • First post
                                            8/71
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.