How do I find out what write continuously on my pfSense SSD

slu

@Gertjan said in How do I find out what write continuously on my pfSense SSD:

so your SSD is accessible and replaceable ?

Yes no problem to replace the M.2 NVMe SSD.

@Gertjan said in How do I find out what write continuously on my pfSense SSD:

Classic iron disk don't last 'forever' neither. Some last over 10 years, some died within a couple of yours.

That's true.

@Gertjan said in How do I find out what write continuously on my pfSense SSD:

IMHO : stay away from 'RAM drives'. They solve some issues, by creating other ones.

This is not clear to me, what issues can it create?
I have enough RAM (16 GB), so it will not be an issue.

On say use RAM disk, the other say not, I'm really unsure at the moment.

stephenw10

Enable it and see what happens. I usually start out at double the minimum values but since you're running some packages etc and have RAM to spare I'd start out at least 4x. So 160 + 240 MB. If somethings fail to start at boot double it again.

Gertjan

@slu said in How do I find out what write continuously on my pfSense SSD:

This is not clear to me, what issues can it create?

The ones I've never seen before ^^
Remember : I've a 4100 and I went for the MAX, but that didn't MAXed out the RAM, though.
So with a mere 4 Gbytes, still very ok to run pfSense, I can't go wild with the big resource eating packages, or use options like "pfBlockerng with all feeds selected ^^"

More serious : every non programmed power down will destroy all recent stats ...

Also : It gives a blow to the KIS concept.

stephenw10

Yup you lose the RAM disk content if power is lost. But only since the last time it wrote out.

Bob.Dig

@stephenw10 But loosing its content is what is wanted here anyways. If it would wrote everything to the disk at some point, there is nothing gained from this.

stephenw10

Writing periodically is still far fewer disk writes. And not everything is written out like that only the RRD files. The other ram disk content is only backed up at shutdown.

Gertjan

@Bob-Dig said in How do I find out what write continuously on my pfSense SSD:

there is nothing gained from this.

There is.
During shutdown, the set of files will get written out to the 'real' disk.
But de xxxxx times the files were updated are all pure bonus.

bmeeks

@Gertjan said in How do I find out what write continuously on my pfSense SSD:

@bmeeks No coffee this morning ?
At line 6530, a SQLite database is queried (read operation) for the 'resolver' field which contains the current PID of unbound.
If it changed = restart of unbound, then that field is updated, and that's a (very small) write operation.

I see both a "read" query and a "write" operation in that function loop. Notice it does a SQL SELECT and then a SQL INSERT INTO depending on what the query found. I'm not sure of all the logic as I did not author this code, so perhaps the write operation is only occasional. But it looks like all this is associated with the Dashboard Widget and is how it gathers and stores the DNS query statistics for populating the pretty graph. It follows then that the more DNS activity you have on your network the more database writes you have happening so long as the Dashboard Widget is active.

slu

@stephenw10

I enable the RAM disk and it look like better now:

iostat -d 5 6
            ada0            pass0            pass1 
 KB/t  tps  MB/s  KB/t  tps  MB/s  KB/t  tps  MB/s 
 20.9   10   0.2   0.4    0   0.0   0.0    0   0.0 
  0.0    0   0.0   0.0    0   0.0   0.0    0   0.0 
  0.0    0   0.0   0.0    0   0.0   0.0    0   0.0 
  0.0    0   0.0   0.0    0   0.0   0.0    0   0.0 
  0.0    0   0.0   0.0    0   0.0   0.0    0   0.0 
  0.0    0   0.0   0.0    0   0.0   0.0    0   0.0

As far as I understand everything is saved during normal shutdown/reboot, only power outage is an problem?
In this case pfSense should boot, but missing maybe some data (no config settings)?

slu

@bmeeks said in How do I find out what write continuously on my pfSense SSD:

It follows then that the more DNS activity you have on your network the more database writes you have happening so long as the Dashboard Widget is active.

We have much DNS activity...

stephenw10

@slu said in How do I find out what write continuously on my pfSense SSD:

but missing maybe some data (no config settings)?

I doesn't lose config. It will lose logs and RRD data since the last periodic write out. It loses cached stuff that will need to be re-downloaded at the next boot.

I run ram disks here on numerous machines and never see any significant issues. Even when deliberately trying!

slu

@stephenw10
this should added to the documentation (IMHO), if someone enable pfBlockerNG on a 1100
it will write "down" the storage.

Maybe an hint here:
https://docs.netgate.com/pfsense/en/latest/packages/pfblocker.html

I learned (again) many thinks of pfSense / FreeBSD.
Thanks @bmeeks @Gertjan @stephenw10

NOCling

This wasn't a big issue with UFS, but it has become a problem with the functionality of ZFS.
With a RAM disk this is no longer a problem, so it should be included in the documentation.

Keep an eye on the RAM disk values when you trigger the pfBlocker reload, then you need space for all the tables.
That's why I use 256/2048MB.

slu

RAM disk works as expected, what a huge difference!

SSD Write.png

slu

Now more than four months later we had the same written data units like before in one week without the ram disk.

Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    68%
Data Units Read:                    25,221 [12.9 GB]
Data Units Written:                 93,370,737 [47.8 TB]
Host Read Commands:                 4,428,272
Host Write Commands:                4,340,924,956
Controller Busy Time:               42,507
Power Cycles:                       25
Power On Hours:                     16,659

michmoor

@stephenw10 curious but do most customers run RAMDISK as part of their standard deployment or is it used mainly for systems with weaker storage drives?

stephenw10

No, RAM disks are pretty rare amongst the general customer base in my experience.

slu

@michmoor said in How do I find out what write continuously on my pfSense SSD:

weaker storage drives?

Define a weaker storage drive?
I my case it is a server SSD with 128 GB, but 47 TB in 1 1/2 year is to much for a router...

stephenw10

Generally, eMMC or CF/SD cards vs SSDs.

Bob.Dig

@NOCling said in How do I find out what write continuously on my pfSense SSD:

Keep an eye on the RAM disk values when you trigger the pfBlocker reload, then you need space for all the tables.
That's why I use 256/2048MB.

Is there a way to monitor it easily? And are your values failsafe? I just noticed I had a much to low value for /tmp...