SG-3100 ram disk question

SteveITS

@mcury It should be in the config. Look for:

<use_mfs_tmp_size>128</use_mfs_tmp_size>
<use_mfs_var_size>256</use_mfs_var_size>
and
<use_mfs_tmpvar/>

mcury

@steveits Oh yes, you are correct, I don't know how I missed that :)

So, to disable I can just remove that line:

use_mfs_tmpvar></use_mfs_tmpvar>

That is useful because I noticed that when you perform a clean install and restore a config, during packages reinstall there is a lot of RAM usage, which could break things for me..

stephenw10

Yes, that's the config line that triggers creating and using the ramdisks.

You'll find that the usage is much higher when pfBlocker updates. Probably around double what you see there now.

Steve

mcury

@stephenw10 said in SG-3100 ram disk question:

You'll find that the usage is much higher when pfBlocker updates. Probably around double what you see there now.

hm, ramdisk for /var is set to 850M, and not problems yet, I've been monitoring and so far so good.
Do you think I'll face problems with this size?

In case of a pfsense reinstall and config restore, I would disable the ram disk just to be safe.

stephenw10

I think that's big enough for pfBlocker to update.
I'm not sure if a RAM disk that size is sustainable on the 3100. But it could be the years of battling kernel memory limits we had before we switched to tmpfs making me paranoid!

Before that ~120MB was the most you could set. That's what I use on most installs:

[22.05-RELEASE][admin@fw1.stevew.lan]/root: df -h
Filesystem                      Size    Used   Avail Capacity  Mounted on
/dev/diskid/DISK-9E18E959s2a    7.0G    2.3G    4.1G    36%    /
devfs                           1.0K    1.0K      0B   100%    /dev
/dev/diskid/DISK-9E18E959s1      34M    2.0M     32M     6%    /boot/u-boot
tmpfs                            80M    2.2M     78M     3%    /tmp
tmpfs                           120M     34M     86M    28%    /var
devfs                           1.0K    1.0K      0B   100%    /var/dhcpd/dev

Steve

mcury

@stephenw10 I'll be testing and in case of problems I'll post here to help others that try this same approach.

Thanks stephenw10

I was getting around 2TB a year of writes before making this change.. Almost 5 years using device already :)

[22.05-RELEASE][root@pfsense.home.arpa]/root: iostat -x
                        extended device statistics  
device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b  
flash/sp       0       0      0.0      0.0     8     0     0     8    0   0 
mmcsd0         0       0      1.5      4.0     1    10     0     6    0   0 
mmcsd0bo       0       0      0.0      0.0     0     0     0     0    0   0 
mmcsd0bo       0       0      0.0      0.0     0     0     0     0    0   0 
[22.05-RELEASE][root@pfsense.home.arpa]/root: uptime 
10:25AM  up 18:38, 1 user, load averages: 0.29, 0.28, 0.25

FSC830

I am a bit confused about the RAM disk limits.
At my SG-3100 the limit is:

Maximum total size of all RAM disks cannot exceed available memory: 269.75 MiB

To which packages/options/values/... is the limit referring/calculating?

I do not use RAM disks any longer since my issues, just to still curiosity .

Regards

stephenw10

You are probably running an older version before we switched to tmpfs?

FSC830

Following your link ends in #12145.

For various reasons we may want to switch the RAM disks from ufs on md devices to tmpfs .

What is a md device, the internal MMC?
Is there a diffence when using a SSD instead (ada0)?

Regards

Edit: Dashboard widget shows tmpfs!?

stephenw10

md there is memory disk: https://www.freebsd.org/cgi/man.cgi?query=md

There's no difference to the ram disks between booting from eMMC or SSD. RAM disks are more common on eMMC installs where drive ware is more of an issue.

If /var/run is tmpfs you must be running 22.01 or newer. It's possible the limit detection was not updated for 22.01 and you are still seeing it rather than in 22.05?

Steve

FSC830

I updated from 22.01 to 22.05 at June, 28th at eMMC and did a complete new install from scratch to SSD at July 1st, 2022.
Afterwards I restored from ACB the settings, no idea, if the limit is also an outcome of the restore.

Regards

stephenw10

Are you actually using most of the reset of the RAM then? Perhaps you really only have 270MB available.

FSC830

The command top shows:

CPU: 43.8% user,  0.0% nice, 50.0% system,  0.0% interrupt,  6.3% idle
Mem: 88M Active, 464M Inact, 168M Wired, 84M Buf, 1297M Free

This matches the total of 2GB.
Are there any other commands available, "free" or "/proc/meminfo" seems to be not included in OS?

Regards

stephenw10

Try running: sysctl -a | grep kmem

FSC830

@stephenw10 said in SG-3100 ram disk question:

sysctl -a | grep kmem

The outcome is:

[22.05-RELEASE][admin@xxx]/root: sysctl -a | grep kmem
vm.uma_kmem_total: 46063616
vm.uma_kmem_limit: 328835072
vm.kmem_map_free: 282771456
vm.kmem_map_size: 46063616
vm.kmem_size_scale: 3
vm.kmem_size_max: 328833434
vm.kmem_size_min: 12582912
vm.kmem_zmax: 65536
vm.kmem_size: 328835072

Regards

stephenw10

Hmm, well there's the ~270MB. I don't expect that to apply with tmpfs though. A 650MB drive would be impossible to set it if was. I wonder if that value was set before the gui validity check was added and it's simply never used more than the available kmem.

I have to say I wouldn't recommend setting anything too large in the 3100 anyway.

stephenw10

Yeah, it still checks against available kernel memory:
https://github.com/pfsense/pfsense/blob/master/src/usr/local/www/system_advanced_misc.php#L356

That could probably be removed with tmpfs there but it's only a significant restriction on arm32 and limiting ram disk size there is not terrible anyway.

https://redmine.pfsense.org/issues/13508

Steve