Changing to RAM Disk - Failure

[jrey]

1st Don't panic --- a few bad words, but don't panic.

So I decided I would change the 2100 (23.09.1) to use ram disk

Step 1 - try it on my test box (2.7.2) -- all good here, came back up everything working

Step 2 - on the 2100 - sizes are default - tmp 40 (using 280 KiB) /var 60 (using 12.69 MiB) - should be fine. (should be no issue given the memory available on the device)

Step 3 - Save, system reboots, but doesn't come back.

Step 4 - this is where the initial bad words start. Can't find the laptop used for console. Now 30 minutes in still nothing on the browser.

Step 5 found it - hook that up. Get connected, notice it is sitting at the menu displayed after this message
*** Welcome to Netgate pfSense Plus 23.09.1-RELEASE (arm64)
But there are a some errors just above that. mostly to do with interfaces. (that explains why there is no way to connect) but the first message I can read at the top of these screen is

Fatal Error: Uncaught Error: Call to undefined function pfSense_interface_listget() in /etc/inc/interfaces.inc

Step 6 there must be more I can't see above what is showing on the screen. so option 5 reboot.. Oh yeah there are more messages going by, piles of it. No chance of trying to read most of that. but I end up at the same place, the menu again. config must be corrupt is my first thought.

Step 7 back over to the test box.. diff the before and after configs, the only change of interest is this line
<use_mfs_tmpvar></use_mfs_tmpvar>

Step 8 back to the 2100 console, confirm that line is in the current config, yes. file looks fine in all other aspects( doesn't seem corrupt)
cp the config.xml to config.xml.bak

Step 9 edit the config.xml, referenced line, delete it and save.
(yes I could have restored the config from a backup, or even used the handy ZFS snap, but this seemed faster since I was right at the console already.)

Step 10 back to the menu and option 5 reboot. Watch again on console, a lot less verbiage scrolling by and no error messages. System comes right up back at the menu.
Interfaces up, webpage loads again. Everything is still in tact for the mentioned /var (logs) etc..

Step 11 Apparently, Nothing to see here. Move along.

Not sure I want to "try it again" this year.

Although not certain, most of the messages going by seemed to have a lot to do with not being able to access, create or write files. (Guessing on the Ram disk)

Question is WTH happened ? (what should I look for next year if I try again)

[SteveITS]

@jrey There should be lines with a size:
<use_mfs_tmp_size>1024</use_mfs_tmp_size>
<use_mfs_var_size>2048</use_mfs_var_size>

...at least, if a size is set. Not sure what happens if you leave it blank but the 40/60 is awfully small, as noted in the docs. I would set it at something like 512/1024 for 4 GB. It's not preallocated anymore.

Note one can get in to trouble at just about any size, depending on usage...for instance I tried to test something with the pfBlocker UT1 adult list once for someone and it turns out that list takes way more than 1 GB to download and extract.

[jrey]

@SteveITS

Thanks -- I can double check that. "next time" Yeah - silly me I was just reading the docs, before you replied.

Quick follow-up the space usage is right about where it will stay - so I was thinking when it says this

the 12MiB being used is < 25% of the 60MiB so that should work ?
Maybe the current usage displayed is what is misleading me on that.

on the 2100 Memory usage is usually around 15% of the 3388MiB the dashboard reports, so I can allocate more

Interesting to note, I just tried this on the test system
The is before (current usage) 6.35

and like this

So configured the ram disk again (even though 40/60 worked here as well with no issue) This time I pick 512 / 1024

and after
current usage: 17.32

and

It's clear now that the Current Usage going in is not reflective of everything going over, and also seems obvious where the difference is from before and after..

taking that original misleading(to me) current usage value and looking at the 2100

Clearly the 12.68 (shown at the very top going in Isn't anything like what will actually be moved)

but even at that just /var/db and /var/log 13 + 16 + 9 + 11 (I get it overhead) what else is moving that perhaps isn't shown in the before total?

the entire du -h of /var on the test system is 501M and the parts that moved with no issues when it was set at 40/60 are 17M and now 2nd with 512/1024 same 17M showing isn't even 2% of the 1024 allocation

the entire du -h of /var on the 2100 is 623M total

[SteveITS]

@jrey I’ve never really tried to dig into it other than packages can use it, and logs of course. I just set it somewhere where it’s unlikely to be near full yet unlikely to cause a RAM shortage if ever full…

[jrey]

@SteveITS

Fair enough.

Just seems odd that it doesn't report the "current usage" from before and after as even near the same value.

Thanks, appreciate the feedback

[stephenw10]

I always start at double the minimum size for those values when using RAM disks and that's good enough for most setups until you start adding packages. So 60+120MB.

[jrey]

@stephenw10 @SteveITS

Thanks both.

I think the curiosity of what gets moved is satisfied. The empty / small directories in the following, have not been considered.

Consider the following from the dashboard where
/var is reported as 53M on tmpfs
/var/cache/pkg is reported as 161M on zfs
/var/db/pkg is reported as 5.2M on ifs

then consider this.

[2.7.2-RELEASE][bob]/var: du -sh /var/*
  0B    /var/at
161M    /var/cache  (on zfs Dashboard /var/cache/pkg 161)
  0B    /var/crash
  0B    /var/cron
 41M    /var/db     (on tmpfs)
8.0K    /var/dhcpd
  0B    /var/empty
 33K    /var/etc
8.3M    /var/log    (on tmpfs)
 94K    /var/run
  0B    /var/spool
  0B    /var/tmp
326M    /var/unbound  (on zfs based on size of var located on tmpfs but not accounted for on dashboard)

total on tmpfs 41 + 8.3 = 49.3 (daahboard shows 53 on tmpfs) - not concerned about this, could just be the way the tmpfs does "It's not preallocated anymore." I don't see any hidden files/directories Also easy enough to test.. I could just drop a large file each and see where it is counted.
curious is "It's not preallocated anymore." a one way street? or if a bunch of stuff gets added and removed does the tmpfs allocation shrink. of course also easy enough to test.

The dashboard doesn't account for the 326M under /var/unbound. This should likely be another of the entries similar to /var/cache/pkg and /var/db/pkg both of which remain on zfs.

Not the end of the world, just things don't add up and therefore is somewhat misleading.

also values before and after are zfs compression vs. tmpfs not