PC Engines apu2 experiences

[logan5247]

@vollans Thanks for this write up! I am installed on UFS but may go back and switch to ZFS now. I'm a Linux guy, so ZFS has always been out of my wheelhouse.

When I do the initial setup and pfSense is working, do I:

1. Perform a snapshot then and leave it around for years and years? Is this safe? I'm thinking like a VM snapshot where you don't want to have a snapshot hang around for long periods of time.
2. Only perform snapshots before an upgrade, do the upgrade, then remove the snapshot after it's working?

Thanks again!

[VAMike]

@logan5247 said in PC Engines apu2 experiences:

@vollans Thanks for this write up! I am installed on UFS but may go back and switch to ZFS now. I'm a Linux guy, so ZFS has always been out of my wheelhouse.

This is an issue mainly because UFS in pfsense performs recovery so incredibly badly. I don't fully understand why something as heavy as ZFS seems to be the only solution.

[Vollans]

@logan5247 I don’t see any inherent dangers in leaving the snapshot hanging around, unless you are really tight for space. Snapshots only record changed files, so it’s not a huge thing. Personally, I use it for a couple of reasons.

1. Fully installed with patches base OS before any fiddling - that way if you screw up you can roll back and undo your “magic” that was more Weasley than Granger.

2. Snapshot once fully tweaked and working, so you’ve got a known working system to roll back to

3. Just before a major upgrade

Here’s my snapshot catalogue:

NAME                              USED  AVAIL  REFER  MOUNTPOINT
zroot                            2.90G  9.21G    88K  /zroot
zroot@210219                         0      -    88K  -
zroot@2-4-5p1-base                   0      -    88K  -
zroot@2-4-5-p1                       0      -    88K  -
zroot/ROOT                       2.14G  9.21G    88K  none
zroot/ROOT@210219                    0      -    88K  -
zroot/ROOT@2-4-5p1-base              0      -    88K  -
zroot/ROOT@2-4-5-p1                  0      -    88K  -
zroot/ROOT/default               2.14G  9.21G  1.84G  /
zroot/ROOT/default@210219         146M      -  1.14G  -
zroot/ROOT/default@2-4-5p1-base  36.3M      -  1.43G  -
zroot/ROOT/default@2-4-5-p1      36.5M      -  1.43G  -
zroot/tmp                         512K  9.21G   512K  /tmp
zroot/var                         776M  9.21G   396M  /var
zroot/var@210219                  183M      -   527M  -
zroot/var@2-4-5p1-base           52.1M      -   409M  -
zroot/var@2-4-5-p1               61.5M      -   434M  -

The space used as it goes along is tiny. The upgrade to 2.5 that I ended up rolling back from only used about 900MB IIRC.

[kevindd992002]

Without doing manual snapshots, is there an advantage of using ZFS over the old UFS? I am on ZFS on a single SSD and I forgot what its advantage is when I posted here a few years ago.

[Vollans]

@kevindd992002 Better resilience if you have a crash. UFS has a horrid habit of collapsing in an unrecoverable heap, ZFS is far more likely to recover gracefully.

[Qinn]

@kevindd992002 said in PC Engines apu2 experiences:

Without doing manual snapshots, is there an advantage of using ZFS over the old UFS? I am on ZFS on a single SSD and I forgot what its advantage is when I posted here a few years ago.

....of course RAID with ZFS gives more redundancy, best is more disks using RAID. As the problem with a single disk and "copies" is the same as creating an mdadm raid-1 using two partitions of the same disk: you have data redundancy, but not disk redundancy, as disk failure will cause the loss of both data sets.

Comparing UFS with ZFS, well ZFS, like btrfs, is copy-on-write, so power surges are never a problem and ZFS requires a system with ECC memory (APU2 has this), otherwise you're still not 100% safeguarded against bit errors.

[logan5247]

@vollans Sorry to keep asking questions.

1. ) If I snapshot zroot do I need to snapshot zroot/ROOT and zroot/ROOT/default? Does zroot not include everything else?

2. Let's say I did a snapshot, made a change, and successfully rolled back:

zfs rollback zroot/var@20210308
zfs rollback zroot/ROOT/default@20210308
zfs rollback zroot/ROOT@20210308
zfs rollback zroot@20210308
shutdown -r now

And now my zfs list looks like this (after the rollback):

NAME                          USED  AVAIL  REFER  MOUNTPOINT
zroot                         674M  12.4G    96K  /zroot
zroot@20210308                   0      -    96K  -
zroot/ROOT                    665M  12.4G    96K  none
zroot/ROOT@20210308              0      -    96K  -
zroot/ROOT/default            665M  12.4G   665M  /
zroot/ROOT/default@20210308   388K      -   665M  -
zroot/tmp                     144K  12.4G   144K  /tmp
zroot/var                    7.02M  12.4G  6.62M  /var
zroot/var@20210308            400K      -  6.62M  -

How do I know what set of filesystems I'm running on? Is there something like an "active" marker in zfs list?

[VAMike]

@logan5247 this really should get its own zfs thread, it has nothing to do with the apu2

[Vollans]

@vamike I agree, but quickly in summary, you're always running the one without the @ sign - that's the current live version. You can see that the size of the "backup" of zroot is nothing. The size of zroot/ROOT/default's backup is bigger. zroot doesn't include the other, effectively, "partitions".

[Qinn]

@dem said in PC Engines apu2 experiences:

@vollans I did a quick test in a virtual machine to figure out what the commands would be. This appears to work:

On a running 2.4.5-p1 system:

zpool checkpoint zroot

Booted from the 2.5.0 installer and in the Rescue Shell:

zpool import -f -N --rewind-to-checkpoint zroot
zpool export zroot
poweroff

I would like to know how you booted from the 2.5.0 installer using a virtual pfsense machine and got to the Rescue Shell?

[dem]

@qinn In VirtualBox I put the file pfSense-CE-2.5.0-RELEASE-amd64.iso in the virtual optical drive and booted to this screen, where I selected Rescue Shell:

[Qinn]

@dem Thanks for the quick reply, I understand that.
What I would like to know is how you get to the Virtual pfSense from here as the virtual pfsense machine is not running and access the checkpoint you made?

Btw I am using VM workstation!

[dem]

@qinn Sorry I wasn't clear: I put the installer image into the virtual optical drive of the same virtual pfSense instance that I checkpointed, so the installer has access to the same virtual disk and can locate the checkpointed zroot pool.

Edited to add: My goal was to simulate booting an apu2 from the memstick image in order to rewind a checkpoint, but I don't have a spare apu2 to test with. If I actually ran pfSense in a virtual machine I would use virtual machine snapshots before any upgrade.

[VAMike]

@qinn I never would have guessed this was somehow specific to the apu2

[dem]

@vamike It's not, but @vollans rolled back an upgrade of his apu2 using ZFS and that sparked interest in using ZFS to recover from a failed upgrade.

[VAMike]

@dem so he should start another thread so that people actually interested in apu2 experiences can find those without digging through unrelated zfs support questions

[Vollans]

@vamike I agreed it wasn't relevant here 2 days ago, and stopped responding.

[sikita]

Why pfsense 2.5 shows "AES-NI CPU Crypto: No" when in 2.4.x there was YES on APU2? Also on 2.5 there is in other line: Hardware crypto AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS

[bigsy]

@sikita said in PC Engines apu2 experiences:

Why pfsense 2.5 shows "AES-NI CPU Crypto: No" when in 2.4.x there was YES on APU2?

Anything to do with this problem? If so, it appears to be fixed in 2.5.1.

[sikita]

@bigsy Ok, thank you. Seems to be GUI bug and does not involve using HW crypto.