Will 2.4, (or a later), support ZFS boot environments?

Arwen

Specifically, ZFS allows alternate boot environments using snapshots and file system clones.
These are useful when performing updates. If the update goes south, then you can roll-back
to before the change with a simple reboot. These snapshots and clones take up as much
space as is changed. Thus, little changes, little extra space used.

In addition to ABEs, (Alternate Boot Environments), a feature could / should be added such
that during activation of a new ABE, (either actual code update, or simply user initiated ABE),
if the user does not login within 5 minutes, (or so), it will automatically reboot back to the
prior BE. Thus, if the management network gets hosed, you don't have to get on the console.

So, the first allows less impacting updates, (either actual code, or user desiring to make lots
of changes). The second, fault tolerance.

I checked the roadmaps for 2.4, 2.4.1, 2.5 and future, nothing seems to exist on this subject.

Anyone want to comment?
Should it be added as a feature request?

beremonavabi

According to:

https://doc.pfsense.org/index.php/2.4_New_Features_and_Changes#Operating_System_.2F_Architecture_changes

2.4 will support ZFS.

Arwen

Yes, I was aware of that.

What I was really asking about, is ZFS alternate boot environments.

Basically, you can snapshot the existing boot environment, (aka root file system), clone
it, upgrade that clone, add a grub entry and boot off the upgraded clone. If the upgrade
works fine, and you want to recover some disk space, you can delete the old ZFS Dataset,
it's snapshot and grub entry. Or keep it for a few weeks / months until you upgrade again.

This also useful if you are making lots of configuration changes that you are not sure are
perfectly well matched. So you create a new boot environment, boot off it, and make your
changes. If changes did not work out, a simple reboot restores everything to what it was
before.

If I understand pfSense correctly, in the past it supported this by using a second OS partition.
Upgrades would be to the un-used OS partition. When the upgrade was ready, reboot to
make it active. This is a bit clumsier, as you have to double your OS space. ZFS snapshots
and clones only take up as much space as has changed. Thus, you can probably keep several
ZFS ABEs around. Just delete the oldest one(s) when space gets tight.

pfBasic

It looks like beadm is what you're looking for?

https://www.freebsd.org/cgi/man.cgi?beadm

https://forums.freebsd.org/threads/31662/

It is not currently included in pfSense, and who knows if it will ever be. I'm sure you can side-load it if you really want to.

kpa

That's only the OS level infrastructure which is already almost there if the mentioned beadm tool is added to the system but see below *). What would be needed on top of those is a WebGUI component to visualize the boot environments and means to manage them.

*) What is missing from the OS level support is a boot loader that can display the boot environments and allows choosing between them at boot time. This requires a custom boot loader such as GRUB or similar because the FreeBSD loader can't do that for technical reasons. Without such loader it's very difficult to make full use of the boot environments in troubleshooting situations.

Arwen

Yes pfBasic, something like BEADM.

But, as kpa pointed out, GUI support would be nice. And of course the ability to choose the boot
environment from the boot loader, like with Grub.

I was not aware that the FreeBSD boot loader did not support multiple boot entries.

So, is this something that could or should be added in the future?

It's just so useful in the other places I've used it, (Solaris 10 & 11; FreeNAS; Linux with ZFS root).

kpa

I have to correct myself on the boot loader part of my post. It is in fact possible to select the boot environments using the standard FreeBSD loader (the /boot/zfsloader executable to be precise). I'm not sure when the support for boot environments was added to but at least I can do so on my FreeBSD 11.0 system. PfSense 2.4 will be based on FreeBSD 11 so that part will be covered.

Edit: The support for choosing ZFS Boot Enviroments in the loader is in 10.3 as well:

https://svnweb.freebsd.org/base?view=revision&revision=293802

There is however one major gotcha with the way the FreeBSD ZFS loader works. The bootcode pre-selects the root filesystem that is used to load the /boot/zfsloader executable by using the bootfs property of the ZFS pool used. This means that if you mess up the /boot/zfsloader file on the currently active boot enviroment somehow you won't be able to boot anymore. To remedy the situation you have tell the bootcode to use another boot environment from the pool to use as the default root filesystem for loading /boot/zfsloader. This is if I remember correctly done by pressing a key (space of shift I'm not sure) at the time the boot code is running and entering something like 'zpool/ROOT/alternate:/boot/zfsloader' to boot using the alternate boot environment.

Arwen

Thanks for the research. I thought FreeBSD should have supported boot environments from the boot menu.

Yes, even with perfect redundancy, like Mirroring and alternate boot environments certain files are critical to the OS.

One of the things I dis-like about Grub, (and all boot loaders), is that they are not fault tolerant. There is no builtin
way to verify the information that the boot loader uses, (partition table, embedded code, "/boot" code & configuration).
It would be nice to have a boot loader keep checksums of all that and run a verify each boot. Plus, have a user option
to run it from the OS, whence you are booted. (So you can check for faults before you reboot.)

Anyway, sounds like we will have some of the base work support: FreeBSD 11 kernel and it's boot loader.

Would there be enough interest to put in a feature request?

rainer_d

Boot-environment support at the boot-loader was available with FreeBSD 10.3.

https://www.freebsd.org/releases/10.3R/relnotes.html#boot