Pfsense 2.4 ZFS File System

pfBasic

Yes if the spare was ada3 or anything that is on the SATA bus but your spare shows up as 'da4' in your earlier post, you have to adjust your commands for da4. Also the spare can not be just 'da4', it has to be the freebsd-zfs partition on it which is 'da4p2' after partioning. If you read the Sun ZFS documentation you probably thought that the spare would be there as a whole disk and the system would automatically sync the partitions on it as well, this is not the case on FreeBSD.

OK thanks I'll give that a shot, I really appreciate your help! I did read Suns documentation and eventually figured that out about the partitions (and some other differences), that's not the way it's setup anymore.

And ignore the earlier post, that's my actual pfsense install. I'm troubleshooting all of this on a VM before I do anything with my actual system. The real box doesn't use a hot spare like this.
On the VM all of the drives are adaX.

Paddy

@kpa:

@Paddy:

When using a single drive how do you tell ZFS to keep 2 copies?

You have to do this right after the pool creation before any datasets are created or files are written on it:
zfs set copies=2 zpool
This property is a dataset property but gets inherited by any child datasets so it applies to them as well.

Thanks for the answer.

Setting this at the command line - is it permanent or does in need to be put into a file?

kpa

@Paddy:

@kpa:
@Paddy:

When using a single drive how do you tell ZFS to keep 2 copies?

You have to do this right after the pool creation before any datasets are created or files are written on it:
zfs set copies=2 zpool
This property is a dataset property but gets inherited by any child datasets so it applies to them as well.
Thanks for the answer.

Setting this at the command line - is it permanent or does in need to be put into a file?

Zfs properties are stored in the pool metadata, they are permanent.

Paddy

Thanks again.

Do you believe there is any benefit in setting copies to 2? I've been reading a bit about it and from what I've read it has an impact on speed.
Also when the page I was reading tested its ability to stop corruption and while it reduced the amount of corruption it didn't eliminate it. So I't left me under the impression that it would have some merrit where important data of photos are stored, but not so much on a firewall.

Would I be correct?

kpa

Any kind of reduncancy has at least a small effect on write speeds, it's unavoidable because the data has to be duplicated somehow be it a straight second copy or some kind of parity system you have on raid-z. Two copies is not a bad idea on a single disk if you can't use two disk mirror for some reason, it can save your bacon because disks don't usually blow up completely just like that but start to slowly develop a bad sector here and bad sector there and it's very unlikely that with two copies of the same data you lose both copies at the same time.

pfBasic

Well I tried modifying the labels on all of the partitions to match their ada#, and reinstalled the bootcode to each p1. Nothing changed though, still same boot error.


gpart modify -l gptboot0 -i 1 ada0
gpart modify -l zfs0 -i 2 ada0
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0

#and so on for the rest of the drives

Any more ideas? It seems like this should be doable?

pfBasic

Finally figured it out. Detach the bad drive from the pool and the hot spare becomes a permanent part of the pool.

I detached the bad disk after resilvering was complete, the hot spare became part of the pool and reboots as if nothing changed. I kept removing two more disks that were part of the original pool and it still boots great with the hot spare and one disk from the original pool.

zpool detach poolname baddiskname

https://blogs.oracle.com/eschrock/entry/zfs_hot_spares

If you want a hot spare replacement to become permanent, you can zpool detach the original device, at which point the spare will be removed from the hot spare list of any active pools.

So at the end of the day it looks like it could potentially be useful to replace a boot disk remotely so long as you SSH in and offline the bad disk. Or if you wrote some sort of script that would offline the bad disk after resilvering was complete. But that's beyond me.

pfBasic

I typed up a basic ZFS How To here if anyone's interested.

https://forum.pfsense.org/index.php?topic=126597.0