Automount ZFS volumes/datasets

senseivita · Sep 30, 2022, 8:09 AM

I have a forward/caching proxy system, like the main firewall it's got a tiny main disk and logs heavily but it rotates very quick so space isn't needed. Despite these systems being fresh installs, I went back to UFS because some odd issues on ZFS that disappear on the former.

The caching disk on the proxy is ZFS-formatted.
login-to-view

Standard locations for logs and Squid symlinked to this disk, it works great except that this time around I messed up something on my notes about this and it seems I omitted (and deleted mention of) the step to mount the disk on boot so I need to manually mount it or the symlinks revert to their literal location and it starts filling up real quick.

~~Any advise? :)~~ (wait--not yet)

Thinking it might be easier to just reformat the disk as UFS to mount it via fstab just now I got a hint of a memory about it being mounted not with zfs mount setName but something around the device's path; "/dev/da1" in this case, which won't appear listed on gpart, so I think I might be onto something. So far though, I haven't found in the FreeBSD's docs (specifically "The Handbook") about ZFS disk setup where the main disk isn't as well ZFS plus there's pfSense's own customizations to be considered after that.

A bit more info:
login-to-view

Any advise? :)

mer · Sep 30, 2022, 9:02 AM

@skilledinept have you looked at the zpool import command? That is typically how you make a system aware of a zpool and the datasets created on it

bingo600 · Sep 30, 2022, 9:39 AM

@skilledinept
I did some fooling around on 2.5.2 w. ZFS and a mount
https://forum.netgate.com/topic/165335/fun-with-zfs-snapshots-and-rollback

/Bingo

senseivita · Oct 1, 2022, 2:32 AM

Hey, thanks for answering,

@mer I had seen that but I figured it was for pools created on other systems, such as it would be in a reinstall.

Reading about it again I though, I rationalized, maybe pfSense just formatted some random disk then it went all "go find your mother, kid" on it, not actually claiming it. That made sense in my head somehow—not proud.

Then @bingo600 dials up the hope with a link, first thing I read, third line in a huge dark box use '-f' to import anyway. "This is it." I rebooted the system to revert to normal conditions (not mounted automatically) and then raced against time, main disk is filling up.

[2.6.0-RELEASE][root@xxx]/root: df
Filesystem                                      1K-blocks    Used    Avail Capacity  Mounted on
/dev/gptid/a5747caa-51b3-11ec-9d4f-005056be0002  19093052 2231788 15333820    13%    /
devfs                                                   1       1        0   100%    /dev
tmpfs                                                4096      60     4036     1%    /var/run
[2.6.0-RELEASE][root@xxx]/root: zpool-import -D
zpool-import: Command not found.

I was/am being guided by a temporary TrueNAS server because pfSense has no man pages. (Why?!)

[2.6.0-RELEASE][root@xxx]/root: zpool import -D
[2.6.0-RELEASE][root@xxx]/root: zpool import -a
[2.6.0-RELEASE][root@xxx]/root: zpool import -a /bottombitch
too many arguments
usage:
	import [-d dir] [-D]
	import [-o mntopts] [-o property=value] ...
	    [-d dir | -c cachefile] [-D] [-f] [-m] [-N] [-R root] [-F [-n]] -a
	import [-o mntopts] [-o property=value] ...
	    [-d dir | -c cachefile] [-D] [-f] [-m] [-N] [-R root] [-F [-n]] [-t]
	    [--rewind-to-checkpoint] <pool | id> [newpool]
[2.6.0-RELEASE][root@xxx]/root: zpool import bottombitch

The name of the pool, by the way, is how we say "caching disk" in my native language. It's pronounced with guttural sounds, hissing noises and a sigh. Any resemblance — if any, I don't know! — to English words is just a coincidence. You believe that, right?

cannot import 'bottombitch': a pool with that name is already created/imported,
and no additional pools with that name were found
[2.6.0-RELEASE][root@xxx]/root: mount
/dev/gptid/a5747caa-51b3-11ec-9d4f-005056be0002 on / (ufs, local, noatime, journaled soft-updates)
devfs on /dev (devfs, local)
tmpfs on /var/run (tmpfs, local)
devfs on /var/dhcpd/dev (devfs, local)
[2.6.0-RELEASE][root@xxx]/root: zfs list
NAME          USED  AVAIL  REFER  MOUNTPOINT
bottombitch  1.04G  60.5G  1.04G  /bottombitch
[2.6.0-RELEASE][root@xxx]/root: zpool import -d /bottombitch
[2.6.0-RELEASE][root@xxx]/root: mount
/dev/gptid/a5747caa-51b3-11ec-9d4f-005056be0002 on / (ufs, local, noatime, journaled soft-updates)
devfs on /dev (devfs, local)
tmpfs on /var/run (tmpfs, local)
devfs on /var/dhcpd/dev (devfs, local)
[2.6.0-RELEASE][root@xxx]/root: zpool import -d /dev/da1
cannot open '/dev/da1/': Not a directory

About to quit the zpool-import manpage back on TrueNAS, just when I pressed the last key I caught a glimpse of the related commands at the end, one of them was:

[2.6.0-RELEASE][root@xxx]/root: zpool export bottombitch
[2.6.0-RELEASE][root@xxx]/root: zpool import bottombitch
[2.6.0-RELEASE][root@xxx]/root: mount
/dev/gptid/a5747caa-51b3-11ec-9d4f-005056be0002 on / (ufs, local, noatime, journaled soft-updates)
devfs on /dev (devfs, local)
tmpfs on /var/run (tmpfs, local)
devfs on /var/dhcpd/dev (devfs, local)
bottombitch on /bottombitch (zfs, local, nfsv4acls)

It imported the volume and mounted it automatically, but on reboot it was unmounted again.

[2.6.0-RELEASE][root@xxx]/root: reboot
Connection to 10.4.0.2 closed by remote host.
Connection to 10.4.0.2 closed.
zx9:~ v$ ssh wf df
Filesystem                                      1K-blocks    Used    Avail Capacity  Mounted on
/dev/gptid/a5747caa-51b3-11ec-9d4f-005056be0002  19093052 2230600 15335008    13%    /
devfs                                                   1       1        0   100%    /dev
tmpfs                                                4096     112     3984     3%    /var/run
devfs                                                   1       1        0   100%    /var/dhcpd/dev
zx9:~ v$

Logged in…

[2.6.0-RELEASE][root@xxx]/root: mount
/dev/gptid/a5747caa-51b3-11ec-9d4f-005056be0002 on / (ufs, local, noatime, journaled soft-updates)
devfs on /dev (devfs, local)
tmpfs on /var/run (tmpfs, local)
devfs on /var/dhcpd/dev (devfs, local)
[2.6.0-RELEASE][root@xxx]/root: zpool list
NAME          SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
bottombitch  63.5G  1.04G  62.5G        -         -     0%     1%  1.00x  ONLINE  -
[2.6.0-RELEASE][root@xxx]/root: zfs list
NAME          USED  AVAIL  REFER  MOUNTPOINT
bottombitch  1.04G  60.5G  1.04G  /bottombitch

It's there but unmounted. I tried it again rebooting from the menu, maybe exiting the shell was required to save changes. Just casual aggressively uninformed speculation.

[2.6.0-RELEASE][root@xxx]/root: zpool export bottombitch && zpool import bottombitch && exit
exit
pfSense - Netgate Device ID: xxxxxxxxxxxxxxxxxxxx

*** Welcome to pfSense 2.6.0-RELEASE (amd64) on wf ***

 WAN (wan)       -> vmx0       -> v4/DHCP4: 10.4.0.2/24
                                  v6: 2001:470:xxxx:xxx::2/64

 0) Logout (SSH only)                  9) pfTop
 1) Assign Interfaces                 10) Filter Logs
 2) Set interface(s) IP address       11) Restart webConfigurator
 3) Reset webConfigurator password    12) PHP shell + pfSense tools
 4) Reset to factory defaults         13) Update from console
 5) Reboot system                     14) Disable Secure Shell (sshd)
 6) Halt system                       15) Restore recent configuration
 7) Ping host                         16) Restart PHP-FPM
 8) Shell

Enter an option: 5


pfSense will reboot. This may take a few minutes, depending on your hardware.
Do you want to proceed?

    Y/y: Reboot normally
    R/r: Reroot (Stop processes, remount disks, re-run startup sequence)
    S: Reboot into Single User Mode (requires console access!)
Enter an option: y

pfSense is rebooting now.
 Stopping package squid3...done.
 Stopping /usr/local/etc/rc.d/haproxy.sh...done.
 Stopping /usr/local/etc/rc.d/lighttpd_ls.sh...done.
 Stopping /usr/local/etc/rc.d/sqp_monitor.sh...done.
Connection to 10.4.0.2 closed by remote host.
Connection to 10.4.0.2 closed.
zx9:~ v$

No…

[2.6.0-RELEASE][root@xxx]/root: mount
/dev/gptid/a5747caa-51b3-11ec-9d4f-005056be0002 on / (ufs, local, noatime, journaled soft-updates)
devfs on /dev (devfs, local)
tmpfs on /var/run (tmpfs, local)
devfs on /var/dhcpd/dev (devfs, local)

But here comes more speculation a little more thought out this time,

Since the system isn't booting the main disk as ZFS, could it be that it's not loading some service that mounts the ZFS pools? Or maybe the pool doesn't have a property, like "compression" for instace, that's required to automount it on boot? Not that I would know how to fix it, I think, but it's something to get guided by. :)

Thanks again to both for responding.

mer · Oct 1, 2022, 8:13 AM

@skilledinept said in Automount ZFS volumes/datasets:

could it be that it's not loading some service that mounts the ZFS pools?

That is a good possibility. On a normal FreeBSD system /etc/rc.conf has "zfs_enable=YES" to enable/start the zfs service.
I don't have a pfSense with ZFS in front of me, but in the Web GUI, look under services and see if there is a ZFS somewhere to enable.