Which Netgate devices are ZFS capable?

renegade

Hello,

I have a SG-3100 and quite happy with it.
But I would like to use ZFS. Is the 3100 capable for ZFS? Which are/are not?

gabacho4

Trying to find the post that I read yesterday from @stephenw10 at Netgate, but the ARM devices are not ZFA capable. Ever. Will provide link if/when I find it.

gabacho4

Found it:

https://forum.netgate.com/topic/169969/netgate-3100-22-01-and-zfs/4?_=1645103576306

No ZFS for 32bit ARM devices.

stephenw10

Yes, it's specifically the arm32 devices which cannot support ZFS. So the 3100 and the SG-1000.

In addition the MBT-2220 and MBT-4220 cannot run ZFS at the same time as the additional graphics driver required for HDMI hotplug.

Steve

renegade

@stephenw10 so the 2100 would support ZFS as it is a 64 Bit CPU? Or is it the same as the 1100?

bigsy

@renegade The 2100 and 1100 are both 64-bit ARM and support ZFS.

ay

@stephenw10
If minnowboards were re-installed with the serial console image, could they use ZFS ?
-- Or does the 2.6 installer actively prevent selecting ZFS when running on a MBT-2220 or MBT-4220 ?

Our minnowboards are headless installs, overseas - I don't have any locally to test on.

I'd prefer ZFS' perceived power loss / crash stability, over someone being able to hot plug in an HDMI monitor.

stephenw10

Yes, you can run ZFS as long as you don't load the i915 driver. That gets added to /boot/loader.conf.local at install time. I've never tried using the serial installer but may well have the same code so you would still want to remove or comment the lines after installing.

Steve

Spacecase

@stephenw10,

I'm having different issues with the MBT-4220. I wasn't aware of the issue with the graphics driver, but I'm attempting to load a fresh image with a ZFS file system.

I'm currently struggling with v2.6.0, but it stalls at the point where it doesn't recognize the default boot partition. It leaves the system stuck at the <mountroot> shell and returns 'error 2: unknown file system.' I've made multiple attempts at resolving it, but have been unsuccessful.

Other problems while struggling for an acceptable configuration included persistent "Cam status" messages no matter what I did. I attempted to load a clean installation of a UFS file system and that failed as well. I even wiped clean the M.2 disk between attempts at changing BIOS settings hoping that might work. Ultimately, I loaded an older version I had on a memory stick (v2.4.5p1) and it loaded with nary a hiccup using a default BIOS configuration.

If anyone had any success with the Minnowboard, I'd like to know how you did it. That might include knowing how to remove the driver ahead of creating the file system during the loading of v2.6.0.

Respectfully...

stephenw10

@spacecase said in Which Netgate devices are ZFS capable?:

It leaves the system stuck at the <mountroot> shell and returns 'error 2: unknown file system.'

Yes, that is expected because it's unable to load the required kernel modules to access ZFS.
You need to prevent it loading the i915 driver and load the zfs modules instead.

Install ZFS as normal then at the end when it asks if you would like to make changes choose Yes.
At the shell prompt you need to edit the loader.conf.local file, you can use the built-in easy editor: ee /boot/loader.conf.local

I chose to comment out the lines which load the i915 driver and prevent loading the ZFS modules. So it should look like:

#i915kms_load=YES
drm.i915.enable_unsupported=1
kern.vt.fb.modes.VGA-1=d
kern.vt.fb.default_mode="1024x768"
#opensolaris_load=NO
#zfs_load=NO

Note that none of those lines now do anything so you could just remove the .local file completely.

Hit esc to exit and save then type exit to quit the shell and reboot. It will boot into ZFS but HDMI hotplug will not work.

Steve

Spacecase

@stephenw10 said in Which Netgate devices are ZFS capable?:

Install ZFS as normal then at the end when it asks if you would like to make changes choose Yes.
At the shell prompt you need to edit the loader.conf.local file, you can use the built-in easy editor: ee /boot/loader.conf.local

I never get to the end of the installation. Has anyone installed ZFS on a Minnowboard with the 2.6.0 image?

FYI: The BIOS might be one you haven't checked:

smbios.bios.version="MNW2MAX1.X64.0097.R01.1709211052"

Perhaps I should update the BIOS and try again.

stephenw10

I just did it to test for the above post on a 4220:

[2.6.0-RELEASE][admin@4220.stevew.lan]/root: kenv smbios.system.product
Minnowboard Turbot D0/D1 PLATFORM
[2.6.0-RELEASE][admin@4220.stevew.lan]/root: kenv smbios.bios.version
MNW2MAX1.X64.0101.R01.1908071815
[2.6.0-RELEASE][admin@4220.stevew.lan]/root: zpool status
  pool: pfSense
 state: ONLINE
  scan: none requested
config:

	NAME        STATE     READ WRITE CKSUM
	pfSense     ONLINE       0     0     0
	  ada0p4    ONLINE       0     0     0

errors: No known data errors

As long as the smbios.system.product value is recognised it will add those loader values.
It looks like you still have the original BIOS version which should be fine.

Exactly where does it fail if it doesn't complete the install?

Have you tried 2.5.2?

Steve

Spacecase

@stephenw10

I've only attempted 2.6.0. I do have an image of 2.5.2 and could try that.

Here are various screen shots:

stephenw10

Hmm, those CAM errors during the install look like a bad drive. Is that the original supplied SSD?

That first screenshot look like booting the SSD with the default loader variables after installing though, is that not the case?

Steve

Spacecase

@stephenw10
Yes, it's the original SSD and after many attempts to load 2.6.0, it cleanly loaded 2.4.5p1 (from a different memory stick). It updated to 2.6.0 over the 'net. No issues. Perhaps I'll try another USB stick and perform an integrity check before using it. If it matters, I've been using the USB-2 port based upon info in the online documents.

After experiencing the issues, I started using an M.2-to-USB adapter on a Windows 10 pc to wipe the disk to a clean state using the 'diskpart' CLI utility prior to further attempts at loading a ZFS image.

I do have a copy of 2.5.2 and could try loading it when I have more time.

Edit: Yes, I believe the first picture was toward the end of the load sequence.

stephenw10

My bad, looking closer at those errors they are on da0 which is the USB drive.
So try a different USB stick or try the other USB port. I usually use the USB3 port with a USB3 drive but YMMV!

I had a bad drive in the one I have which showed exactly like that except on ada0.

You shouldn't need to wipe the drive beyond the install. Occasionally if you've had something else installed on it there may be backup partition table present that causes problems where it can help.

Steve

Spacecase

@stephenw10 Thanks for all the help. Things worked much more smoothly with a healthy memory stick . I tried loading 2.6.0, but after editing the loader.conf file, the Minnowboard 'booted' into a BIOS shell. Ultimately, I wound up loading 2.5.2 and upgrading to 2.6.0. So far, so good.

stephenw10

It's the loader.conf.local file which has those values. The main loader.conf file should be left as set at install.

Spacecase

@stephenw10 Yes, "/boot/loader.conf.local" is the file I edited. It was 4 lines vs the 6 in the 2.6.0 file.

rcfa

@stephenw10 I think I'm affected by this bug, at least I can't get my MBT-4220 to boot anymore since attempting to upgrade to pfSense 2.6.0

The issue is, booting from a USB stick, and mounting the zfs partition on the regular boot drive, I can't find a /boot/loader.conf.local file, just a /boot/loader.conf

So how do I disable the console driver from getting loaded? By removing/renaming the /boot/kernel/i915kms.ko file?