Is ZFS possible on APU2C4

BrucexLing

I have been using UFS file system on my PCEngines APU2C4 for several years now, currently with 2.4.5-RELEASE-p1. I have just tried to do a fresh install to a ZFS system but without success. I am getting an Error – gpart: geom ‘ada0’: File exists, when picking file system - using the Tinycore serial install method. Could I please ask if anyone is successfully using ZFS on an APU2C4 and if they recall any such issues along the way?

I am still able to install to UFS which is what I am using right now.

KOM

@brucexling It's possible as I see others with an APU board running ZFS. If that ada0 partition/slice already exists then perhaps you need to manually delete it first.

viragomann

@brucexling
Yes, I did a fresh install on a blank disk using ZFS. Can't remember of any issue.

BrucexLing

This is the error message. It is not clear to me where to look for the file that it refers to:

If I open the Partition Editor at this stage of the installation I see the following.

To my limited understanding of freeBSD it seems that 3 slices are in use by the installing process with free space for a fourth slice to contain the ZFS slice/partition of the main system. I would be grateful for any suggestions to troubleshoot this further.

KOM

@brucexling Delete the entire ada0 partition.

BrucexLing

I am afraid I am not clear on what you are asking KOM. Before I run the installer from Tinycore, I do delete every partition present on the target disk. I even zeroed out all space on the target disk on one occasion, prior to installing. When booting freeBSD on the target disk for the first time I see the partition info in the screengrab above. The partition ada0s1 appears to be for EFI. The partition ada0s2a appears to be the main stage of the installer to which I have just booted. I’ve no idea what the ada0s3 partition is for. The partition manager does allow the deselection of ada0s1 and ada0s3 but any attempt to commit these changes results in the following:

Even without any changes to the partition selections, I get the above result when attempting to commit.

If I create a fourth partition in the Partition Editor and commit this, the installer goes on automatically to complete a UFS installation that appears to be almost viable. (fdisk reports: mounted root fs resource doesn’t match expectation (regexec returned 1)). A viable ZFS system continues to elude me.

KOM

@brucexling I don't have one of those units so I can't give you any further guidance. Others have gotten past this by getting a shell either via this installer (no idea if that's an option or not) or by booting some other OS and then using fdisk to wipe all partitions.

BrucexLing

Many thanks KOM. I hope one of the others you refer to passes by here some time. At least you have reassured me that it can be done.

BrucexLing

I have made some progress and I now have a working pfSense system working on a ZFS file system on my APU2C4. The only issue I seem to have now is am getting the warning/error:
“fdisk: mounted root fs resource doesn't match expectations (regexec returned 1)”
when doing a fdisk from the pfSense shell. Here is the output for fdisk and gpart that might shed light on what could be done to improve bsd’s expectations.

[2.4.5-RELEASE][admin@pfSense.localdomain]/root: fdisk
fdisk: mounted root fs resource doesn't match expectations (regexec returned 1)


[2.4.5-RELEASE][admin@pfSense.localdomain]/root: gpart list
Geom name: ada0
modified: false
state: OK
fwheads: 16
fwsectors: 63
last: 31277191
first: 40
entries: 128
scheme: GPT
Providers:
1. Name: ada0p1
   Mediasize: 524288 (512K)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 20480
   Mode: r0w0e0
   efimedia: HD(1,GPT,dc32bdf5-aa2a-11eb-b19d-000db941e06c,0x28,0x400)
   rawuuid: dc32bdf5-aa2a-11eb-b19d-000db941e06c
   rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f
   label: gptboot0
   length: 524288
   offset: 20480
   type: freebsd-boot
   index: 1
   end: 1063
   start: 40
2. Name: ada0p2
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 1048576
   Mode: r1w1e0
   efimedia: HD(2,GPT,dc4a08f6-aa2a-11eb-b19d-000db941e06c,0x800,0x400000)
   rawuuid: dc4a08f6-aa2a-11eb-b19d-000db941e06c
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: swap0
   length: 2147483648
   offset: 1048576
   type: freebsd-swap
   index: 2
   end: 4196351
   start: 2048
3. Name: ada0p3
   Mediasize: 13865320448 (13G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 2148532224
   Mode: r1w1e1
   efimedia: HD(3,GPT,dc5bdced-aa2a-11eb-b19d-000db941e06c,0x400800,0x19d3800)
   rawuuid: dc5bdced-aa2a-11eb-b19d-000db941e06c
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: zfs0
   length: 13865320448
   offset: 2148532224
   type: freebsd-zfs
   index: 3
   end: 31277055
   start: 4196352
Consumers:
1. Name: ada0
   Mediasize: 16013942784 (15G)
   Sectorsize: 512
   Mode: r2w2e3

The key to getting past the earlier “file exists” blockage was to do the first part of the install to a second usb flash drive rather than the hard disk. I could then boot from that second flash drive and do the second part of the install to the clean hard disk. In this way it sailed through the ZFS selection.

Do you think redoing the install onto a completely zeroed hard disk would resolve the current problem?

KOM

@brucexling No idea but I'm wondering why you're installing 2.4.5.

BrucexLing

I am under the impression that 2.5.1 has some issues with multi-Wan, and I need that capability. I did attempt a trial install of 2.5.0 when it first appeared but I had problems with the install. I think one problem was with ZFS selection, but it was a while ago so not real sure. When 2.5.2 lands and beds down a bit I’ll probably move to that.

KOM

@brucexling Makes sense.

bigsy

@brucexling said in Is ZFS possible on APU2C4:

using the Tinycore serial install method. Could I please ask if anyone is successfully using ZFS on an APU2C4 and if they recall any such issues along the way?

No problems installing to ZFS here multiple times on APU2E4.

Why the 'Tinycore' method and not straight from the pfSense AMD64 USB Memstick serial console image?

BrucexLing

Good call bigsy. The TinyCore method was what PCEngines suggested in the nanobsd days, and it still works with current Memstick serial console images for UFS but not ZFS unless you do what I finally did, at least that has been my experience. Etcher is a cleaner method and I see it is now preferred by PCEngines so I’ll use it from now on.

Either way the result remains that fdisk is reporting the mounted root fs mismatch issue. I have just had a chat over on the freeBSD forum and the upshot seems to be that I shouldn’t believe the warning that the obsoleted fdisk produces and rather put my trust in gpart. Looks like I need to update my practices.

viragomann

@brucexling said in Is ZFS possible on APU2C4:

Either way the result remains that fdisk is reporting the mounted root fs mismatch issue.

I get the same in my 2.4.5 installations and they are running for years.

JeGr

@brucexling I'd go with rufus. -> https://rufus.ie/
Can't see the reason why a simply "dd" frontend to write to freakin' USB sticks should take a whole electron app thingy with hundreds of MB to install just to have a fancy UI ;)

Besides that - just use a memstick-serial image for APUs and use Auto (ZFS) to install. Nothing special with that, we're using that with almost any device now. Only thing with PCengines thingies was with their BIOS/CoreBIOS and early problems with booting onboard SSDs etc. and if you've tried installing ZFS on USB sticks or SD/CF cards but that's waaaay back. Can't remember seeing boxes without a halfway decent SSD installed, so nothing special to the installer necessary :)

Qinn

ZFS has been my choice since it is available on pfSense. Comparing UFS with ZFS, well ZFS, like btrfs, is copy-on-write, so power surges are never a problem and ZFS requires a system with ECC memory (APU2 has this), otherwise you're still not 100% safeguarded against bit errors.

JeGr

@qinn said in Is ZFS possible on APU2C4:

and ZFS requires a system with ECC memory (APU2 has this),

No it doesn't. It is recommended - and always nice to have - but it isn't required at all.

Qinn

@jegr said in Is ZFS possible on APU2C4:

@qinn said in Is ZFS possible on APU2C4:

and ZFS requires a system with ECC memory (APU2 has this),

No it doesn't. It is recommended - and always nice to have - but it isn't required at all.

Not claiming to be an expert and maybe I formulated it badly, but I said; "that it is required if you want to be 100% safeguarded against bit errors".

@everyone, you could read https://hectronic.se/ecc-memory-solves-bit-errors/

Cheers Qinn

Gil

I ran a number of APU2 devices on ZFS.
The memstick method was successful but slow,
so I created my own dd image (via ubuntu live disk).
and then used Etcher to create new SD Cards in order build the rest of the APU2 devices.