PC Engines APU4C4 Experience
-
Recently I went over to from UFS to ZFS on a APU2C4 (RAM 4GB) with mSATA 16Gb, fresh install from USB then I restored the config.xml. I have a few Q's maybe someone can help, log file below...
[1+ Disks] Striping provides maximum storage but no redundancy pfSense Installer qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq lqqqqqqqqqqqqqqqqZFS Configurationqqqqqqqqqqqqqqqqqqqk x Configure Options: x x lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk x x x >>> Install Proceed with Installation x x x x T Pool Type/Disks: stripe: 0 disks x x x x - Rescan Devices * x x x x - Disk Info * x x x x N Pool Name zroot x x x x 4 Force 4K Sectors? YES x x x x E Encrypt Disks? NO x x x x P Partition Scheme GPT (BIOS) x x x x S Swap Size 2g x x x x M Mirror Swap? NO x x x x W Encrypt Swap? NO x x x mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj x tqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqu x <Select> <Cancel> x mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj Create ZFS boot pool with displayed options pfSense Installer qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq lqqqqqqqqqqqqqZFS Configurationqqqqqqqqqqqqqqqk x Select Virtual Device type: x x lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk x x x stripe Stripe - No Redundancy x x x x mirror Mirror - n-Way Mirroring x x x x raid10 RAID 1+0 - n x 2-Way Mirrors x x x x raidz1 RAID-Z1 - Single Redundant RAID x x x x raidz2 RAID-Z2 - Double Redundant RAID x x x x raidz3 RAID-Z3 - Triple Redundant RAID x x x mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj x tqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqu x < OK > <Cancel> x mqqqqqqqq[Press arrows, TAB or ENTER]qqqqqqqqqj [1+ Disks] Striping provides maximum storage but no redundancy pfSense Installer qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq lqqqqqqqqZFS Configurationqqqqqqqqqqk x lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk x x x[*] ada0 SATA SSD x x x x[ ] da0 VendorCo ProductCode x x x mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj x tqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqu x < OK > < Back > x mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj pfSense Installer qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq lqqqqqqqqqqqqqqZFS Configurationqqqqqqqqqqqqqqqqqk x Last Chance! Are you sure you want to destroy x x the current contents of the following disks: x x x x ada0 x x x tqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqu x < YES > < NO > x mqqqqqqqqq[Press arrows, TAB or ENTER]qqqqqqqqqqqj ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is present; to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf. ZFS filesystem version: 5 ZFS storage pool version: features support (5000) kern.geom.part.mbr.enforce_chs: 0 -> 0 pfSense Installer qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq lqqqqqqqqqqFetching Distributionqqqqqqqqqqqqk x x x MANIFEST [ Done ] x x base.txz [ 100% ] x x x x Fetching distribution files... x x x x lqOverall Progressqqqqqqqqqqqqqqqqqqqqk x x x 100% x x x mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj x mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj pfSense Installer qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq lqqqqqqqqqqChecksum Verificationqqqqqqqqqqqqk x x x base.txz [In Progress ] x x x x Verifying checksums of selected x x distributions. x x x x lqOverall Progressqqqqqqqqqqqqqqqqqqqqk x x x 0% x x x mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj x mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj pfSense Installer qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk x Checking distribution archives. x x Please wait... x mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj pfSense Installer qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq lqqqqqqqArchive Extractionqqqqqqqqqk x Extracting distribution files... x x x x x x Overall Progress: x x lqqqqqqqqqqqqqqqqqqqqqqqqqqqqk x x x 100% x x x mqqqqqqqqqqqqqqqqqqqqqqqqqqqqj x mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj Looking for config.xml on da0s3: [found msdos] [no config.xml on da0s3, unmounting] diskid/DISK-920702597E285218784s3: [Could not locate an existing config.xml file!] Done. pfSense Installer qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq lqqqqqqqManual Configurationqqqqqqqqqqk x The installation is now finished. x x Before exiting the installer, would x x you like to open a shell in the new x x system to make any final manual x x modifications? x tqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqu x < Yes > < No > x mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj pfSense Installer qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq lqqqqqqqqqCompleteqqqqqqqqqqqk x Installation of pfSense x x complete! Would you like x x to reboot into the x x installed system now? x tqqqqqqqqqqqqqqqqqqqqqqqqqqqqu x <Reboot> <Shell > x mqqqqqqqqqqqqqqqqqqqqqqqqqqqqj Nov 6 10:28:00 reboot: rebooted by root Nov 6 10:28:00 Waiting (max 60 seconds) for system process `vnlru' to stop... done Waiting (max 60 seconds) for system process `bufdaemon' to stop... done Waiting (max 60 seconds) for system process `syncer' to stop... Syncing disks, vnodes remaining... 0 0 done All buffers synced. Uptime: 26m58s umass0: detached uhub2: detached PC Engines apu2 coreboot build 20190810 BIOS version v4.10.0.2 4080 MB ECC DRAM SeaBIOS (version rel-1.12.1.3-0-g300e8b7) Press F10 key now for boot menu Booting from Hard Disk... Boot failed: could not read the boot disk Trying next device: 2 Booting from Hard Disk... gptzfsboot: error 12 lba 32 gptzfsboot: error 12 lba 544 gptzfsboot: error 12 lba 1965088 gptzfsboot: error 12 lba 1965600 gptzfsboot: error 12 lba 1966072 gptzfsboot: error 12 lba 1 /boot/config: -S115200 -h Consoles: serial port BIOS drive C: is disk0 BIOS drive D: is disk1 read 1 from 0 to 0xcdf89440, error: 0xcread 1 from 0 to 0xcdf89440, error: 0xcBy FreeBSD/x86 ZFS enabled bootstrap loader, Revision 1.1 (Wed Nov 21 08:03:10 EST 2018 root@buildbot2.nyi.netgate.com) Loading /boot/defaults/loader.conf - __ _ __ / _|___ ___ _ __ ___ ___ | '_ \| |_/ __|/ _ \ '_ \/ __|/ _ \ | |_) | _\__ \ __/ | | \__ \ __/ | .__/|_| |___/\___|_| |_|___/\___| |_| +============Welcome to pfSense===========+ __________________________ | | / ___\ | 1. Boot Multi User [Enter] | | /` | 2. Boot [S]ingle User | | / :-| | 3. [Esc]ape to loader prompt | | _________ ___/ /_ | | 4. Reboot | | /` ____ / /__ ___/ | | | | / / / / / / | | Options: | | / /___/ / / / | | 5. [K]ernel: kernel (1 of 2) | | / ______/ / / | | 6. Configure Boot [O]ptions... | |/ / / / | | 7. Select Boot [E]nvironment... | / /___/ | | | / | | | /_________________________/ +=========================================+ /boot/kernel/kernel text=0x17c1930 data=0xb93d38+0x557b28 syms=[0x8+0x197400+0x] /boot/entropy size=0x1000 /boot/kernel/zfs.ko size 0x3821b0 at 0x2fde000 loading required module 'opensolaris' /boot/kernel/opensolaris.ko size 0xa3b0 at 0x3361000First Q
As you can see, there are a few error on first boot (did not see them at second boot)
gptzfsboot: error 12 lba 32
gptzfsboot: error 12 lba 544
gptzfsboot: error 12 lba 1965088
gptzfsboot: error 12 lba 1965600
gptzfsboot: error 12 lba 1966072
gptzfsboot: error 12 lba 1What are they?
Second Q
The log mentioned
ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is present;
to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf.
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
kern.geom.part.mbr.enforce_chs: 0 -> 0
pfSense InstallerAs the APU2C4 has 4Gb memory, do I have to adjust it (see loader.conf below) and what does it bring?
[2.4.4-RELEASE][root@pfSense.localdomain]/boot: cat loader.conf kern.cam.boot_delay=10000 kern.ipc.nmbclusters="1000000" kern.ipc.nmbjumbop="524288" kern.ipc.nmbjumbo9="524288" kern.geom.label.disk_ident.enable="0" kern.geom.label.gptid.enable="0" zfs_load="YES" boot_serial="YES" console="comconsole" comconsole_speed="115200" autoboot_delay="3" hw.usb.no_pf="1"Third Q
Why does fstyp still mention UFS?
[2.4.4-RELEASE][root@pfSense.localdomain]/root: df -h Filesystem Size Used Avail Capacity Mounted on zroot/ROOT/default 12G 487M 11G 4% / devfs 1.0K 1.0K 0B 100% /dev zroot/tmp 11G 2.9M 11G 0% /tmp zroot/var 12G 150M 11G 1% /var zroot 11G 88K 11G 0% /zroot /dev/md0 3.4M 148K 3.0M 5% /var/run devfs 1.0K 1.0K 0B 100% /var/dhcpd/dev /lib 12G 487M 11G 4% /var/unbound/lib devfs 1.0K 1.0K 0B 100% /var/unbound/dev /var/log/pfblockerng 12G 150M 11G 1% /var/unbound/var/log/pfblockerng /usr/local/share/GeoIP 12G 487M 11G 4% /var/unbound/usr/local/share/GeoIP /usr/local/bin 12G 487M 11G 4% /var/unbound/usr/local/bin /usr/local/lib 12G 487M 11G 4% /var/unbound/usr/local/lib [2.4.4-RELEASE][root@pfSense.localdomain]/root: fstyp -l /dev/md0 ufsFourth Q
From what I read TRIM is enabled on ZFS, but how can I check it? ...
[2.4.4-RELEASE][root@pfSense.localdomain]/root: tunefs -p / tunefs: /: could not open special deviceThanks for any help
Cheers Qinn
-
-
Looks like some problem with the GPT header that it was able to recover. Of you don't see it again then it's likely been fixed at the first boot.
-
You don't have to set that. It would probably only be a marginal drive performance increase and disk performance is not hugely important for pfSense. If you do choose to set it put it in /boot/loader.conf.local to avoid it being overwritten.
-
Ram drives are still formatted UFS.
-
No idea.
Steve
-
-
Thanks Steve, would it be wise to set ( then maybe I have data redundancy, but not disk redundancy)
zfs set copies=2 yourpoolnameBTW I hope that anyone can shed a light on TRIM?
-
That's not something I've ever tried. If you have the space I guess it makes sense. It would double the writes to the drive though.
-
@Qinn said in PC Engines APU4C4 Experience:
BTW I hope that anyone can shed a light on TRIM?
What is the output of
sysctl vfs.zfs.trimYou should see something like:
[2.4.4-RELEASE] [root@b..]/root: sysctl vfs.zfs.trim vfs.zfs.trim.max_interval: 1 vfs.zfs.trim.timeout: 30 vfs.zfs.trim.txg_delay: 32 vfs.zfs.trim.enabled: 1 [2.4.4-RELEASE] [root@b..]/root:vfs.zfs.trim.enabled: 1 means TRIM is enabled.
-
Thanks @bigsy
[2.4.4-RELEASE][root@pfSense.localdomain]/root: sysctl vfs.zfs.trim vfs.zfs.trim.max_interval: 1 vfs.zfs.trim.timeout: 30 vfs.zfs.trim.txg_delay: 32 vfs.zfs.trim.enabled: 1Did not change anything after install, seems @doktornotor was right https://forum.netgate.com/topic/84867/2-2-x-zfs-full-install-howto/9
Also here https://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/all-about-zfs.html
Nice thread here https://forum.netgate.com/topic/112490/how-to-2-4-0-zfs-install-ram-disk-hot-spare-snapshot-resilver-root-drive from @pfBasic every command uses "yourpoolname" although I think it's in most cases "zroot" is there anyway to check, it seems he did not mentioned it.
For now I used :
[2.4.4-RELEASE][root@pfSense.localdomain]/root: zpool status pool: zroot state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 ada0p3 ONLINE 0 0 0 errors: No known data errorsbtw whats ada0p3 and why is the ram drive on UFS?
-
Hello
Today i got msata Drives and i tryed to do a ZFS install - Install runs fine - no errors at all - but after Reboot System dont boot from msata
Tryed 5-6 Times - allways same issue - then installed standard UFS Auto installer - and this Installation boot without any problems.
Can anyone confirm this ?
-
Some Additional Information:
I have now successfully installed pfsense with ZFS on internal SATA Drive - System boot from MSATA - everything seem to be fine.
Do some Software Reboots everything is ok
The i do a Power Off (hardware) and then Reboot - and then System dont boot from ZFS Installation anymore.
Dont get me wrong - i choose ZFS because i read about that ZFS is more stable at Unplaned Power OFF Situations but thats does not seem to work here ?!?
Does anyone have other Information or better Results here ?
-
I just grabbed an APU4C4 (BIOS 4.10.0.2) and did a zfs install to test. Boots fine. Power off, pull plug, replace plug- boots fine. Reboot from console-works. Yank power cord, then plug back in- boots normally.
What are you seeing on the console when it doesn't boot? -
Hi
just make Bios Update to 4.10.0.2
Then make new install using ZFS (AUTO)
I Dont make any changes only "Stripe" and sda0 for the single msata Disk
Nothing else changed - is this right ?Then Installation runs trough until end is reached.
Then Reboot.
Looking good.
I will try again in report ...
-
My test was a vanilla install using zfs. My msata (Samsung 64 GB) shows as ada0, not sda0.
-
Hi
just did 25 Power Shutdowns and It boot well - dont know why other tests today wernt wrong ....
Perhaps its a Firmware Issue - i dont used 4.10.0.2 .... -
@gtrdriver said in PC Engines APU4C4 Experience:
Some Additional Information:
I have now successfully installed pfsense with ZFS on internal SATA Drive - System boot from MSATA - everything seem to be fine.
Do some Software Reboots everything is ok
The i do a Power Off (hardware) and then Reboot - and then System dont boot from ZFS Installation anymore.
Dont get me wrong - i choose ZFS because i read about that ZFS is more stable at Unplaned Power OFF Situations but thats does not seem to work here ?!?
Does anyone have other Information or better Results here ?
What I normally do is this:
- accept the fact that stuff can break (it will break sooner or later... it's a fact)
- keep a spare USB thumb drive with a working pfSense install image
- create a backup after changing the configuration - and store it safely
- enable auto backup and document the details required for a restore
- connect the APU2/3/4 to a UPS to overcome (short) power outages
- never pull the power cord; except when there isn't another option
I did install quite some APU2/3/4 devices with pfSense(R) open source firewall/router software distribution. Some mSATA SSDs are UFS formatted and others are ZFS formatted. Some are connected to an UPS and some aren't. Over the years we have had some rare cases where we had to reinstall pfSense and restore the configuration from backup.
Also be aware of facts like these: [quote]msata16a on shipment hold. Some customer reported problems leading to data loss, e.g. with Sophos UTM. We suspect that it is related to the TRIM function included in modern file systems (e.g. Linux EXT4). We have finally received a firmware update from our supplier. Customers may return modules to us for firmware reprogramming (data is lost), or for full credit. Modules reprogrammed by us can be recognized by a + mark on the label.[/quote] Source: https://www.pcengines.ch/msata16a.htm
-
ZFS seems the most vigorous FS available in pfS, that's why I moved over from UFS to ZFS. The (re)install is a breeze, with a current config file at hand.
My APU2C4 has ECC and to me this and the ZFS FS seems max what I can do against sudden power surges, that being said, I would always use the "halt" command before I pull the power cord ;)@bouke Kudos for pointing out the flaky mSATA issue link, didn't knew that.
