SG-1100 update from 23.09.1 to 24.03 keeps failing

stephenw10 · May 9, 2024, 7:32 PM

The -p switch cause it to print the output in fstab format and includes verbose info.
https://man.freebsd.org/cgi/man.cgi?query=mount

Yes so ZFS think it's all mounted read-write but that doesn't help if the actual storage is read-only.

mrneutron · May 9, 2024, 9:21 PM

Yeah, it sure looks like the emmc in my SG-1100 is no longer writable (upgradable).
It still boots in the configuration I had it set in, but I wonder how long before it goes non-functional?

"Certain types of disks, such as SSD and eMMC disks, may fail into a read only state where disk writes fail or are discarded, but data can still be read."

from: https://docs.netgate.com/pfsense/en/latest/troubleshooting/disk-lifetime.html

stephenw10 · May 9, 2024, 10:24 PM

Yes unfortunately I have to agree. You can try installing to USB. I tested that yesterday.

mrneutron · May 9, 2024, 10:25 PM

@stephenw10 said in SG-1100 update from 23.09.1 to 24.03 keeps failing:

Yes unfortunately I have to agree. You can try installing to USB. I tested that yesterday.

You can run the pfsense OS off of a USB flash drive, plugged into the SG-1100?

stephenw10 · May 9, 2024, 11:02 PM

Yes, the new Net Installer allows you select another USB drive as the target.

mrneutron · May 9, 2024, 11:32 PM

@stephenw10 said in SG-1100 update from 23.09.1 to 24.03 keeps failing:

Yes, the new Net Installer allows you select another USB drive as the target.

Can you use either USB port on the SG-1100? (like the USB3 port?)
Does loading into a USB flash drive automatically set the boot config to point to the USB flash drive as the boot drive?
If not, how do you manually edit the boot config?

stephenw10 · May 9, 2024, 11:59 PM

Technically you could use either port and the USB3 port would likely be much faster. However when you run either usbrecovery or usbboot it boots the first USB device and if you have two drives inserted that is the USB3 port. Which means that when you boot the installer the target drive must be in the USB2 slot. That shouldn't matter, I'd expect to be able to move the drive after installing. However I was unable to do so. The 1100 can be picky about booting USB drives though so it could just be the drive I'm using.

The default boot device is not automatically changed. You need to set the bootcmd uboot env from the uboot prompt like:

setenv bootcmd 'run usbboot; run emmcboot;'
saveenv

Now in your case you may have an additional issue that the emmc is still holding a ZFS filesystem and cannot be wiped. If that is that case you'd need to either make sure the zfs pool names are different or use UFS on the USB drive.

mrneutron · May 11, 2024, 5:13 PM

@stephenw10 said in SG-1100 update from 23.09.1 to 24.03 keeps failing:

Technically you could use either port and the USB3 port would likely be much faster. However when you run either usbrecovery or usbboot it boots the first USB device and if you have two drives inserted that is the USB3 port. Which means that when you boot the installer the target drive must be in the USB2 slot. That shouldn't matter, I'd expect to be able to move the drive after installing. However I was unable to do so. The 1100 can be picky about booting USB drives though so it could just be the drive I'm using.

The default boot device is not automatically changed. You need to set the bootcmd uboot env from the uboot prompt like:
setenv bootcmd 'run usbboot; run emmcboot;'
saveenv
Now in your case you may have an additional issue that the emmc is still holding a ZFS filesystem and cannot be wiped. If that is that case you'd need to either make sure the zfs pool names are different or use UFS on the USB drive.

Yeah, given the read-only status of the emmc, it definitely appears the file system can't be wiped.
Using UFS on a USB flash drive seems like a less-complicated workaround - less other stuff to change.
How do you change the ZFS pool names, if you want/need to?

stephenw10 · May 11, 2024, 5:56 PM

Yes, good question! There isn't an option to do that in the Net Installer TUI. Yet.

The legacy x86 installer allows setting the pool name. The legacy aarch64 installer never did because it used a prebuilt image.

So for now using UFS is the only option if you cannot remove a ZFS pool named 'pfSense' from the eMMC.

However I would argue that when running from USB the drive writes are even more important and running UFS with RAMdisks minimises that. So I would run UFS anyway.

Steve

flatsphere · May 15, 2024, 2:47 PM

@stephenw10 said in SG-1100 update from 23.09.1 to 24.03 keeps failing:

Yes, good question! There isn't an option to do that in the Net Installer TUI. Yet.

The legacy x86 installer allows setting the pool name. The legacy aarch64 installer never did because it used a prebuilt image.

So for now using UFS is the only option if you cannot remove a ZFS pool named 'pfSense' from the eMMC.

However I would argue that when running from USB the drive writes are even more important and running UFS with RAMdisks minimises that. So I would run UFS anyway.

Steve

Hi gang,

Is there a resolution to this problem yet? My SG-1100 is having the same problem.

Cheers!
Rafael

stephenw10 · May 15, 2024, 3:09 PM

So you see the 8 disks shown by uboot?

flatsphere · May 15, 2024, 5:10 PM

@stephenw10 Hi Stephen,

The latest attempt shows that I'm running out of memory.
mount -p shoes this:

Following notices also appeared:

Upgrade
pkg (3): out of space @ 2024-05-15 09:06:30
PHP errors
PHP ERROR: Type: 1, File: /usr/local/pkg/pfblockerng/pfblockerng.inc, Line: 8804, Message: Uncaught TypeError: Unsupported operand types: int + string in /usr/local/pkg/pfblockerng/pfblockerng.inc:8804
Stack trace:
#0 /usr/local/www/pfblockerng/pfblockerng.php(628): sync_package_pfblockerng('cron')
#1 /usr/local/www/pfblockerng/pfblockerng.php(155): pfblockerng_sync_cron()
#2 {main}
thrown @ 2024-05-15 10:00:35

And following disk usage:

Cheers!
Rafael

SteveITS · May 15, 2024, 5:14 PM

@flatsphere given the 2.1 GB you probably have old boot environments to delete:
https://docs.netgate.com/pfsense/en/latest/troubleshooting/filesystem-shrink.html

Also if your RAM disk is full consider disabling that.

flatsphere · May 15, 2024, 5:48 PM

@SteveITS said in SG-1100 update from 23.09.1 to 24.03 keeps failing:

https://docs.netgate.com/pfsense/en/latest/troubleshooting/filesystem-shrink.html

I deleted all but the current boot environment, turned off VRAM and still failed.

R

stephenw10 · May 15, 2024, 8:23 PM

Yeah you will either need bigger RAM disks or to just disable them for the update. I would disable them.

However this looks like a completely different error. It's not indicative of a failed eMMC. At least not what you've shown us so far.

flatsphere · May 15, 2024, 9:01 PM

@stephenw10 I just tried installing from USB. Here's the message:

R

stephenw10 · May 15, 2024, 9:21 PM

Hmm, that was installing to the eMMC from the Net Installer?

Check the logs in /var/log/messages and /tmp/bsdinstall_log

MxcZXAKM · Jun 12, 2024, 10:19 PM

Mine fails as well. SG-1100

mrneutron · Jun 20, 2024, 9:50 PM

Netgate support people were not offering any remedy for the EMMC read-only state, so I ditched my failed SG-1100. It is a failed hardware design, in my opinion.
A microSD card or an M2 SSD should have been used as the boot and file system memory, rather than EMMC.
That would make the SG-1100 recoverable by simply switching out the boot media.

I installed pfsense CE 2.7.2 on a mini PC with 2 Ethernet PHYs onboard.
[stephanw10 keeps removing the web link...so helpful]
It boots and runs SO much faster than the SG-1100.

stephenw10 · Jun 18, 2024, 9:31 PM

Yup, please don't link to competing products on amazon directly. They are easy enough to find for anyone looking.