Cannot boot 23.05.1 (sg-2100 w/zfs)

[leres]

I recently got a sg-2100 and did a usb stick clean install of 23.05.1 with zfs. I've run the system for months and rebooted several times without issue.

Today I got back from vacation and rebooted to pick up the new unbound, openvpn, etc. and got this:

FreeBSD/arm64 EFI loader, Revision 1.1mvebu_spi_ofdata_to_platdata: ofdata clock(Wed Jun 28 04:23:05 UTC 2023 root@freebsd)
SF: Detected w25q32bv with page size 256 Bytes, erase size 4 KiB, total 4 MiB
   Command line arguments: loader.efi
   Image base: 0x1000000
   EFI version: 2.70000 [PRIME], eth1: neta@40000
   EFI Firmware: Das U-Boot (rev 0.00)
   Console: efi,comconsole (0)
   Load Path: /\efi\boot\bootaa64.efi
   Load Device: /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(0,0)/HD(1,0x01,0,0x1,0x82000)
Trying ESP: /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(0,0)/HD(1,0x01,0,0x1,0x82000)
Setting currdev to disk1p1:TA Prod.: TS128GMTS430S Rev: 22Z2
Trying: /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(0,0)/HD(2,0x01,0,0x82001,0x20000) Capacity: 122104.3 MB = 119.2 GB (250069680 x 512)
Setting currdev to disk1p2:gate-1100.dtb
Trying: /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(0,0)/HD(3,0x01,0,0xa2001,0xedda2af)rmada-3720-netgate-2100.dtb
Setting currdev to disk1p3:100.dtb
ERROR: cannot open /boot/lua/loader.lua: no such file or directory.
4 file(s), 0 dir(s)

I'm totally hosed, the only way I could get on line to search for solutions was to plug a laptop into the fiber box...

Here's what "show" gives:

OK show
COLUMNS=80
LINES=25
autoboot_delay=NO
console=efi,comconsole
currdev=disk1s2:
efi-version=2.70
efi_com_port=0
efi_com_speed=0
hint.smbios.0.mem=0xe7ff5000
interpret=OK
loaddev=disk1s2:
loader.efi=1
module_verbose=2
prompt=${interpret}
script.lang=lua
smbios.bios.reldate=02/07/2020
smbios.bios.vendor=U-Boot
smbios.bios.version=2018.03-devel-1.2.0ROGUE2-01.00.00.02+
smbios.chassis.maker=Silicom
smbios.chassis.serial=????????58
smbios.chassis.type=Desktop
smbios.chassis.version=R305
smbios.planar.maker=Silicom
smbios.planar.product=80500-0205-G00
smbios.planar.serial=????????58                         
smbios.planar.version=R305                              
smbios.socket.enabled=1                                 
smbios.system.maker=Silicom                             
smbios.system.product=SG-2100
smbios.system.serial=????????58
smbios.system.version=R305
smbios.version=3.0
twiddle_divisor=16

I hate to have to reinstall (especially since I don't understand what went wrong).

I can probably remove the M.2 and mount it on another system.

Can I use the usb media I used to install to fix it?

Help?

[leres]

Ok so it looks like I was confused about what I was running. I had installed 23.05.1 but later upgraded to 23.09. I'm not sure how many times I rebooted that version.

[SteveITS]

@leres IIRC we’ve seen that once but someone had posted a similar error at the time. Around last year? We did recover that one via reinstall, but had needed to do that anyway since it had the tiny EFI partition. It’s been used sporadically since then but has been fine.

You can install with the 23.05 stick, upgrade, then restore your 23.09 config file.

[stephenw10]

Better to just recover to 23.09 directly otherwise you are importing a config into an older version which can cause problems.

[SteveITS]

@stephenw10 By “upgrade” I meant to 23.09. :) I was assuming OP couldn’t or didn’t want to create a newer stick.

[leres]

@SteveITS
I did the initial install on new 128G M.2 media so I expect partition size was not an issue.

And in the end I did exactly what you suggested: reinstall (then upgrade) and restore the last config.xml. This was a little bit painful in that my "lan" port only trunks vlans so I had to run patch cord to be able to access the web gui.

[SteveITS]

@leres said in Cannot boot 23.05.1 (sg-2100 w/zfs):

expect partition size was not an issue.

Glad you got it working.

The EFI size wasn't an issue if it had ZFS already and/or was newer than early 2022, IIRC. I just mentioned it because we had started planning to reinstall all those 2100s. :-/

We had similar experiences with two clients' 2100s where installing from the same USB stick was not stable (second boot/restart fails, boots up and installs packages then drops offline, etc.) and simply using a different USB stick to do the install has worked fine since then (this past spring). Very strange but seems to be the stick...which we tossed.

ref:
https://forum.netgate.com/topic/180755/23-05-firmware-upgrade-crashed-a-3100-and-an-1100/
https://forum.netgate.com/topic/180432/certificate-verification-failed/
23.05.1 was supposed to have fixes already though for those threads.