24.03 Update not booting

maverickws

Got a possible culprit

https://forum.netgate.com/topic/187655/24-03-install-failed-in-1-out-of-3

hypnosis4u2nv

@stephenw10 here's the console when trying to boot up and hangs.And yes, the upgrade completes successfully but hangs at this point.

stephenw10

Ah that's a drive or drive controller error. It's possible the new BE is using some bad part of the drive. Also possible the newer drivers in 24.03 are causing some problem there.

Do you know what drive/drive controller it's using?

hypnosis4u2nv

@stephenw10 Intel Pro/1000 5.6.10 PCI-E

Samsung 256GB SSD.

I did try updating again and using another boot environment, same issue persists. I also got rid of some older BEs and tried that, it still fails. This was a problem with the development builds as well.

stephenw10

Can you connect a serial console and get a full log from the upgrade?

Do you see any errors in the 23.09.1 boot log from the drive or controller when they are detected?

hypnosis4u2nv

@stephenw10 I actually ordered a console cable last night so I can troubleshoot during the install. I do have the full update log from the GUI during the update process.

I looked at the BootOS log and didn't see anything abnormal, although it's only the log for the 23.09.1 BE, there's nothing for 24.03.

Another interesting thing is that after the update, my 23.09. BE shows errors with package manager. These packages should be all up to date.

stephenw10

It's because the repo branch is set to 24.03. It's seeing the newer version there. But you can't install them because they are not compatible with 23.09.1. If you set the branch back those should show as up to data again. Unless there actually are updates.

Mike-moon

Hi,
I am curious on your findings as my system behaves the probably the same. No clue whether it is also a driver issue but it always worked fine with 23.09.1 but does not start with 24.03. (The old start log shows a missing driver for my onboard WiFi but this should certainly not prevent the new system from starting.)
Regards, Michael

hypnosis4u2nv

@stephenw10 Ah, makes sense. Didn't realize it stayed on that branch after reverting back.

Here's the update log from the GUI for 24.03 if that helps.Update.txt

hypnosis4u2nv

@stephenw10 Here's the boot log for 23.09.1 boot log.txt

Mike-moon

@stephenw10 : In case it helps, here also my dmesg.txt from 23.09.1.

stephenw10

Ok different drives and different driver controllers.

@Mike-moon Are you actually seeing the same identical error?

@hypnosis4u2nv Your boot log from 23.09.1 shows the same error:

ahci0: <Intel Sunrise Point-LP AHCI SATA controller> port 0xf090-0xf097,0xf080-0xf083,0xf060-0xf07f mem 0xdf628000-0xdf629fff,0xdf62c000-0xdf62c0ff,0xdf62b000-0xdf62b7ff irq 16 at device 23.0 on pci0
ahci0: AHCI v1.31 with 3 6Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
...
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <SAMSUNG MZMTE256HMHP-000MV EXT41M0Q> ACS-2 ATA SATA 3.x device
ada0: Serial Number S1F1NSAG782964
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 244198MB (500118192 512 byte sectors)
ahcich0: Timeout on slot 28 port 0
ahcich0: is 00000000 cs 00000000 ss 10000000 rs 10000000 tfd 40 serr 00000000 cmd 0000c217
(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 10 90 2d 20 40 0b 00 00 00 00 00
(ada0:ahcich0:0:0:0): CAM status: Command timeout
(ada0:ahcich0:0:0:0): Retrying command, 3 more tries remain

It seems likely to be the disk. Can you test a different drive?

hypnosis4u2nv

@stephenw10 Don't have a spare drive. Wouldn't hurt to get another one to swap out. Although I'll be starting a fresh install when I do. It's interesting though that it continues to boot past the error while on 24.03 it wants to hang.

Mike-moon

@stephenw10 : Seems that I do not have exactly the identical error. System seems to hang already in the bootloader.

stephenw10

It is. It could be a driver difference there causing that as a symptom and not a cause. I'm not aware of anything known that presents like that.

I would try to compare the boot log from 23.09.1 to that from 24.03. You might find, for example, that 24.03 has additional drivers that attach to some device you have and that is causing a conflict.

Mike-moon

@stephenw10 : But the system did not start at all after four upgrade attempts. Can I nevertheless find log files that may point on the conflict? Is there a boot log from 24.03 in case the boot was not successful?

stephenw10

Nope sorry our replies got out of order, my last was for the drive/drive controller issue.

You are stuck on the EFI frame buffer so some UEFI issue. I assume that's not a VM?

Mike-moon

@stephenw10 : No, it's a real machine. Is there anything I could do?

stephenw10

What hardware is it? It must have some UEFI quirk we have not seen locally.

hypnosis4u2nv

@stephenw10 Anyway to get the boot logs for 24.03 without a console cable? I ordered an mSATA drive for the box so I'll have to wait on that or the console cable, whichever gets here first. Curiously, are you seeing any reports similar to this issue?