MBT-4220 update to 2.6.0 failed - TWICE

[rcfa]

@rcfa Assuming, and it's highly likely, that I ran into the ZFS/VGA-console issue; I think I'm now caught in a chicken and egg problem: I can't edit the /boot/loader.conf.local file without being booted, but I can't successfully boot without...

...so, is there a way that one can manually disable the VGA driver, e.g. through the BIOS settings, thus preventing a successful loading of the driver, and thus getting the system to boot?

Trying to figure out a way to solve this, without a road-trip...

As for the future: Any plans to fix this, e.g. by using CoreBoot or something like that?
Attempts at fixing this are made double difficult, as the minnowboard.org web site is largely defunct, almost all the links to the the tutorial and documentation pages just link back to the main page, and the internet archive doesn't seem to have crawled many/most of these relevant pages, either.

[rcfa]

@rcfa One more thing that's not totally standard: I have the GPS-lure installed, and use that as a ntp time source...
...again, wasn't an issue until with 2.6.0

[rcfa]

@jimp So, after much ado, I managed to boot off a USB stick and mount the zfs partition, in full anticipation to edit /boot/loader.conf.local such as to not load the graphics driver.

Needless to say, this has me baffled: (from the working non MBT-4220 system, equivalent result on the the MBT-4220, but I can't copy/paste from there...)

[2.6.0-RELEASE][root@pfSense]/root: find / -name "loader.conf*" -print
/boot/defaults/loader.conf
/boot/loader.conf
/boot/loader.conf.d
[2.6.0-RELEASE][root@pfSense]

There's no /boot/loader.conf.local !!

So the instructions as per https://forum.netgate.com/topic/170008/which-netgate-devices-are-zfs-capable/8
don't really seem to apply.

[stephenw10]

Hmm, have you updated the BIOS on that 4220? There are two device types we have seen that differ by BIOS version. The installer uses these to recognise the board but perhaps yours is reporting something else.

The panic you saw above is not what I expect to see after trying to install and boot ZFS.

By far the easiest way to do this, if you want to run ZFS, is to choose the 'make changes' oprion at the end of the installer and edit the loader file then.

Steve

[rcfa]

@stephenw10 Thing is, I didn't install from scratch, but updated a stable running 2.5.x installation through the web interface.
Up until now, I always had access to the VGA console and was running zfs, only with 2.6.0 was there an issue, that was attributed to the zfs/VGA-console combination.

This is the reported UEFI BIOS version: MNW2MAX1.X64.0100.R01.1811141644

This is the reported platform:
Minnowboard Turbot D0/D1 Platform

So its the/an updated BIOS, but things were running fine under 2.5.x, just the 2.6.0 update broke things.

[stephenw10]

If you installed a much earlier version that code may not have been in the installer and if you didn't create loader.conf.local at that time it would not have been added.
It doesn't get created at upgrade only during install.

What you're hitting does not look like that issue.

I would install 2.6 clean and see if you're still hitting a problem.

Steve

[rcfa]

@stephenw10 Doing a clean install is going to be difficult...
I'm literally walking a computer novice step by step through what to do, by sending the command line commands through iMessage, and getting screen shots made with an iPhone back as a reply.
So burning a new USB stick is pretty much out of the question, particularly since the network access at that location, aside from the mobile internet on my friend's iPhone, is depending on the pfSense unit being operational.

So I have two options: reinstalling the original 2.x version that's on the USB-stick that came with the MBT-4220, getting it minimally configured for remote access (which may be a challenge in itself: how does one configure remote admin access through the CLI?) and then upgrading again to 2.6.0..
...OR...
...figuring out what is the problem, fixing the configuration through accessing the file system by means of a rescue session boot.

[stephenw10]

@rcfa said in MBT-4220 update to 2.6.0 failed - TWICE:

how does one configure remote admin access through the CLI?

You can use the easyrule command to add a WAN firewall rule form your source IP.
https://docs.netgate.com/pfsense/en/latest/firewall/easyrule.html

Steve

[rcfa]

@stephenw10 Awesome, thanks!
Is the web interface automatically available on the WAN, once the firewall lets traffic pass, or does remove admin still be enabled somehow? (Can't remember, it's been such a long time since I initially set up these systems; since then it's just been updates and minor tweaks...)

[stephenw10]

Yes, the webgui listens on all interfaces. You only need open a firewall rule to access it.

[rcfa]

In case anyone finds this having similar issues:

1. it was NOT the ZFS/i915 driver incompatibility, since the i915 driver was never loaded and the /boot/loader.conf.local file was never generated (continuous updates from an old install)

2. the actual cause for the update failure could not be determined, however configuration issues can be ruled out, as the very configuration in place before the upgrade was backed up, and restored in the end.

3. the solution was to install the OS from the original USB key shipped with the MBT-4220 and then update in two stages to 2.6.0, and then to restore the backed up config,

4. the easiest way to do this remotely was to walk someone through the install from the USB stick (remote location), and at the end guide them to enter the shell and punch in

easyrule pass wan any any any any

after which one can take over remotely with the web UI, update the OS, and then restore the backed up configuration.
Unless one's a high value target under constant attack, the few minutes during which the system is fully open to the wan, I consider rather a low risk, and the simplicity of the approach is worth it, unless one has trained staff at hand on the remote end.

Things are back up and working, same configuration, so the upgrade failure remains a mystery.

Hope this may help someone if anyone reals with the same or similar mess.

[stephenw10]

Hmm, you were never able to boot from the 2.6 installer then?

Weird.

[rcfa]

@stephenw10 I don’t know; the original USB stick I had, didn’t feel like walking a non-IT person through downloading and burning a USB stick over a iPhone tethering network connection…

It was simpler using the existing stick and updating…