Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    MBT-4220 update to 2.6.0 failed - TWICE

    Scheduled Pinned Locked Moved Problems Installing or Upgrading pfSense Software
    16 Posts 3 Posters 2.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • rcfaR
      rcfa
      last edited by

      I have two pfSense units, one an ancient Dell server: update to 2.6.0 worked (at least as far as I can tell) flawlessly.

      Updating the Netgate MBT-4220 looked the same, except it never came back after the reboot.
      As a matter of fact, it was stuck at a kernel panic after the reboot.

      It was possible for a little while to get it to boot by stopping the boot at the boot menu, and selecting option 6, at which it would boot.
      Then things would ALMOST work as they should: except no traffic would be routed to/from the DMZ.

      I was silly enough to follow the instructions under “Upgrade troubleshooting” to reinstall everything with

      pkg-static clean -ay; pkg-static install -fy pkg pfSense-repo pfSense-upgrade
      pkg-static upgrade -f
      

      which worked seemingly without major errors, except, after trying to reboot, the same issue: system panic; except now, the old kernel was also overwritten, so now not even option 6 allowed for a reboot anymore.

      So, I figured, time for a re-install. I grabbed my hand-dandy USB-key with the ancient pfSense version on it, that shipped with the MBT-4220. Except, I don’t seem to be able to convince the device to boot from the USB stick. (Any hints there?)

      So, why won’t this 2.6.0 release boot a MBT-4220 without a panic? Are there hardware requirements in 2.6.0 that the MBT-4220 doesn’t fulfill?

      1 Reply Last reply Reply Quote 0
      • jimpJ
        jimp Rebel Alliance Developer Netgate
        last edited by

        Did you install with ZFS or UFS? There is an issue with those MBT systems where they can't boot with both the ZFS modules and the video driver kernel module loaded, which might be what you're hitting. Without seeing the panic backtrace text it's hard to say that with any certainty, though.

        There are other posts around which cover how to work around that:

        • https://forum.netgate.com/post/1029206
        • https://forum.netgate.com/post/991209

        Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

        Need help fast? Netgate Global Support!

        Do not Chat/PM for help!

        rcfaR 2 Replies Last reply Reply Quote 0
        • rcfaR
          rcfa @jimp
          last edited by rcfa

          @jimp While I can't say that's it, it very much sounds like it, because I always had ZFS and video console active, which wasn't an issue until I guess 2.6.0 :(

          Now what? I can attempt to install an older version of the OS, but I need a video console, I have no other somewhat user-friendly way of configuring/maintaining the system. And ZFS was what I was happy to use for years now, on these very systems...

          Again, I had a working VGA-console with ZFS setup up and including 2.5.x, so what is the issue with 2.6.0 not allowing both to be active? This decidedly sounds like a bug, not a feature, because if up till 2.5.x can do it, why not 2.6.0?

          Here’s where things panic:ADA3D338-9F40-4879-878F-71E46C4E131A.jpeg

          rcfaR 1 Reply Last reply Reply Quote 0
          • rcfaR
            rcfa @rcfa
            last edited by

            @rcfa Assuming, and it's highly likely, that I ran into the ZFS/VGA-console issue; I think I'm now caught in a chicken and egg problem: I can't edit the /boot/loader.conf.local file without being booted, but I can't successfully boot without...

            ...so, is there a way that one can manually disable the VGA driver, e.g. through the BIOS settings, thus preventing a successful loading of the driver, and thus getting the system to boot?

            Trying to figure out a way to solve this, without a road-trip...

            As for the future: Any plans to fix this, e.g. by using CoreBoot or something like that?
            Attempts at fixing this are made double difficult, as the minnowboard.org web site is largely defunct, almost all the links to the the tutorial and documentation pages just link back to the main page, and the internet archive doesn't seem to have crawled many/most of these relevant pages, either.

            rcfaR 1 Reply Last reply Reply Quote 0
            • rcfaR
              rcfa @rcfa
              last edited by

              @rcfa One more thing that's not totally standard: I have the GPS-lure installed, and use that as a ntp time source...
              ...again, wasn't an issue until with 2.6.0

              1 Reply Last reply Reply Quote 0
              • rcfaR
                rcfa @jimp
                last edited by rcfa

                @jimp So, after much ado, I managed to boot off a USB stick and mount the zfs partition, in full anticipation to edit /boot/loader.conf.local such as to not load the graphics driver.

                Needless to say, this has me baffled: (from the working non MBT-4220 system, equivalent result on the the MBT-4220, but I can't copy/paste from there...)

                [2.6.0-RELEASE][root@pfSense]/root: find / -name "loader.conf*" -print
                /boot/defaults/loader.conf
                /boot/loader.conf
                /boot/loader.conf.d
                [2.6.0-RELEASE][root@pfSense]
                

                There's no /boot/loader.conf.local !!

                So the instructions as per https://forum.netgate.com/topic/170008/which-netgate-devices-are-zfs-capable/8
                don't really seem to apply.

                1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by

                  Hmm, have you updated the BIOS on that 4220? There are two device types we have seen that differ by BIOS version. The installer uses these to recognise the board but perhaps yours is reporting something else.

                  The panic you saw above is not what I expect to see after trying to install and boot ZFS.

                  By far the easiest way to do this, if you want to run ZFS, is to choose the 'make changes' oprion at the end of the installer and edit the loader file then.

                  Steve

                  rcfaR 1 Reply Last reply Reply Quote 0
                  • rcfaR rcfa referenced this topic on
                  • rcfaR rcfa referenced this topic on
                  • rcfaR rcfa referenced this topic on
                  • rcfaR rcfa referenced this topic on
                  • rcfaR
                    rcfa @stephenw10
                    last edited by rcfa

                    @stephenw10 Thing is, I didn't install from scratch, but updated a stable running 2.5.x installation through the web interface.
                    Up until now, I always had access to the VGA console and was running zfs, only with 2.6.0 was there an issue, that was attributed to the zfs/VGA-console combination.

                    This is the reported UEFI BIOS version: MNW2MAX1.X64.0100.R01.1811141644

                    This is the reported platform:
                    Minnowboard Turbot D0/D1 Platform

                    So its the/an updated BIOS, but things were running fine under 2.5.x, just the 2.6.0 update broke things.

                    1 Reply Last reply Reply Quote 0
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by

                      If you installed a much earlier version that code may not have been in the installer and if you didn't create loader.conf.local at that time it would not have been added.
                      It doesn't get created at upgrade only during install.

                      What you're hitting does not look like that issue.

                      I would install 2.6 clean and see if you're still hitting a problem.

                      Steve

                      rcfaR 1 Reply Last reply Reply Quote 1
                      • rcfaR
                        rcfa @stephenw10
                        last edited by

                        @stephenw10 Doing a clean install is going to be difficult...
                        I'm literally walking a computer novice step by step through what to do, by sending the command line commands through iMessage, and getting screen shots made with an iPhone back as a reply.
                        So burning a new USB stick is pretty much out of the question, particularly since the network access at that location, aside from the mobile internet on my friend's iPhone, is depending on the pfSense unit being operational.

                        So I have two options: reinstalling the original 2.x version that's on the USB-stick that came with the MBT-4220, getting it minimally configured for remote access (which may be a challenge in itself: how does one configure remote admin access through the CLI?) and then upgrading again to 2.6.0..
                        ...OR...
                        ...figuring out what is the problem, fixing the configuration through accessing the file system by means of a rescue session boot.

                        stephenw10S 1 Reply Last reply Reply Quote 0
                        • stephenw10S
                          stephenw10 Netgate Administrator @rcfa
                          last edited by

                          @rcfa said in MBT-4220 update to 2.6.0 failed - TWICE:

                          how does one configure remote admin access through the CLI?

                          You can use the easyrule command to add a WAN firewall rule form your source IP.
                          https://docs.netgate.com/pfsense/en/latest/firewall/easyrule.html

                          Steve

                          rcfaR 1 Reply Last reply Reply Quote 1
                          • rcfaR
                            rcfa @stephenw10
                            last edited by

                            @stephenw10 Awesome, thanks!
                            Is the web interface automatically available on the WAN, once the firewall lets traffic pass, or does remove admin still be enabled somehow? (Can't remember, it's been such a long time since I initially set up these systems; since then it's just been updates and minor tweaks...)

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              Yes, the webgui listens on all interfaces. You only need open a firewall rule to access it.

                              1 Reply Last reply Reply Quote 0
                              • rcfaR
                                rcfa
                                last edited by rcfa

                                In case anyone finds this having similar issues:

                                1. it was NOT the ZFS/i915 driver incompatibility, since the i915 driver was never loaded and the /boot/loader.conf.local file was never generated (continuous updates from an old install)

                                2. the actual cause for the update failure could not be determined, however configuration issues can be ruled out, as the very configuration in place before the upgrade was backed up, and restored in the end.

                                3. the solution was to install the OS from the original USB key shipped with the MBT-4220 and then update in two stages to 2.6.0, and then to restore the backed up config,

                                4. the easiest way to do this remotely was to walk someone through the install from the USB stick (remote location), and at the end guide them to enter the shell and punch in

                                easyrule pass wan any any any any
                                

                                after which one can take over remotely with the web UI, update the OS, and then restore the backed up configuration.
                                Unless one's a high value target under constant attack, the few minutes during which the system is fully open to the wan, I consider rather a low risk, and the simplicity of the approach is worth it, unless one has trained staff at hand on the remote end.

                                Things are back up and working, same configuration, so the upgrade failure remains a mystery.

                                Hope this may help someone if anyone reals with the same or similar mess.

                                1 Reply Last reply Reply Quote 0
                                • stephenw10S
                                  stephenw10 Netgate Administrator
                                  last edited by

                                  Hmm, you were never able to boot from the 2.6 installer then?

                                  Weird.

                                  rcfaR 1 Reply Last reply Reply Quote 0
                                  • rcfaR
                                    rcfa @stephenw10
                                    last edited by

                                    @stephenw10 I don’t know; the original USB stick I had, didn’t feel like walking a non-IT person through downloading and burning a USB stick over a iPhone tethering network connection…

                                    It was simpler using the existing stick and updating…

                                    1 Reply Last reply Reply Quote 1
                                    • First post
                                      Last post
                                    Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.