Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Failed update from 2.6.0 -> 2.7.0 causing lots of issues

    Scheduled Pinned Locked Moved
    General pfSense Questions
    5
    22
    2.6k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • bmeeksB
      bmeeks
      last edited by

      What is your underlying hardware platform?

      I suspect you are having these issues after reboot because you are winding up with a mixed-version system somehow (some 2.6.0 stuff and some 2.7.0 stuff).

      Is your BIOS configured to use UEFI by chance? Some hardware vendors and pfSense (really FreeBSD) don't like each other when using UEFI.

      Can you login directly to the firewall console and watch the messages during the actual upgrade process AND during the follow-on reboot? Might be some clues there.

      S 1 Reply Last reply Reply Quote 0
      • S
        Stewart @bmeeks
        last edited by

        @bmeeks I can only find one other instance of this when I search online and that is that jail wasn't upgraded when the host was or something like that. Their answer from here was:

        I was upgrading the jail and that showed `13.0-RELEASE p2`, but that wasn't enough.
I created a new 13.0-RELEASE jail (after fighting with bsdinstall 12.1 see here) and that was it, configured pf just like in a physical machine.

        These units are all PCEngine APU2x4 units. This unit that just had problems have the following packages installed that I didn't remove beforehand:

        Cron
        Darkstat
        Mailreport
        nmap
        Notes
        RRD_Summary
        Status_Traffic_Totals
        sudo
        zabbix-agent4

        I've never uninstalled them before for an update but if any of them could be the culprit them I'm willing to take them out. Normally I only pull out pfBlocker and Suricata as those are the only ones that have ever given me issues (and Squid but I don't use that one anymore).

        1 Reply Last reply Reply Quote 0
        • bmeeksB
          bmeeks
          last edited by bmeeks

          It certainly will not hurt to first remove all the packages, then reboot the box just to be sure it comes back fine and healthy (it should). After this, attempt the upgrade.

          To be clear though, at this point the box in discussion for this thread should get a complete reinstall from an ISO or memstick image (either restore 2.6.0 or just go ahead and re-image to the 2.7.2 version). This box is currently hosed.

          As I mentioned, for future updates if you can login to the console of the device during the update, I would monitor the screen to see what scrolls by. That way you can see any errors that may print. Ditto for the reboot that follows the unpacking/installation of the new pfSense kernel. If the boxes are all remote, I realize this will not be possible unless you can rig some sort of backdoor process with a laptop connected to the console and a cellphone modem or something.

          Once the box is up and the basic firewall configuration appears intact, then reinstall the packages.

          What you are experiencing is definitely not normal. As for "why" it's happening, the only way to begin to make a guess is to watch the messages that are printed to the console. There may some limited info in the system log, but it depends on exactly when/where the upgrade is failing to complete.

          The error message you posted from the system log clearly signals to me you have maybe the 2.6.0 pfSense kernel but 2.7.0 PHP code (or some combination of the two). The reason I say this is that pfctl uses the relatively new libpftcl library and that library was updated with the recent pfSense release. The message "Operation not supported by device" is my clue here. The PHP code is asking pfctl to do something that the corresponding library in 2.6.0 pfSense does not support (but 2.7.0 and up does).

          Edit: one other random thought -- is the BIOS firmware current on these PCEngine APU2x4 units? I did find some posts with a Google search where old firmware in the units can have issues with the most current FreeBSD kernels. Although in the posts I found the symptom was failure to mount the system disk and thus never getting beyond the boot prompt. Yours appear to be getting farther than that, but also do not appear to have actually completed the kernel upgrade.

          S 1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            That pfctl error is because there's a kernel/world mismatch. It's trying to use a version of pfctl against a different version of pf.

            So that implies the upgrade only partially completed. Running pkg-static info -x pfsense should show some mismatched pkgs.

            You can probably just upgrade pkgs to complete the upgrade if they show as available.

            Steve

            1 Reply Last reply Reply Quote 0
            • S
              Stewart @bmeeks
              last edited by

              @bmeeks I'm not sure on the firmware. I had that thought last night. Normally our units are 4.11.0.6. I know the first one was that version. The one I swapped today I'm not so sure. It's relatively old at this point but since all of our devices are remote I'm not too keen on updating that unless I need to. I can't install or uninstall packages. It says there aren't any installed and it can't check for any. If I need to monitor each unit for the upgrade I'd rather just build up a new box in the office and import the config on site and swap them. But that'll be hundreds of miles of driving, maybe more, so I'd like to not go that route.

              @stephenw10 What commands would I run to complete the package upgrades?

              Running pkg-static clean -ay; pkg-static install -fy pkg pfSense-repo pfSense-upgrade appears to do nothing in that it just goes to the next line.

              Updating from the command line gives me:

              Segmentation fault (core dumped)
failed to update the repository settings!!!
failed to update the repository settings!!!

              I'm going to hold off on more upgrades until I can get a handle on these. I have 2 of these failed units with me now so any commands I can run to either fix or get more information on what's happening would be appreciated.

              1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by

                If it's partially upgraded you should run pkg-static upgrade and agree to the offered pkg upgrades.

                S 1 Reply Last reply Reply Quote 0
                • S
                  Stewart @stephenw10
                  last edited by Stewart

                  @stephenw10 Got a lot of projects today so I'm not sure I'll get to it. I'll reply back when I do, though. Thanks for the help on this one.

                  I decided to run these real quick before I head out.

                  [2.6.0-RELEASE][root@pfSense]/root: pkg-static info -x pfsense
[2.6.0-RELEASE][root@pfSense]/root: pkg-static upgrade
[2.6.0-RELEASE][root@pfSense]/root: pkg-static install
[2.6.0-RELEASE][root@pfSense]/root: pkg-static inst
[2.6.0-RELEASE][root@pfSense]/root: pkg-static i
[2.6.0-RELEASE][root@pfSense]/root: pkg-static bootstrap -f

                  All just go on to the next line. Nothing is output to the screen.

                  [2.6.0-RELEASE][root@pfSense]/root: pkg-static ?
pkg-static: No match.

                  This was just to see what would happen. Is pkg-static prt of pkg? Is there a way to force the reinstall of it?

                  1 Reply Last reply Reply Quote 0
                  • stephenw10S
                    stephenw10 Netgate Administrator
                    last edited by

                    Yes, but only if you can access the pkg servers:
                    pkg-static -d upgrade -f pkg

                    S 1 Reply Last reply Reply Quote 0
                    • S
                      Stewart @stephenw10
                      last edited by Stewart

                      @stephenw10 said in Failed update from 2.6.0 -> 2.7.0 causing lots of issues:

                      Yes, but only if you can access the pkg servers:
                      pkg-static -d upgrade -f pkg

                      That just goes to the next line as well.

                      Can I pull something out of /var/cache/pkg ?

                      1 Reply Last reply Reply Quote 0
                      • stephenw10S
                        stephenw10 Netgate Administrator
                        last edited by

                        Yes, you can force a reinstall from there using the full path if there are any cached versions of pkg available.

                        1 Reply Last reply Reply Quote 0
                        • S
                          Stewart
                          last edited by

                          I just wanted to follow up on this and let everyone know I could never get anywhere. While I did backup/restore last month I kept this box around to try to troubleshoot. No pkg command would yield any results, even trying to install from local.

                          If anyone winds up in the same boat, I'm afraid this post likely won't help.

                          1 Reply Last reply Reply Quote 1
                          • First post
                            Last post