Attempting update from 2.4.4_3 to 21.02 hangs at "Please wait while the update system initializes"

pzanga

I just tried updating the SG-1100 at my office, going from 2.4.4_3 to 21.02 and system update hangs at the "Please wait while the update system initializes" message. After about 10 minutes I navigate back to the dashboard and everything is fine. Tried this 2 times so far. Prior to the update attempt I uninstalled pfblockerng, with the intent of installing pfblockerng-devel after the upgrade. Only other issue I have noted in last couple of days is that package manager does not display any available packages, which I am thinking may be related to the new release being available??
I don't see any posts referring to this upgrade path and don't want to try anything until I see what people have to say. I am wondering if I need to upgrade from the 2.4.5 deprecated branch first, then upgrade to 21.02? Other than that not sure of next steps.
Not sure what other info about my current configuration may be helpful. Just let me know and I will post it here. Thanks ahead of time for any and all assistance.

pzanga

Here is some more info which may or may not be helpful. (tried to add this to the original post but didn't realize you can only edit for 1 hour)

<edit> One other thing I just remembered - the SG-1100 unit itself was not/is not indicating an update is available (circle and diamond LEDs both solid green). The diamond had been flashing when the 2.4.5 upgrade was available. I can explain why we didn't install that upgrade if that helps at all, but, suffice to say, we didn't.

The pfblockerng version I uninstalled was 2.1.4_17. The other packages that are still installed are
current packages.png
Snort is enabled and Suricata is not.

We have a single WAN and single LAN interface, no VLANS, no VPN. We have a couple of port forwards set up for our chat server and terminal server. SG-1100 is in the DMZ of our Comcast gateway.

I did look at the system log from last night and nothing stands out to me other than there is no system log entries for the period between when I started each update attempt (and automatic restore point created) and when I next logged out/logged in (see below). Not sure if that is expected and not sure what other logs I might need to be checking.

           Feb 24 19:43:21 pfSense check_reload_status: Syncing firewall
           
           Feb 24 19:43:21 pfSense php-fpm[368]: /pkg_mgr_install.php: Beginning configuration backup to .https://acb.netgate.com/save
           
           Feb 24 19:43:23 pfSense php-fpm[368]: /pkg_mgr_install.php: End of configuration backup to https://acb.netgate.com/save (success).
           
           Feb 24 19:57:45 pfSense php-fpm[366]: /index.php: User logged out for user 'admin' from: 192.168.0.203 (Local Database)
           
           Feb 24 19:58:07 pfSense php-fpm[366]: /index.php: Successful login for user 'admin' from: 192.168.0.203 (Local Database)
           
           Feb 24 19:58:35 pfSense check_reload_status: Syncing firewall
           
           Feb 24 19:58:35 pfSense php-fpm[368]: /pkg_mgr_install.php: Beginning configuration backup to .https://acb.netgate.com/save
           
           Feb 24 19:58:37 pfSense php-fpm[368]: /pkg_mgr_install.php: End of configuration backup to https://acb.netgate.com/save (success).
           
           Feb 24 21:29:30 pfSense php-fpm[63868]: /index.php: User logged out for user 'admin' from: 192.168.0.203 (Local Database)

So, any thoughts/ideas? Any other logs I should be checking? Again, any help is greatly appreciated.

<edit> Just checked conf/upgrade_log and all I see is this. Again, not sure what to make of it.

>>> Updating repositories metadata... failed.

pzanga

Okay, I hope this isn't poor forum etiquette, to keep replying to my own post, but I have had some time to do a deeper dive into the forums and my logs and have some more info and questions.

So, when I tried to update, in System/Update/System Update I had the branch set to latest stable version (21.02.x) as below
update 21.02.png

And the system hangs as described.

When I check conf/upgrade_log, with that branch config, I see the following:

  >>> Updating repositories metadata... failed.

I have now changed the branch in System Update to Previous stable version (2.4.5 deprecated), but have not attempted to upgrade again.

update 2.4.5 deprecated.png

And when I look at conf/upgrade_log now it says

   2.4.5_1 version of pfSense is available

It makes me think I could and probably should at least upgrade to 2.4.5_1 and that the upgrade to 21.02 may then work. But I would really like to get some confirmation that that may work (or not). I don't plan on making a reattempt until the weekend, so I will keep researching and await any responses.
Thanks again.

bennyc

@pzanga
I just stumbled into the same situation, and what does one do first? Read the forum

I had a 2.4.4_3 on my spare APU, so it is a CE 2.5.0 I'm after. But I also had to upgrade 1st to 2.4.5_1 (previous stable branch) as direct to 2.5 didn't work (stays in the screen "please wait while the update system initializes" as you described)
So I changed the Branch, and hit the confirm button. After the system got to 2.4.5_1, had to change the Branch again so it could detect the 2.5 (sounds logic), and upgraded again to latest.

The good news is that (at least for me) this has worked out fine, it has become a 2-step upgrade but it managed to update to 2.5.0 (ymmv)

pzanga

@bennyc
Thanks for the input. I figured there had to be someone out there in a similar situation. I was going to go ahead and attempt the 2.4.4_3 --> 2.4.5_1 --> 21.02 path anyway today, but seeing it worked for you gave me some confidence.
I was able to get to 2.4.5_1. I changed the branch and 21.02 showed as available update, including the diamond light on the box flashing. Clicked confirm to start the upgrade, got past the "please wait..." message, and the update failed right after package 42/185 downloaded
Final line of the error message was

Child process pid=54998 terminated abnormally: Segmentation fault

Time to check logs and search the forums again.

bigsy

@pzanga Might it not be easier to make sure you've a backup of your config and then just do a fresh install of 21.02p1, not an upgrade, particularly the multiple-step process you're describing?

I've found a reinstall to be an easier and quicker process than a major upgrade on the SG-1100. Your system can't have much free resources with the various packages you've installed as it's pretty limited on RAM.

pzanga

@bigsy
That may be the way I go, although I am on 2.4.5 now, so only 1 step left. I may try to upgrade via console first and see how that goes, then do a fresh install if that fails.
As for the packages, the only one on that list we are using is Snort. Not sure why the first 2 are on that list, since we don't use any VPN at this time (I didn't do the initial config), and we aren't using Suricata. I'll likely delete those, and just add pfblockerng-devel (to replace the pfblockerng that was there).

Thanks

roncbk

@pzanga I had the same issue as you - update would hang going from 2.4.4_x to 21.02. I first upgraded to 2.4.5_x and then I was able to successfully upgrade to 21.02.

scottlindner

@pzanga I just encountered the same problem dusting off an old SG-2220 that I hadn't used in nearly two years (due to divorce). After doing the factory reset I couldn't get any packages and I speculated it was due to not being able to do a successful update. I had already done this but came here to post this as the solution and here it was.. the solution was right here for me all along.

Just wanted to drop the note here this is a very valid path and that it is still relevant.

itinfo

Thanks for the insight with this problem. I thought it was me.

Does anyone from Netgate/PFsense follow these forums? I see a lot of good problem solving for something that should not be a problem. If 21.02 is released and 2.4.5 is listed as Deprecated, then the implied assumption is the programmers want you to go from what ever you are on to the latest version. With the absence of some explicit instructions stating you have to be at 2.4.5 to get to 21.02 we, the users, assume we can go from 2.4.3 or 2.4.4 etc. to 21.02. I would think Netgate/PFsense would want to know this issue and make the upgrade process steps more transparent.

My thoughts

scottlindner

@itinfo This isn't a programmer thing as much as it is a business decision. I am an engineer that does a lot of software development in recent years and upgrading from a version to a version is always a big discussion topic. The simplest answer is always to do all upgrades in the path. So if you are ten versions behind you'll need to upgrade ten times. The complexity in skipping a version is possible but expensive.

A good question to ask is, why are the release channels in the UI limited to previous, present and next? I think that kinda makes sense to me just to keep the UI manageable.

So the only remaining question is why the software doesn't identify this issue and inform the user they are in this situation and insteuct the user for a common solution to it? I think I know why. How many users ever find themselves so far behind in updates? Still, this took some effort to sort out. It did for me at least.

itinfo

Well stated, and as you noted, the 'line' for an upgrade is not readily available, not obvious, and as noted, not encouraged (Deprecated). I agree that the path 'SHOULD' be A to B to C, etc. however that journey should not be one of Braille, but rather of intent with the proper instructions and resources readily available. I am a software developer and as a rule I try my best to prevent an 'Easter Egg Hunt' whether it be with using my software or upgrading, Et. Al.

Thus, as noted, I am curious as to why Netgate/PFsense is not on this thread to glean some insight into a very well navigated problem solving labyrinth. I think they could incorporate some of the good advice and intent into their processes with little effort.

My thoughts

scottlindner

I think it's a matter of what the lone should be. It appears the lone is present date Previous/Current/Next rather than system installed Previous/Current/Next. Having stated it like this, it is easier to implement.

I have to deal with these sorts of logical issues daily so it is easier for me to rationalize.

Why Netgate isn't on this thread. Who says they aren't? There may already be an issue on the backlog already for this. A growing (or growth minded) company will actively asses these communities. A stagnant or company in sustain their business will only look at the big issues. A company in decline will completely ignore it.

JeGr

Also please don't forget that skipping multiple versions is always a "bigger" problem/endeavour as just upgrading to the next version. Re-Installing with config-recovery in most cases is the far better alternative then simply updating and updating again. As the Plus devices got ZFS as filesystem relatively late in the game with 21.05, I'd always recommend to everyone that has the ability to be on site with the device to just take the latest stable version (21.05.2 or 22.01 if you're adventurous) and do a fresh install with config-recovery. That is available on the console prior to selecting a new install with either UFS/ZFS. Just let it find & recover the current config (or save it yourself) and do a new install with ZFS of the current CE/Plus release of your choice and afterwards watch it boot up the first time with your configuration intact and your interface setup already in place.

Had no problem with that with variuos customers that "skipped a few" updates "because the system was OK and just ran..." you know the kind