Problems upgrading from 2.4.5 to 2.5.0

powerextreme

I attempted the upgrade from 2.4.5 with my backup config restored. I consoled into the APU and was able to watch the process. It looked like the system upgraded completed but I cannot access it via the webgui.
Typically the console starts up with Screen Shot 2021-04-10 at 8.58.38 AM.png

However, when the upgrade finished it showed up with:
Screen Shot 2021-04-10 at 9.00.56 AM.png

Any thoughts? If I do a fresh 2.4.5 install and upgrade it to 2.5.0 it works. It seems like problems occur when I try to restore my config.

powerextreme

@powerextreme

I reboot the device and these are the messages after pfsense logo appears:

Screen Shot 2021-04-10 at 9.06.03 AM.png

Any thoughts?

palesius

I'm having EXACTLY the same issue.
Upgrade from 2.4.5 to 21.01 (which I believe is the pfSense Plus counterpart to 2.5.0).

Same messages about "cannot unset string offsets" and ending with a # prompt, but the firewall doesn't seem to actually come up.

SteveITS

It's starting packages...are you uninstalling packages before upgrade as recommended?

powerextreme

@steveits
Steve, thanks for the response. I don't understand. Prior to upgrading I am to uninstall all the packages I use?

palesius

@powerextreme YMMV but this worked for me (having a similar issue):

find the line:

unset($ph1ent['reauth_enable'], $ph1ent['rekey_enable'], $ph1ent['margintime'], $ph1ent['over_time']);

in /etc/inc/upgrade_config.inc

and replace with

		if (isset($ph1ent['reauth_enable'])) {
			unset($ph1ent['reauth_enable']);
		}
		if (isset($ph1ent['rekey_enable'])) {
			unset($ph1ent['rekey_enable']);
		}
		if (isset($ph1ent['margintime'])) {
			unset($ph1ent['margintime']);
		}
		if (isset($ph1ent['over_time'])) {
			unset($ph1ent['over_time']);
		}

PHP isn't my 1st language, or even my 2nd/3rd, so there is undoubtedly a more concise way of doing this, but it got it past the upgrade and working again.

Basically it is now only doing the unset if it is actually necessary for each of those variables.

SteveITS

@powerextreme Yes, uninstall packages first. I’m not saying that is the cause of your problems, just that it’s recommended. Packages I’ve used expect this and have a checkbox (default on) to keep settings after an uninstall.

powerextreme

@steveits
So I reinstalled 2.4.5 and then restored my config. I see these messages in the console:

Screen Shot 2021-04-11 at 12.20.36 PM.png

I got a message saying that if re-installing packages is still taking a few hours then do a 'clear package lock'.

I went to check packages and pfblockerNG and an explanation mark next to it. I removed it. But the package removal screen says "Please wait while the update system initializes" and hangs there.

Seems to be an issue with packages. I can't really remove them...well at least pfblockerNG.

SteveITS

@powerextreme not sure about the messages. However did you change to “previous stable version” before installing packages (see my sig).

Gertjan

@powerextreme said in Problems upgrading from 2.4.5 to 2.5.0:

So I reinstalled 2.4.5

That insures a clean setup.

@powerextreme said in Problems upgrading from 2.4.5 to 2.5.0:

and then restored my config.

And that is the one that is good for a :

@powerextreme said in Problems upgrading from 2.4.5 to 2.5.0:

I got a message saying that if re-installing packages is still taking a few hours then do a 'clear package lock'.

Normally, it goes much faster then that.
But what about this method :
Before you load your own config, think, and ask yourself : "what packages had I installed ?". If you can't remember, look in your config.xml file : it lists them all.
Now, before doing anything else, install this list manually, one by one. No need to set them up.
When they are all installed, now load you config, and have it reboot.

You system comes up, totally ready.

powerextreme

@gertjan @SteveITS

I am still having issues getting it to successfully upgrade. I did

list fresh install of 2.4.5.
Set the 'Settings->Update->Update Settings to Previous Stable Version (2.4.5 Deprecated)
Installed my packages (NTOP, pfBlockerNG, Iperf) and restored my config.
[This setup worked once but not subsequent time]
Upgraded to 2.5.1. It would boot up and then end at the command prompt. The GUI was unavailable.

My second idea was to install a fresh 2.5.1. and then install all those packages before restoring my config.
I did the following:

Set the 'Settings->Update->Update Settings to Previous Stable Version (2.4.5 Deprecated)
Went to install Ntop and go an error:

"WARNING: Current pkg repository has a new pHP major version. pfSense should be upgraded before installing any new Package"

Could this a cause of my issues with upgrading? Is there a way to remove the references to these packages in my config files so they don't try to install them?

powerextreme

I also tried a fresh install of 2.5.1 and installed all the packages. It worked fine. But when I restored my config it looks like it came up on the console but I can't access the GUI. The DHCP server isn't up. Packet captures show no responses to DHCP queries. The WAN isn't obtaining an address (via DHCP).

I am really perplexed.

SteveITS

@powerextreme One must install packages for the version of pfSense in use. Netgate recommends uninstalling packages before upgrade, then reinstall. Many have an option to save or delete settings when uninstalling. It should work to install fresh and restore your configuration though.

palesius

@powerextreme Have you tried the changes to /etc/inc/upgrade_config.inc I suggested earlier?

If you are seeing the Fatal error: Uncaught Error message, that may fix it.
I had the same issue and no extra packages, and it fixed it for me. While it could be an issue with a package as well, it certainly looks like the same problem I was having, which had nothing to do with my packages.

Simplest way would be to load 2.5.1 without your config (just use the default and get to the point where you can access it from the lan).

download /etc/inc/upgrade_config.inc
from diagnostics -> command line
make your edits locally
upload the changed file from that same page.
run something like:

mv /tmp/upgrade_config.inc /etc/inc/

I think when you upload it that will be the path, I know it definitely tells you where it was uploaded once you upload, so if it didn't go into /tmp then you would need to change that command a tiny bit.

powerextreme

@palesius

I tried your fix and it worked! Thanks a lot for that!
I have two more remote pfsense boxes at 2.4.5. Will I have to do the same thing for these two boxes?

What did I do wrong? I was told is hold have removed all packages before an upgrade. Does this include point releases? On my own pfsense I have 2.5.0 and its asking to upgrade to 2.5.1. Should I uninstall all my packages for this upgrade?

SteveITS

@powerextreme said in Problems upgrading from 2.4.5 to 2.5.0:

removed all packages before an upgrade. Does this include point releases?

I asked this question before a few years ago and got some conflicting answers as I recall. I generally have uninstalled/reinstalled packages, to be safe.

hockeyfreak

I think I've got most problems solved for my setup. First off, I'm not a Linux guy (trying to learn it). I'm not an .XML guy (learned more than I wanted to know). I upgraded to 2.5.1 after a couple of years of pfSense just running flawlessly other than occasional tweaking, so I've forgotten most of what I did when I put it all together. I have about a hundred smart devices, phones, computers, cameras, Unifi infrastructure, and media players. I have them all segregated on 4 VLANs (.10 LAN, .20 Cameras, .30 IoT, .40 Filtered, .50 Unfiltered), and for the most part, everything works. I was terrified when all kinds of issues described above started happening, and I realized I hit "Update" before backing up config or reading up on uninstalling packages. Rebuilding this whole thing would have been monumental (for me).

I ended up getting on a path of resetting to defaults of 2.5.1 instead of installing 2.4.5 again and praying my 2.5.1 config.xml I backed up after upgrading would load properly in 2.4.5. I edited the .XML file and removed everything between <installedpackages> and </installedpackages>, leaving it looking like <installedpackages></installedpackages> and saved it. I reset pfSense to default, plugged directly into it, went through the wizard, and then restored the config.xml. It took a couple of tries to get the .XML right, but it finally restored.

The results are the DHCP list now loads, although a lot slower than before. I still get unable to check for updates on the dashboard, and the System->Update takes forever to open, then fails the check there also. The DHCP leases list does not have any devices in there that have no Hostname, even though I know I've had devices in there before I never identified and couldn't assign them static IPs and type in hostnames (I've been trying to identify on my network lately).

I plugged back into my switch and put everything back to normal to see if it would find and assign DHCP IPs to those unknown devices again. It took a long time to refresh the list, like 2-3 minutes of spinning, None of the devices without hostnames reappeared. That will actually make it easier to figure out what they were now, because they will no longer work. But how do I get pfSense to give them dynamic IPs if it won't assign them without hostnames?

So is this an official bug, or just hits people that didn't upgrade properly and have to muddle through? Thanks for ANY advice (besides backup before upgrading....)

Gertjan

@hockeyfreak

I'll start with the rather famous joke :
"Windows" is not Linux is not MACos is not FreeBSD.
(althought MACos is based on FreeBSD)
So, remove Linux from your list, look at 'FreeBSD' documenation ^^
And keep in mind : no FreeBSD knowledge is needed to operate a router/firewall. Normally, the network admin doesn't care/ doesn't want to know/doesn't understand what OS is used to run the router.

Removing packages before upgrade :
Not needed, and I can proof that easily : I never did so over more then 10 years.
I use several very light weight packages like Notes, Cron, ShellCmd, Avahi, acme, openvpn-client-export. All these offer extend functionality, and can not - do not influence the basic usage of the router. It's like putting an image on your desktop's background : that won't stop Windows from working.

Even a huge one like pfBlockerNG (latest version) can be disabled with one click. When unchecked, the entire "pfBlockerNG " code stops.
FreeRadius : same thing.

So, no need to edit the backed up config file. I even advise you not doing so.
One example : you removed <installedpackages>boatload</installedpackages>
But, one of these packages could also have created Firewall aliases and or firewall rules. These are now based on non initialized data after reboot. You just created a totally unprecedented situation. No way that everything works well now. No manual or user experience can guide you here.
So : make your live easy : don't.

He is what I do :
Save one copy of the config.
Get an USB drive ready with the current version of pfSense (so you can get back in a snap - and having the drive prepared in your hands activated an extension of Murphy's law : you won't need it).
Now upgrade.
Works fine for me for the last decade or so.

DHCP :
The DHCP status page : nice ..... but that's a page you look at when all goes well, etc.
For DHCP to work (the most stable process on planet earth) interfaces need to be set up correctly.
Then you set up / check every DHCP server (one for each interface == LAN == VLAN).
Done.
Checking afterwards is done with : the DHCP logs. The log pages are the most important info pages on the system.
You can see DHCP requests coming in, and answers from the server going back to the network clients.

Be aware :

@hockeyfreak said in Problems upgrading from 2.4.5 to 2.5.0:

I plugged back into my switch

the interface down to up event is seen by pfSense, on it's VLAN interface, and the switch. Not the devices that are hooked up behind it. These device keep 'thinking' that they have a valid lease, and renew it when it times out, not before.
That's why you don't see much of a traffic as soon as you hook up your system like hat.

Try this ( ) : shut down the power every where. Shut down manually devices that are battery powered.
Start pfSense.
Now, activate the power on ALL your devices.
You'll see a DHCP storm, guaranteed.

SteveITS

@hockeyfreak I've seen other posts about DHCP status page being slow but haven't paid much attention. Note devices will only show if they have a lease so they probably won't all show right after router restart. (not sure offhand if the leases survive a restart)

@Gertjan raises a good point about aliases disappearing. If you are remote make sure you will have access to the router if a pfBlocker alias will become nonexistent and thus a NAT or firewall rule not apply.

re: removing packages, per Netgate, "The safest practice is to remove all packages before upgrading to a new release. The upgrade process will handle packages automatically, but packages are frequently a source of problems. To ensure a smooth upgrade, note the installed packages, remove them, perform the upgrade, and then reinstall when the upgrade is complete." So not saying it won't or can't work, just that that is their advice. Or, if an update ever doesn't work try removing packages.

hockeyfreak

@gertjan

So, remove Linux from your list, look at 'FreeBSD' documentation

I really didn't mean I was trying to learn Linux, that was a general term I guess. I know there are tons of other flavors out there, and I'm trying to narrow down what I want to start focusing on learning.

So, no need to edit the backed up config file. I even advise you not doing so.

I tried reloading it with the backup and all my problems were still present. That's when I decided to edit it. The only thing that really doesn't work anymore is my OpenVPN connection, and I can set that up again when I find a good tutorial (again). I just don't do this stuff enough for it to stick in my mind over the years. I guess I should start keeping a word doc process list for all these tasks I do so I can go back and recreate them if something happens. I usually just relearn everything through tuts

Get an USB drive ready with the current version of pfSense

Do you mean the config.xml on the USB stick, or the whole install of pfSense, or both? I'm using a FW4B for my pfSense box. Probably overkill, but I wanted a stand alone appliance to also emlimate my Frontier router.

DHCP :

Thanks for that explanation. I kind of understood it, but I've never really looked at the logs. Maybe that will help isolate what those blank hostname devices are. I've tried MAC searching, checking the hundreds of apps needed to run smart home for identifying MAC/IP address, etc. Some just refuse to be identified until everything is on a static in pfSense and identified, then when something is installed it's obvious. I should have been doing this from the start...

Try this ( )

I would have to time that perfectly with all the comings and goings around here. Or send the family on a vaca without me! I can live with a few unidentified for now. Something will happen to help me identify them eventually. I just put "Unknown1" in hostname and then check it now and then. Maybe seeing where it's communicating with in Pihole might help?

As for the update with pfSense, after settling in from 4am until noon, everything is loading faster now, and the update is able to check status. Maybe it just needed a settle in time? I followed this tut part 1 and 2 on setting up Pihole with DNS filtering (other bad tutorials are what started this whole mess) and everything seems to be working properly. The fog of DNS is starting to lift, but not totally. I did a backup before implementing all his strategies though!! Lesson learned.

Thanks for all your attention and guidance!!