Upgrade 2.3.5 to 2.4.3 ... crash report

ColinJack

I have updated from 2.3.5 to 2.4.3 and am now getting a crash report regularly:

amd64
11.1-RELEASE-p10
FreeBSD 11.1-RELEASE-p10 #13 r313908+293707af843(RELENG_2_4): Thu May 10 15:09:24 CDT 2018 root@buildbot2.netgate.com:/builder/ce-243/tmp/obj/builder/ce-243/tmp/FreeBSD-src/sys/pfSense

Crash report details:

PHP Errors:
[06-Jun-2018 15:05:32 Europe/London] PHP Warning: PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20131226/rrd.so' - Shared object "librrd.so.8" not found, required by "rrd.so" in Unknown on line 0
[06-Jun-2018 15:05:36 Europe/London] PHP Warning: PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20131226/rrd.so' - Shared object "librrd.so.8" not found, required by "rrd.so" in Unknown on line 0

Thoughts.

jimp

Sounds like maybe the rrdtool package didn't upgrade or isn't present.

Try running pkg upgrade -f rrdtool, if it claims the package is not installed or gives a different error, then do pkg install -f rrdtool.

You can also check the output of pkg upgrade -n and see if anything else didn't get fully updated. It's possible it's stuck in between part of the upgrade, another reboot may fix it, or you might need some other manual corrections depending on the output of that command.

ColinJack

Ahaa ..

pkg: Warning: Major OS version upgrade detected. Running "pkg-static install -f pkg" recommended
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
pkg: rrdtool is not installed, therefore upgrade is impossible
Checking integrity... done (0 conflicting)
Your packages are up to date.

If I run pkg install -f rrdtool it then comes up with a proceed y/n but I cannot input the y. :)
Do I have to SSH onto the box or can I add the y in the initial command.
Sorry for the noob question - haven't worked with pfSense command from GUI before.

Thanks

jimp

That would need to be done from the CLI (ssh or console), not from the GUI.

Also it's a bit worrying that it is printing the first message as that would seem to indicate that it didn't actually upgrade pkg itself first, so it's possible other parts didn't get upgraded either.

First, connect to ssh or the console and use option 8 to get to a shell prompt, then run:

# pkg-static clean -ya; pkg-static install -yf pkg pfSense-repo pfSense-upgrade
# pfSense-upgrade -d

And see what that prints.

ColinJack

@jimp said in Upgrade 2.3.5 to 2.4.3 ... crash report:

pfSense-upgrade -d

Okay - I won't print it all out unless you wnat it but:

pfSense-upgrade -d

Warning: PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20131226/rrd.so' - Shared object "librrd.so.8" not found, required by "rrd.so" in Unknown on line 0

Updating repositories metadata...
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
Setting vital flag on pkg...

The following 96 package(s) will be affected (of 0 checked):

Installed packages to be REMOVED:
php56-gettext-5.6.35
php56-curl-5.6.35
pfSense-Status_Monitoring-1.6.5
pfSense-2.3.5_2
libzmq4-4.2.3
glib-2.50.3_2,1
dnsmasq-2.79,1
bind-tools-9.12.1

New packages to be INSTALLED:
python2: 2_3 [pfSense]
devcpu-data: 1.16 [pfSense]
php-suhosin: 0.9.38_3 [pfSense]
pear-Crypt_CHAP: 1.5.0 [pfSense]
pear: 1.10.5_1 [pfSense]
openvpn: 2.4.4_1 [pfSense]
liblz4: 1.8.0,1 [pfSense]
pecl-radius: 1.4.0.b1 [pfSense]
pear-Net_IPv6: 1.3.0.b2_2 [pfSense]
pear-Net_Growl: 2.7.0 [pfSense]
pear-Auth_RADIUS: 1.1.0 [pfSense]
mobile-broadband-provider-info: 20170310 [pfSense]
pear-Mail: 1.4.1,1 [pfSense]
pear-Net_SMTP: 1.8.0 [pfSense]

Is thiat enough to give you a clue?
That is followed by:

Number of packages to be removed: 8
Number of packages to be installed: 15
Number of packages to be upgraded: 10
Number of packages to be reinstalled: 60
Number of packages to be downgraded: 3

The operation will free 45 MiB.
Thanks
59 MiB to be downloaded.

**** WARNING ****
Reboot will be required!!
Proceed with upgrade? (y/N)

Thanks

jimp

Yeah that looks like it's lodged halfway between versions. I'd tell pfSense-upgrade to proceed and see where it lands. Before you do that, take a backup and download a 2.4.3 installer just to be safe.

If all else fails, you can reinstall 2.4.3 and recover the configuration and be back up in a few minutes.

ColinJack

@jimp

That is my concern ... the pfSense is in Dallas, Texas (a VM) and I am in Norfolk UK ... if it goes bang I'm stuck! :(
Gut feeling? Should be okay? I have the config backed up.
Worst case I could get the datacentre guys to upload a vanilla 2.4.3 for me.

jimp

Yeah that might be tricky. As it is, the inconsistent state will most likely lead to a failure in some way, though. I'd proceed with the upgrade but have the datacenter crew prepped with an installer image in case. During the install they can choose the Recover config.xml option and then proceed through the installation and it will retain your current settings. That will only take a few minutes to complete and avoids having to do anything special to restore the config.

ColinJack

Thanks - that was my thoughts also.
I have asked to datacentre guys to confirm they can put the pieces together if required - just waiting for their reply. :)

ColinJack

Hi jimp,

Okay - tried the update from the CLI but still failed. This error on boot:

Warning: PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20131226/rrd.so' - Shared object "librrd.so.8" not found, required by "rrd.so" in Unknown on line 0

Thanks

Colin

ColinJack

Any thoughts. Presume I just need to install the missing bits?

Thanks

jimp

What was the output from pfSense-upgrade? Is the output of pkg upgrade -n still the same?

You might be able to force an update of everything with something like pkg unlock pfSense-kernel-pfSense; pkg upgrade -f.

I'd still be prepared for a reinstallation, though.

ColinJack

Thanks jimp,

Got the datacentre guys on standby (with a config file). :)

When I run pfsense-upgrade /d it all appears to upgrade okay and then reboots and throws the error:
Warning: PHP Startup: Unable to load dynamic library ‘/usr/local/lib/php/20131226/rrd.so’ - Shared object “librrd.so.8” not found, required by “rrd.so” in Unknown on line 0
Am I not able to just install the missing library?

Thanks

Colin

jimp

@colinjack said in Upgrade 2.3.5 to 2.4.3 ... crash report:

Am I not able to just install the missing library?

I gave the command for that up at the start of this thread. But what worries me is if other things are also missing or failing to upgrade, there may be a deeper issue on the system as a whole.

ColinJack

@jimp said in Upgrade 2.3.5 to 2.4.3 ... crash report:

Sorry - you did:
Try running pkg upgrade -f rrdtool, if it claims the package is not installed or gives a different error, then do pkg install -f rrdtool.

I think your worries are valid but I agree with you that it needs to be sorted because there may be underlying problems which will surface later.

I will have another go and then see what happens. As I said, I have the datacentre crew on standby!

Thanks for your help on this.

ColinJack

@colinjack said in Upgrade 2.3.5 to 2.4.3 ... crash report:

Hmmm ...

[2.3.5-RELEASE][root@firewall]/root: pkg upgrade -f rrdtool
Shared object "libssl.so.8" not found, required by "pkg"
[2.3.5-RELEASE][root@firewall]/root: pkg install -f rrdtool
Shared object "libssl.so.8" not found, required by "pkg"
[2.3.5-RELEASE][root@firewall]/rootpkg upgrade -n
Shared object "libssl.so.8" not found, required by "pkg"

So forcing upgrade will also throw the pkg error?

jimp

Use pkg-static instead of pkg in that case, but that is another sign that your system is still stuck between 2.3.x and 2.4.x.

ColinJack

[2.3.5-RELEASE][root@firewall]/root: pkg-static upgrade -f rrdtool
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
Fetching meta.txz: 100% 944 B 0.9kB/s 00:01
Fetching packagesite.txz: 100% 131 KiB 134.3kB/s 00:01
Processing entries: 100%
pfSense repository update completed. 498 packages processed.
All repositories are up to date.
pkg-static: rrdtool is not installed, therefore upgrade is impossible
Checking integrity... done (0 conflicting)
Your packages are up to date.

ColinJack

I am loathe to post the full output from 'pkg-static upgrade /n' (bad forum etiquette) but would that be useful?
Can I stick it a text file and attach it to my post?

jimp

You can attach a text file to a post. But it's also OK if you wrap it in code tags (select it all, hit the code button in the composer, or put ``` before and after the block.)