2.3.2_1 crash report
-
Does pkg find a problem with either of those?
pkg check -s | & egrep -v '(Checking all packages|missing file)'Also, use sha256 (or md5) rather than just "sum" for a more accurate hash.
-
Does pkg find a problem with either of those?
pkg check -s | & egrep -v '(Checking all packages|missing file)'pkg does not report problems with any packages other than expected missing files.
-
And "goodsys" and "badsys" are both amd64?
-
And "goodsys" and "badsys" are both amd64?
Goodsys is an SG-4860, running factory image. Upgraded with release versions only since install in June 2015. This is the system that was turning the crash report that started this thread, and was fixed by running pkg installs by hand following the upgrade to 2.3.2_1.
Badsys is an SG-2220, running factory image. Upgraded with release versions only since install in September 2015.
-
OK, mostly I wanted to be sure they were running versions that should actually be the same and not different in some way.
-
This should be easy to reproduce: I just pulled a brand new SG-4860 from the box. It came shipped with 2.3.2. I did a config restore with the configuration file download from the old SG-4860. After reboot, I did an upgrade to 2.3.2_1.
This output is from the old SG-4860 (aka goodsys above):
[2.3.2-RELEASE][root@goodsys]/root: pkg which /usr/local/lib/php/20131226/pfSense.so /usr/local/lib/php/20131226/pfSense.so was installed by package php56-pfSense-module-0.12 [2.3.2-RELEASE][root@goodsys]/root: sha1 /usr/local/lib/php/20131226/pfSense.so SHA1 (/usr/local/lib/php/20131226/pfSense.so) = f29b07c11d823ca0b5a75113122abcd1925073fe [2.3.2-RELEASE][root@goodsys]/root: pkg which /usr/local/lib/php/20131226/suhosin.so /usr/local/lib/php/20131226/suhosin.so was installed by package php-suhosin-0.9.38 [2.3.2-RELEASE][root@goodsys]/root: sha1 /usr/local/lib/php/20131226/suhosin.so SHA1 (/usr/local/lib/php/20131226/suhosin.so) = 531f0ffb1b33b18e3ca41a264589355f1f29e46e [2.3.2-RELEASE][root@goodsys]/root:This output is from the new SG-4860 after the upgrade to 2.3.2_1:
[2.3.2-RELEASE][root@newsys]/root: pkg which /usr/local/lib/php/20131226/pfSense.so /usr/local/lib/php/20131226/pfSense.so was installed by package php56-pfSense-module-0.12 [2.3.2-RELEASE][root@newsys]/root: sha1 /usr/local/lib/php/20131226/pfSense.so SHA1 (/usr/local/lib/php/20131226/pfSense.so) = 90eecff835a534b878fecfb2086141632e640448 [2.3.2-RELEASE][root@newsys]/root: pkg which /usr/local/lib/php/20131226/suhosin.so /usr/local/lib/php/20131226/suhosin.so was installed by package php-suhosin-0.9.38 [2.3.2-RELEASE][root@newsys]/root: sha1 /usr/local/lib/php/20131226/suhosin.so SHA1 (/usr/local/lib/php/20131226/suhosin.so) = db27be42a763a4f9386ef4dfc9a72c4076f7a672 [2.3.2-RELEASE][root@newsys]/root:In the /usr/local/lib/php/20131226 directory, the following files were not updated by the 2.3.2_1 upgrade: pfSense.so, xdebug.so, zmq.so, radius.so, suhosin.so, rrd.so, ssh2.so. You can easily see it with ls -lt.
-
Jim? Any update?
-
Packages having minor changes between releases but their revision not explicitly bumped?
-
Packages having minor changes between releases but their revision not explicitly bumped?
The opposite. The package repository shows that the versions have been bumped, but the package files have not been updated.
-
Out of town for a few days, unless someone else can test it in the meantime I'll give it a shot when I'm back on Wednesday
-
Out of town for a few days, unless someone else can test it in the meantime I'll give it a shot when I'm back on Wednesday
Okay, thanks Jim.
-
https://forum.pfsense.org/index.php?topic=112543.0 is this related?
-
The system was updated several times from 2.3 to 2.3.2-p1
[2.3.2-RELEASE][root@pf.net]/root: pkg which /usr/local/lib/php/20131226/pfSense.so /usr/local/lib/php/20131226/pfSense.so was installed by package php56-pfSense-module-0.12 [2.3.2-RELEASE][root@pf.net]/root: sha1 /usr/local/lib/php/20131226/pfSense.so SHA1 (/usr/local/lib/php/20131226/pfSense.so) = 90eecff835a534b878fecfb2086141632e640448 [2.3.2-RELEASE][root@pf.net]/root: pkg which /usr/local/lib/php/20131226/suhosin.so /usr/local/lib/php/20131226/suhosin.so was installed by package php-suhosin-0.9.38 [2.3.2-RELEASE][root@pf.net]/root: sha1 /usr/local/lib/php/20131226/suhosin.so SHA1 (/usr/local/lib/php/20131226/suhosin.so) = 3d2cdcbb696ee37d76885918ce805497f329eb17EDIT: 2.3.2-RELEASE-p1 (amd64)
built on Tue Sep 27 12:13:07 CDT 2016
FreeBSD 10.3-RELEASE-p9 -
Using the 4860 formally known as goodsys, I did the following:
1. Perform a fresh install from factory image for 2.3.2
2. Collect checksums of all files on the system
3. Perform an on-line upgrade to 2.3.2_1
4. Collect checksums of all files on the system
5. Perform a fresh install from factory image for 2.3.2_1
6. Collect checksums of all files on the systemBase on the results of steps 4 & 6, I find the following 85 files to be incorrect:
usr/local/bin/delv usr/local/bin/dig usr/local/bin/host usr/local/bin/nslookup usr/local/bin/nsupdate usr/local/bin/rrdtool usr/local/lib/ipsec/libcharon.a usr/local/lib/ipsec/libradius.a usr/local/lib/ipsec/libsimaka.a usr/local/lib/ipsec/libstrongswan.a usr/local/lib/ipsec/libtls.a usr/local/lib/ipsec/libvici.a usr/local/lib/ipsec/plugins/libstrongswan-addrblock.a usr/local/lib/ipsec/plugins/libstrongswan-aes.a usr/local/lib/ipsec/plugins/libstrongswan-attr.a usr/local/lib/ipsec/plugins/libstrongswan-blowfish.a usr/local/lib/ipsec/plugins/libstrongswan-cmac.a usr/local/lib/ipsec/plugins/libstrongswan-constraints.a usr/local/lib/ipsec/plugins/libstrongswan-curl.a usr/local/lib/ipsec/plugins/libstrongswan-curl.so usr/local/lib/ipsec/plugins/libstrongswan-des.a usr/local/lib/ipsec/plugins/libstrongswan-dnskey.a usr/local/lib/ipsec/plugins/libstrongswan-eap-dynamic.a usr/local/lib/ipsec/plugins/libstrongswan-eap-identity.a usr/local/lib/ipsec/plugins/libstrongswan-eap-md5.a usr/local/lib/ipsec/plugins/libstrongswan-eap-mschapv2.a usr/local/lib/ipsec/plugins/libstrongswan-eap-peap.a usr/local/lib/ipsec/plugins/libstrongswan-eap-radius.a usr/local/lib/ipsec/plugins/libstrongswan-eap-sim-file.a usr/local/lib/ipsec/plugins/libstrongswan-eap-sim.a usr/local/lib/ipsec/plugins/libstrongswan-eap-tls.a usr/local/lib/ipsec/plugins/libstrongswan-eap-ttls.a usr/local/lib/ipsec/plugins/libstrongswan-fips-prf.a usr/local/lib/ipsec/plugins/libstrongswan-hmac.a usr/local/lib/ipsec/plugins/libstrongswan-ipseckey.a usr/local/lib/ipsec/plugins/libstrongswan-kernel-pfkey.a usr/local/lib/ipsec/plugins/libstrongswan-kernel-pfroute.a usr/local/lib/ipsec/plugins/libstrongswan-md4.a usr/local/lib/ipsec/plugins/libstrongswan-md5.a usr/local/lib/ipsec/plugins/libstrongswan-nonce.a usr/local/lib/ipsec/plugins/libstrongswan-openssl.a usr/local/lib/ipsec/plugins/libstrongswan-pem.a usr/local/lib/ipsec/plugins/libstrongswan-pgp.a usr/local/lib/ipsec/plugins/libstrongswan-pkcs1.a usr/local/lib/ipsec/plugins/libstrongswan-pkcs12.a usr/local/lib/ipsec/plugins/libstrongswan-pkcs7.a usr/local/lib/ipsec/plugins/libstrongswan-pkcs8.a usr/local/lib/ipsec/plugins/libstrongswan-pubkey.a usr/local/lib/ipsec/plugins/libstrongswan-random.a usr/local/lib/ipsec/plugins/libstrongswan-rc2.a usr/local/lib/ipsec/plugins/libstrongswan-resolve.a usr/local/lib/ipsec/plugins/libstrongswan-revocation.a usr/local/lib/ipsec/plugins/libstrongswan-sha1.a usr/local/lib/ipsec/plugins/libstrongswan-sha2.a usr/local/lib/ipsec/plugins/libstrongswan-socket-default.a usr/local/lib/ipsec/plugins/libstrongswan-sshkey.a usr/local/lib/ipsec/plugins/libstrongswan-stroke.a usr/local/lib/ipsec/plugins/libstrongswan-unbound.a usr/local/lib/ipsec/plugins/libstrongswan-unity.a usr/local/lib/ipsec/plugins/libstrongswan-updown.a usr/local/lib/ipsec/plugins/libstrongswan-vici.a usr/local/lib/ipsec/plugins/libstrongswan-whitelist.a usr/local/lib/ipsec/plugins/libstrongswan-x509.a usr/local/lib/ipsec/plugins/libstrongswan-xauth-eap.a usr/local/lib/ipsec/plugins/libstrongswan-xauth-generic.a usr/local/lib/ipsec/plugins/libstrongswan-xcbc.a usr/local/lib/libgio-2.0.a usr/local/lib/libglib-2.0.a usr/local/lib/libgmodule-2.0.a usr/local/lib/libgobject-2.0.a usr/local/lib/libgthread-2.0.a usr/local/lib/librrd.a usr/local/lib/php/20131226/pfSense.so usr/local/lib/php/20131226/rrd.so usr/local/lib/php/20131226/suhosin.so usr/local/lib/php/20131226/xdebug.so usr/local/lib/php/20131226/zmq.so usr/local/sbin/dnssec-dsfromkey usr/local/sbin/dnssec-importkey usr/local/sbin/dnssec-keyfromlabel usr/local/sbin/dnssec-keygen usr/local/sbin/dnssec-revoke usr/local/sbin/dnssec-settime usr/local/sbin/dnssec-signzone usr/local/sbin/dnssec-verifyUntil this is addressed with a new release, I don't think one should trust anything short of a full install.
-
Bug report filed: https://redmine.pfsense.org/issues/6858
-
Thanks for the bug report. Assigned, bumped priority.
Will get this looked at early tomorrow.
-
Thanks Jim. Much appreciated.
-
Actually it's not a bug, it's expected and it's how pkg is designed to work.
When we moved to 2.3.2_1 we cherry-picked some package upgrades from FreeBSD ports tree since these upgrades fixes some vulnerabilities listed by 'pkg audit'. Following ports were updated:
php56
perl5
libxml2
libidn
curlDue to these updates, when poudriere starts to build our ports set, it deletes all packages that depends of above listed packages and rebuild them. At this time, strongswan was rebuilt as many other ports, and a new package with same version was created.
This new package was included in 2.3.2-p1, so when you install it directly you will see the new package, and checksum differs.
During upgrade, since any shared library version has bumped, pkg understands packages like strongswan don't need to be reinstalled, because libraries it depends didn't have any ABI changes. Then you end up with the version built for 2.3.2.
If you compare the built date for strongswan package on both systems you will see this:
- 2.3.2
[2.3.2-RELEASE][admin@pf232.home]/root: pkg info strongswan strongswan-5.5.0 Name : strongswan Version : 5.5.0 Installed on : Wed Jul 20 15:39:17 2016 UTC Origin : security/strongswan Architecture : freebsd:10:x86:64 Prefix : /usr/local Categories : security Licenses : GPLv2 Maintainer : strongswan@nanoteq.com WWW : http://www.strongswan.org Comment : Open Source IKEv2 IPsec-based VPN solution- 2.3.2-p1
[2.3.2-RELEASE][admin@pfs232-1.home]/root: pkg info strongswan strongswan-5.5.0 Name : strongswan Version : 5.5.0 Installed on : Mon Oct 17 23:14:33 2016 UTC Origin : security/strongswan Architecture : freebsd:10:x86:64 Prefix : /usr/local Categories : security Licenses : GPLv2 Maintainer : strongswan@nanoteq.com WWW : http://www.strongswan.org Comment : Open Source IKEv2 IPsec-based VPN solution -
Renato, thank you for the write up.
Does this cover file /usr/local/lib/php/20131226/suhosin.so? This shared object is the one that triggered the original crash report. It is different in 2.3.2_1 install vs upgrade. Was package php-suhosin-0.9.38 also part of the cherry picking?
Is there another explanation for the crash report?
-
Renato, thank you for the write up.
Does this cover file /usr/local/lib/php/20131226/suhosin.so? This shared object is the one that triggered the original crash report. It is different in 2.3.2_1 install vs upgrade. Was package php-suhosin-0.9.38 also part of the cherry picking?
Is there another explanation for the crash report?
php-suhosin was rebuilt because lang/php56 was updated. But the original issue is a problem we faced sometimes on pfSense 2.3. In some hard-to-reproduce circumstances extensions.ini ended up out of order and suhosin was loaded before session, that is a dependency.
The real fix for this was done in FreeBSD ports and is already on 2.4 and 2.3.3 snapshots. It was reworked the way PHP extensions create their config files to make sure they are loaded in correct order.
