Getting crash reports after updating to 2.4.4

nesense · Sep 25, 2018, 11:54 AM

Crash report begins.  Anonymous machine information:

amd64
11.2-RELEASE-p3
FreeBSD 11.2-RELEASE-p3 #17 e6b497fa0a3(RELENG_2_4_4): Thu Sep 20 09:04:45 EDT 2018     root@buildbot3:/crossbuild/ce-244/obj/amd64/WvDslnYb/crossbuild/ce-244/pfSense/tmp/FreeBSD-src/sys/pfSense

Crash report details:

PHP Errors:
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20170718/session.so' - /usr/local/lib/php/20170718/session.so: Undefined symbol "zend_empty_string" in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: bcmath: Unable to initialize module
Module compiled with module API=20170718
PHP    compiled with module API=20131226
These options need to match
 in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: bz2: Unable to initialize module
Module compiled with module API=20170718
PHP    compiled with module API=20131226
These options need to match
 in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: ctype: Unable to initialize module
Module compiled with module API=20170718
PHP    compiled with module API=20131226
These options need to match
 in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20170718/curl.so' - /usr/local/lib/php/20170718/curl.so: Undefined symbol "zend_empty_string" in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20170718/dom.so' - /usr/local/lib/php/20170718/dom.so: Undefined symbol "zend_ce_countable" in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20170718/filter.so' - /usr/local/lib/php/20170718/filter.so: Undefined symbol "zend_empty_string" in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: gettext: Unable to initialize module
Module compiled with module API=20170718
PHP    compiled with module API=20131226
These options need to match
 in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20170718/hash.so' - /usr/local/lib/php/20170718/hash.so: Undefined symbol "zend_ce_error" in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20170718/intl.so' - /usr/local/lib/php/20170718/intl.so: Undefined symbol "zend_empty_string" in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20170718/json.so' - /usr/local/lib/php/20170718/json.so: Undefined symbol "zend_empty_string" in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20170718/ldap.so' - /usr/local/lib/php/20170718/ldap.so: Undefined symbol "zend_empty_string" in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20170718/mbstring.so' - /usr/local/lib/php/20170718/mbstring.so: Undefined symbol "zend_empty_string" in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: mcrypt: Unable to initialize module
Module compiled with module API=20170718
PHP    compiled with module API=20131226
These options need to match
 in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: openssl: Unable to initialize module
Module compiled with module API=20170718
PHP    compiled with module API=20131226
These options need to match
 in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20170718/pcntl.so' - /usr/local/lib/php/20170718/pcntl.so: Undefined symbol "zend_interrupt_function" in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20170718/pdo.so' - /usr/local/lib/php/20170718/pdo.so: Undefined symbol "zend_empty_string" in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: pfSense: Unable to initialize module
Module compiled with module API=20170718
PHP    compiled with module API=20131226
These options need to match
 in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: posix: Unable to initialize module
Module compiled with module API=20170718
PHP    compiled with module API=20131226
These options need to match
 in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20170718/radius.so' - /usr/local/lib/php/20170718/radius.so: Undefined symbol "zend_empty_string" in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: readline: Unable to initialize module
Module compiled with module API=20170718
PHP    compiled with module API=20131226
These options need to match
 in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: rrd: Unable to initialize module
Module compiled with module API=20170718
PHP    compiled with module API=20131226
These options need to match
 in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: shmop: Unable to initialize module
Module compiled with module API=20170718
PHP    compiled with module API=20131226
These options need to match
 in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20170718/simplexml.so' - /usr/local/lib/php/20170718/simplexml.so: Undefined symbol "zend_ce_countable" in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20170718/sockets.so' - /usr/local/lib/php/20170718/sockets.so: Undefined symbol "zend_empty_string" in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20170718/sqlite3.so' - /usr/local/lib/php/20170718/sqlite3.so: Undefined symbol "zend_empty_string" in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: sysvmsg: Unable to initialize module
Module compiled with module API=20170718
PHP    compiled with module API=20131226
These options need to match
 in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: sysvsem: Unable to initialize module
Module compiled with module API=20170718
PHP    compiled with module API=20131226
These options need to match
 in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: sysvshm: Unable to initialize module
Module compiled with module API=20170718
PHP    compiled with module API=20131226
These options need to match
 in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20170718/tokenizer.so' - /usr/local/lib/php/20170718/tokenizer.so: Undefined symbol "zend_one_char_string" in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: xml: Unable to initialize module
Module compiled with module API=20170718
PHP    compiled with module API=20131226
These options need to match
 in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20170718/xmlwriter.so' - /usr/local/lib/php/20170718/xmlwriter.so: Undefined symbol "zend_empty_string" in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20170718/zlib.so' - /usr/local/lib/php/20170718/zlib.so: Undefined symbol "zend_empty_string" in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: zmq: Unable to initialize module
Module compiled with module API=20170718
PHP    compiled with module API=20131226
These options need to match
 in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20170718/pdo_sqlite.so' - /usr/local/lib/php/20170718/pdo_sqlite.so: Undefined symbol "zend_empty_string" in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/local/lib/php/20170718/xmlreader.so' - /usr/local/lib/php/20170718/xmlreader.so: Undefined symbol "zend_empty_string" in Unknown on line 0
[25-Sep-2018 12:45:17 Asia/Beirut] PHP Fatal error:  Call to undefined function gettext() in /etc/inc/certs.inc on line 46


No FreeBSD crash data found.

Packages installed: mailreport, Squid, pfblockerng, notes, service_watchdog

jimp · Sep 25, 2018, 1:25 PM

Those look like the typical errors you see during an upgrade. If you delete that report, do the same error messages come back?

nesense · Sep 25, 2018, 1:30 PM

@jimp they come back if I reboot or if WAN goes down and up again.

jimp · Sep 25, 2018, 1:35 PM

The exact same set of errors?

If so, it looks like maybe the upgrade did not complete. It's complaining about PHP module versions not matching, which I've only ever seen happen in the middle of an upgrade, not before or after repeatedly.

nesense · Sep 25, 2018, 2:32 PM

Yes, same errors everytime, everything seems to be functioning normally though, but should I reinstall freshly from 2.4.4 to be safe?

jimp · Sep 25, 2018, 2:35 PM

Before doing that, try running pfSense-upgrade -d from an ssh or console shell prompt to see if it attempts to correct the issues.

nesense · Sep 25, 2018, 2:45 PM

Here's the output of pfSense-upgrade -d:

>>> Updating repositories metadata...
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
Fetching meta.txz: . done
Fetching packagesite.txz: .......... done
Processing entries: .......... done
pfSense repository update completed. 499 packages processed.
All repositories are up to date.
>>> Upgrading pfSense-upgrade...
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
        pfSense-upgrade: 0.52 -> 0.53 [pfSense]

Number of packages to be upgraded: 1

13 KiB to be downloaded.
[1/1] Fetching pfSense-upgrade-0.53.txz: .. done
Checking integrity... done (0 conflicting)
[1/1] Upgrading pfSense-upgrade from 0.52 to 0.53...
[1/1] Extracting pfSense-upgrade-0.53: ..... done
>>> Setting vital flag on lang/php72...
>>> Updating repositories metadata...
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
Your packages are up to date

I rebooted after and got the another exact crash report.

jimp · Sep 25, 2018, 3:34 PM

Nothing showing there. What is the output of pkg info?

jwt · Sep 25, 2018, 3:34 PM

What does php -v output?

% php -v
PHP 7.2.10 (cli) (built: Sep 14 2018 11:38:12) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
with Zend OPcache v7.2.10, Copyright (c) 1999-2018, by Zend Technologies]

nesense · Sep 25, 2018, 4:08 PM

I already reinstalled.

wdup · Sep 25, 2018, 7:08 PM

Experiencing the same issue - had pfBlockerNG-devel installed prior to upgrade from 2.4.3-p1 to 2.4.4. Didn't appear to have any problems during upgrade. Have since uninstalled pfBlockerNG-devel, issue remains.

TheNarc · Sep 25, 2018, 7:20 PM

Same issue here, exact same set of error messages. Output from php -v:

PHP 7.2.10 (cli) (built: Sep 14 2018 11:32:18) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.2.10, Copyright (c) 1999-2018, by Zend Technologies

I too had pfBlockerNG-devel installed prior to the upgrade. I haven't tried uninstalling it, but I have noticed that the pfb_filter service isn't running, and if I attempt to start it manually, I can provoke the crash report.

nesense · Sep 25, 2018, 7:24 PM

I had pfblockerng-devel installed before but I uninstalled it and reinstalled the stable package before updating.

REM · Sep 26, 2018, 6:11 PM

I have same issue0_1537985459724_pfsence.txt

jimp · Sep 26, 2018, 6:16 PM

@rem said in Getting crash reports after updating to 2.4.4:

I have same issue0_1537985459724_pfsence.txt

Your upgrade did not complete. It's still referencing old PHP 5.6 items. If you have WAN connectivity, go to an ssh or console shell and run pfSense-upgrade again.

REM · Sep 26, 2018, 6:56 PM

Thank you, this message after first reboot, if in console i have:
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
Your packages are up to date

all ok, safety use?

jimp · Sep 26, 2018, 7:02 PM

If you still receive those errors after a reboot, then something must not be OK. If they went away, then it's fine.

If you still have problems, go through some of the suggestions here: https://www.netgate.com/docs/pfsense/install/upgrade-troubleshooting.html

Ramosel · Sep 26, 2018, 7:10 PM

@wdupreez said in Getting crash reports after updating to 2.4.4:

Experiencing the same issue - had pfBlockerNG-devel installed prior to upgrade from 2.4.3-p1 to 2.4.4. Didn't appear to have any problems during upgrade. Have since uninstalled pfBlockerNG-devel, issue remains.

Seeing the EXACT same thing as you. Working with Anthony but it seems to go deeper than pfBNG.

Ramosel · Oct 1, 2018, 9:31 PM

@thenarc said in Getting crash reports after updating to 2.4.4:

Same issue here, exact same set of error messages. Output from php -v:
PHP 7.2.10 (cli) (built: Sep 14 2018 11:32:18) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.2.10, Copyright (c) 1999-2018, by Zend Technologies
I too had pfBlockerNG-devel installed prior to the upgrade. I haven't tried uninstalling it, but I have noticed that the pfb_filter service isn't running, and if I attempt to start it manually, I can provoke the crash report.

Same issue here.

Ramosel · Sep 26, 2018, 8:26 PM

@jimp said in Getting crash reports after updating to 2.4.4:

@rem said in Getting crash reports after updating to 2.4.4:

I have same issue0_1537985459724_pfsence.txt

Your upgrade did not complete. It's still referencing old PHP 5.6 items. If you have WAN connectivity, go to an ssh or console shell and run pfSense-upgrade again.

Jim,
I too am seeing the same errors. I uninstalled all packages rebooted, then did the "pkg-static clean -ay; pkg-static install -fy pkg pfSense-repo pfSense-upgrade" from the link you sent. System came up clean reinstalled packages as well as the pfBlockerNG devel from Anthony and as soon as I try to start pfb_filter service it provokes a crash report full of PHP errors.