Error going from 2.3 to 2.3.1_1
-
Hi all,
Having an issue updating one of my 5 boxes to 2.3.1_1, on the dashboard it says: 'unable to check for updates', the actual System Updages page gives me: 'Unable to retrieve system versions.'
SSH in and use option 13, returns this:
PHP: syntax error, unexpected '(' in /usr/local/etc/php/extensions.ini on line 2
/usr/local/sbin/pkg: Shared object has no run-time symbol table
/usr/local/sbin/pkg: Shared object has no run-time symbol table
ERROR: It was not possible to identify which pfSense kernel is installed
PHP: syntax error, unexpected '(' in /usr/local/etc/php/extensions.ini on line 2
PHP: syntax error, unexpected '(' in /usr/local/etc/php/extensions.ini on line 2The update settings page is even stranger, instead of Stable and Dev, it shows 'FreeBSD: {enable', and 'FreeBSD: {enabled:no'
Any ideas?
-
There's something screwy in your /usr/local/etc/php/extensions.ini
What are the contents of that file?
-
ok, file was full of garbage, copied it over from a working unit, now I get the same on webconfigurator, and the below on SSH using 13.
/usr/local/sbin/pkg: Shared object has no run-time symbol table
/usr/local/sbin/pkg: Shared object has no run-time symbol table
ERROR: It was not possible to identify which pfSense kernel is installed/usr/local/sbin/pkg is not readable on this system, nor a working one.
- M
-
pkg is a compiled binary, not something that would be legible. But that system that isn't working clearly has a variety of issues. Sounds like it corrupted a number of files (which might be indicative of a hardware problem). I'd get a config backup from that system, reinstall it clean, and restore the backup. And be wary of the system given that level of weird file corruption.
-
You already identified yourself the real problem:
@Marc:ok, file was full of garbage, copied it over ….
I propose that you run
fsck
and you will see what up really : file system damaged …. your disk is probably dying (they so do all the time)./usr/local/sbin/pkg is not readable on this system, nor a working one.
What do you mean by not readable ?
'pkg' is a program and as such, before it exectues in memory, it has to be read into memory first.
It's just a file, and is readable.Proof :
[2.3.1-RELEASE][admin@pfsense.brit-hotel-fumel.net]/etc/inc: hexdump /usr/local/sbin/pkg | more
0000000 457f 464c 0102 0901 0000 0000 0000 0000
0000010 0002 003e 0001 0000 6340 0040 0000 0000
0000020 0040 0000 0000 0000 a320 0002 0000 0000
0000030 0000 0000 0040 0038 0008 0040 001c 001b
0000040 0006 0000 0005 0000 0040 0000 0000 0000
0000050 0040 0040 0000 0000 0040 0040 0000 0000
0000060 01c0 0000 0000 0000 01c0 0000 0000 0000
0000070 0008 0000 0000 0000 0003 0000 0004 0000
0000080 0200 0000 0000 0000 0200 0040 0000 0000
0000090 0200 0040 0000 0000 0015 0000 0000 0000
00000a0 0015 0000 0000 0000 0001 0000 0000 0000
00000b0 0001 0000 0005 0000 0000 0000 0000 0000
00000c0 0000 0040 0000 0000 0000 0040 0000 0000
00000d0 8a5c 0002 0000 0000 8a5c 0002 0000 0000
00000e0 0000 0020 0000 0000 0001 0000 0006 0000
00000f0 9000 0002 0000 0000 9000 0062 0000 0000
0000100 9000 0062 0000 0000 1068 0000 0000 0000
0000110 2758 0000 0000 0000 0000 0020 0000 0000
0000120 0002 0000 0006 0000 9028 0002 0000 0000
0000130 9028 0062 0000 0000 9028 0062 0000 0000
0000140 0270 0000 0000 0000 0270 0000 0000 0000
0000150 0008 0000 0000 0000 0004 0000 0004 0000
0000160 0218 0000 0000 0000 0218 0040 0000 0000
…...The message
/usr/local/sbin/pkg: Shared object has no run-time symbol table
means : file f*cked up.So : check your drive.
-
I'm surprisingly happy with that, it's the only 32bit machine in the fleet. 3 are actual pfSense SG appliances, and my home unit is a VM running on my vsphere host. Might be that final kick to get rid of that old machine.
-
I've had a couple of issues trying to upgrade my pfSense installation from 2.3 to 2.3.1_1 on Hyper V that sound similar to this issue.
At first, my original installation would simply break when the update was applied in a completely repeatable manner (restore the VM from backup, rinse, repeat). Basically, I would update using the standard update feature in the UI, which would download, install, then reboot the system. At that point it'd install a couple more things, then continue to boot up the point where it said "Configuring WAN interface…" and immediately reboot. At that point it would then display a bunch of things after the boot prompt and "BTX halted" (as attached).
At that point, I thought that it'd be a good idea to simply start fresh, so I made a config backup in the web interface and created a whole new VM and installed 2.3.1 on it from scratch. What I found was that I could successfully update it to 2.3.1_1 after making the minimal configuration required to get it onto the Internet, but if I tried to restore my config.xml file prior to upgrading, then I'd have issues during the upgrade process similar to the above (with the same "crash" at "Configuring WAN interface…" and reboot after installation), although it would at least attempt to boot the system, but with various issues (which seemed to change each time I tried it). For example, here is an excerpt of the console I captured while booting one of these attempted upgrades:
KDB: current backend: ddb Copyright (c) 1992-2016 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016 root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense amd64 FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 CPU: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz (1664.84-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306a9 Family=0x6 Model=0x3a Stepping=9 Features=0x1f83fbff <fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,mmx,fxsr,sse,sse2,ss,htt>Features2=0xfe982203 <sse3,pclmulqdq,ssse3,cx16,sse4.1,sse4.2,popcnt,aesni,xsave,osxsave,avx,f16c,rdrand,hv>AMD Features=0x20100800 <syscall,nx,lm>AMD Features2=0x1 <lahf>Structured Extended Features=0x200 <erms>XSAVE Features=0x1 <xsaveopt>Hypervisor: Origin = "Microsoft Hv" real memory = 536870912 (512 MB) avail memory = 478343168 (456 MB) ... SMP: AP CPU #1 Launched! Timecounter "Hyper-V" frequency 10000000 Hz quality 10000000 Event timer "HyperV" frequency 10000000 Hz quality 1000 storvsc0 on vmbus0 storvsc1 on vmbus0 da0 at blkvsc0 bus 0 scbus1 target 0 lun 0 hyperv-utils0 on vmbus0 hyperv-utils0: Hyper-V Service attaching: Hyper-V Heartbeat Service da0: hyperv-utils1 on vmbus0 hyperv-utils1: Hyper-V Service attaching: Hyper-V KVP Service <msft virtual="" disk="" 1.0="">Fixed Direct Access SPC-2 SCSI device da0: 300.000MB/s transfershyperv-utils2 on vmbus0 hyperv-utils2: Hyper-V Service attaching: Hyper-V Shutdown Service hyperv-utils3 on vmbus0 hyperv-utils3: Hyper-V Service attaching: Hyper-V Time Synch Service da0: Command Queueing enabled hn0: <synthetic network="" interface=""> on vmbus0 da0: 5120MB (10485760 512 byte sectors) hn0: unknown status 1073872902 received hn0: hv send offload request succeeded hn0: Using defaults for TSO: 65518/35/2048 hn0: Ethernet address: 00:15:5d:00:04:17 hn1: <synthetic network="" interface=""> on vmbus0 hn1: unknown status 1073872902 received hn1: hv send offload request succeeded hn1: Using defaults for TSO: 65518/35/2048 hn1: Ethernet address: 00:15:5d:00:04:18 hn2: <synthetic network="" interface=""> on vmbus0 hn2: unknown status 1073872902 received hn2: hv send offload request succeeded hn2: Using defaults for TSO: 65518/35/2048 hn2: Ethernet address: 00:15:5d:00:04:19 hn3: <synthetic network="" interface=""> on vmbus0 hn3: unknown status 1073872902 received hn3: hv send offload request succeeded hn3: Using defaults for TSO: 65518/35/2048 hn3: Ethernet address: 00:15:5d:00:04:1a Trying to mount root from ufs:/dev/ufsid/574a68fd984d1bc6 [rw]... May 29 15:48:35 init: login_getclass: unknown class 'daemon' May 29 15:48:35 init: login_getclass: no default/fallback class 'default' Configuring crash dumps... Filesystems are clean, continuing... Mounting filesystems... rm: /conf: Read-only file system ln: /conf/conf: Read-only file system ___ ___/ f \ / p \___/ Sense \___/ \ \___/ Welcome to 2.3.1-RELEASE on the 'pfSense' platform... rm: /COPYRIGHT: Read-only file system rm: /bin/cat: Read-only file system ... /etc/rc: cannot create /dev/null: Operation not supported Creating symlinks...ln: /tmp/tmp: Read-only file system rm: /tmp/config.cache: Read-only file system rm: /tmp/config.lock: Read-only file system rm: /tmp/mnt/cf: Read-only file system rm: /tmp/mnt: Read-only file system rm: /tmp/php_errors.txt: Read-only file system ..cp: /dev/null: No such file or directory ./etc/rc: cannot create /dev/null: Operation not supported done. /etc/rc: /usr/local/sbin/-upgrade: not found /etc/rc: cannot create /tmp/php_errors.txt: Read-only file system /etc/rc: cannot create /dev/null: Operation not supported Launching the init system...rm: /cf/conf/backup/backup.cache: Read-only file system done. Initializing.................. done. Error: cannot open dmesg.boot in system_dmesg_save(). cannot create /dev/null: Operation not supported cannot create /dev/null: Operation not supported Starting device manager (devd)...done. Loading configuration.....PHP Fatal error: Call to undefined function xml_parser_create() in /etc/inc/xmlparse.inc on line 205 Fatal error: Call to undefined function xml_parser_create() in /etc/inc/xmlparse.inc on line 205 PHP ERROR: Type: 1, File: /etc/inc/xmlparse.inc, Line: 205, Message: Call to undefined function xml_parser_create()Starting CRON... /etc/rc: cannot create /dev/null: Operation not supported done. fcgicli: Could not connect to server(/var/run/php-fpm.socket). /etc/rc: /usr/local/sbin/-upgrade: not found (pfSense) 2.3.1-RELEASE Tue May 17 18:46:53 CDT 2016 Bootup complete /etc/rc: cannot create /dev/null: Operation not supported rm: /tmp/config.cache: Read-only file system</synthetic></synthetic></synthetic></synthetic></msft></xsaveopt></erms></lahf></syscall,nx,lm></sse3,pclmulqdq,ssse3,cx16,sse4.1,sse4.2,popcnt,aesni,xsave,osxsave,avx,f16c,rdrand,hv></fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,mmx,fxsr,sse,sse2,ss,htt>In this particular case, it seems it's trying to rm -rf itself, but it hasn't mounted the filesystem in r/w mode in order to do that.
My next attempt (i.e. start again with a fresh install, then restore config.xml, then try upgrade) did this on boot (which looks similar to the OP's behavior where it's trying to parse an invalid file):
start_init: trying /sbin/init May 29 19:07:12 init: login_getclass: unknown class 'daemon' May 29 19:07:12 init: login_getclass: no default/fallback class 'default' Configuring crash dumps... Using /dev/label/swap0 for dump device. Filesystems are clean, continuing... Mounting filesystems... [: -eq: unexpected operator Welcome to (Patch ) on the '' platform... Dump device does not exist. Savecore not run. Creating symlinks......done. /etc/rc: /usr/local/sbin/-upgrade: not found /etc/rc: cannot create /tmp/php_errors.txt: Read-only file system pid 89 (php-fpm), uid 0: exited on signal 11 Segmentation fault fcgicli: Could not connect to server(/var/run/php-fpm.socket). Launching the init system...pid 94 (php-cgi), uid 0: exited on signal 11 Segmentation fault Starting CRON... done. fcgicli: Could not connect to server(/var/run/php-fpm.socket). /etc/rc: /usr/local/sbin/-upgrade: not found () (Patch ) Bootup complete[/code] I did boot this machine back up off the installation disc and found that most of the files in /sbin and /bin were 0 bytes in length - even fsck and ls were not usable in this state. If I restored my config.xml file [i]after[/i] performing the upgrade (which I did try multiple times to see if it was definitely not a fluke), the upgrade itself would always work correctly (and it wasn't crashing at the "Configuring WAN interface..." step of the boot process either). Since this has worked, I haven't really worried about it and most of the testing I did was more from a curiosity point of view, since my main gateway was working again after I did it the aforementioned way. In my case, the disk isn't a suspect as this disk is used by both other VMs and the host for storing data; and the consistency at which I can replicate the issue by restoring my config.xml, [i]then[/i] upgrading.  
