ERROR: It was not possible to identify which pfSense kernel is installed
-
Latest update from 170106_1956, I remember.
Boot message kernel version is missed, then update unable…
... Starting syslog...done. Starting CRON... done. pkg: No package(s) matching pfSense-kernel-* ERROR: It was not possible to identify which pfSense kernel is installed pfSense 2.4.0-BETA amd64 Sat Dec 31 11:18:03 CST 2016 // this is the previous one installed (as update (ZFS)) Bootup complete FreeBSD/amd64 (apu2b2.localdomain) (ttyu0) *** Welcome to pfSense 2.4.0-BETA (amd64) on apu2b2 *** ...System is running OK, and no updated version info and:
System > Update:
Current Base System [empty] Latest Base System [empty] Retrieving [red]Unable to check for updates[/red]How to recover from the inability to update ?
update via console (13) no result
reboot: the option [K] old.kernel, no result.So, Reinstall the image huh?
-
You could reinstall, or manually reinstall the missing pkg items. What does "pkg info" show, if you haven't already reinstalled?
-
You could reinstall, or manually reinstall the missing pkg items. What does "pkg info" show, if you haven't already reinstalled?
Yes, still running 20161231 OK.
No output:
[2.4.0-BETA][root@apu2b2.localdomain]/root: pkg info [2.4.0-BETA][root@apu2b2.localdomain]/root:I updated with 20170106 and that solved the issue of redmine #6906
That last reboot I remember answering 'Y' on the override question.
Did that screw-up my install update ? [it is on ZFS & mSata] -
It's possible it did. Turn off RAM disks and see if your pkg info list is back. If so, then you might be safe in activating the option again from that point.
But if you turn off RAM disks and they don't come back, you'll have to reinstall
-
RAM disks was already turned off since my beginning of 2.4B on ZFS.
(at that time I had to, otherwise was no ability to update via GUI)So… I will reinstall when THE 2.4 comes out. Anyway thanks for your response.
-
Actually you could reinstall the packages to get back to a good state:
pkg install pfSense pfSense-base pfSense-default-config pfSense-kernel-pfSense pfSense-rc pfSense-repo pfSense-upgrade -
So what's up with this pkg db corruption? Only happens with RAM disks?
-
Until a week or so ago there were some corner cases like ZFS where the difference in how/when the disk was mounted stopped the code from working as expected.
On current snaps it should be OK all-around.
-
OK, thanks. ;)
-
Actually you could reinstall the packages to get back to a good state:
[2.4.0-BETA][root@apu2b2.localdomain]/root: pkg install pfSense pfSense-base pfSense-default-config pfSense-kernel-pfSense pfSense-rc pfSense-repo pfSense-upgrade Updating pfSense-core repository catalogue... Fetching meta.txz: 100% 940 B 0.9kB/s 00:01 Fetching packagesite.txz: 100% 2 KiB 1.8kB/s 00:01 Processing entries: 100% pfSense-core repository update completed. 7 packages processed. Updating pfSense repository catalogue... Fetching meta.txz: 100% 940 B 0.9kB/s 00:01 Fetching packagesite.txz: 100% 120 KiB 122.6kB/s 00:01 Processing entries: 100% pfSense repository update completed. 441 packages processed. The following 149 package(s) will be affected (of 0 checked): New packages to be INSTALLED: pfSense: 2.4.0.b.20170116.1330 [pfSense] pfSense-base: 2.4.0.b.20170116.1536 [pfSense-core] pfSense-default-config: 2.4.0.b.20170116.1536 [pfSense-core] pfSense-kernel-pfSense: 2.4.0.b.20170116.1536 [pfSense-core] pfSense-rc: 2.4.0.b.20170116.1536 [pfSense-core] pfSense-repo: 2.4.0.b.20170116.1330 [pfSense] pfSense-upgrade: 0.12 [pfSense] libxml2: 2.9.4 [pfSense] clog: 1.0.1 [pfSense] dpinger: 2.0 [pfSense] php56-pdo_sqlite: 5.6.29 [pfSense] php56-pdo: 5.6.29 [pfSense] php56: 5.6.29 [pfSense] pcre: 8.39_1 [pfSense] sqlite3: 3.15.1_1 [pfSense] readline: 6.3.8 [pfSense] indexinfo: 0.2.6 [pfSense] expiretable: 0.6_1 [pfSense] libltdl: 2.4.6 [pfSense] smartmontools: 6.5_1 [pfSense] php56-mbstring: 5.6.29 [pfSense] oniguruma5: 5.9.6_1 [pfSense] ipmitool: 1.8.17_1 [pfSense] php56-json: 5.6.29 [pfSense] php56-sysvshm: 5.6.29 [pfSense] pecl-radius: 1.4.0.b1 [pfSense] php56-bz2: 5.6.29 [pfSense] isc-dhcp43-server: 4.3.5 [pfSense] beep: 1.0_1 [pfSense] pear-Net_Growl: 2.7.0 [pfSense] pear: 1.10.1 [pfSense] php56-zlib: 5.6.29 [pfSense] php56-xml: 5.6.29 [pfSense] bwi-firmware-kmod: 3.130.20 [pfSense] ca_root_nss: 3.28.1 [pfSense] mpd5: 5.8 [pfSense] php56-hash: 5.6.29 [pfSense] mobile-broadband-provider-info: 20151214 [pfSense] bind-tools: 9.11.0P2 [pfSense] idnkit: 1.0_6 [pfSense] json-c: 0.12.1 [pfSense] libedit: 3.1.20150325_2,1 [pfSense] php56-mcrypt: 5.6.29 [pfSense] libmcrypt: 2.5.8_3 [pfSense] pear-XML_RPC2: 1.1.3 [pfSense] php56-curl: 5.6.29 [pfSense] curl: 7.52.1_1 [pfSense] libnghttp2: 1.18.0 [pfSense] pear-Cache_Lite: 1.7.16,1 [pfSense] pear-HTTP_Request2: 2.3.0,1 [pfSense] pear-Net_URL2: 2.2.1 [pfSense] pear-Mail: 1.3.0,1 [pfSense] pear-Net_SMTP: 1.7.2 [pfSense] pear-Net_Socket: 1.0.14 [pfSense] check_reload_status: 0.0.8 [pfSense] libevent2: 2.0.22_1 [pfSense] ataidle: 2.7.2 [pfSense] php56-tokenizer: 5.6.29 [pfSense] minicron: 0.0.2 [pfSense] dhcp6: 20080615.1 [pfSense] php56-ctype: 5.6.29 [pfSense] openvpn: 2.4.0 [pfSense] lzo2: 2.09 [pfSense] liblz4: 131 [pfSense] ntp: 4.2.8p9_1 [pfSense] libwww: 5.4.0_6 [pfSense] gettext-runtime: 0.19.8.1_1 [pfSense] pecl-zmq: 1.1.3_2 [pfSense] libzmq4: 4.1.5 [pfSense] norm: 1.5r6 [pfSense] libsodium: 1.0.11_1 [pfSense] ssh_tunnel_shell: 0.1_1 [pfSense] php56-simplexml: 5.6.29 [pfSense] cpdup: 1.18 [pfSense] pear-Net_IPv6: 1.3.0.b2_2 [pfSense] miniupnpd: 1.9.20160113,1 [pfSense] pecl-ssh2: 0.13 [pfSense] libssh2: 1.8.0,2 [pfSense] sshlockout_pf: 0.0.2 [pfSense] pfSense-Status_Monitoring: 1.6.0_2 [pfSense] pecl-rrd: 1.1.3_4 [pfSense] rrdtool: 1.6.0_1 [pfSense] perl5: 5.24.1.r4_1 [pfSense] glib: 2.46.2_4 [pfSense] python27: 2.7.13_1 [pfSense] libffi: 3.2.1 [pfSense] libiconv: 1.14_10 [pfSense] pear-Auth_RADIUS: 1.1.0 [pfSense] radvd: 1.9.1 [pfSense] libdaemon: 0.14_1 [pfSense] unbound: 1.6.0 [pfSense] expat: 2.2.0_1 [pfSense] ldns: 1.6.17_5 [pfSense] isc-dhcp43-relay: 4.3.5 [pfSense] dhcpleases: 0.3_1 [pfSense] php56-opcache: 5.6.29 [pfSense] php56-dom: 5.6.29 [pfSense] gogoc: 1.2_1 [pfSense] wol: 0.7.1_3 [pfSense] php56-xmlwriter: 5.6.29 [pfSense] choparp: 20150613 [pfSense] nginx: 1.10.2_3,2 [pfSense] luajit: 2.0.4_1 [pfSense] dnsmasq: 2.76,1 [pfSense] nettle: 3.3 [pfSense] gmp: 5.1.3_3 [pfSense] libidn: 1.33_1 [pfSense] pftop: 0.7_7 [pfSense] strongswan: 5.5.1 [pfSense] vstr: 1.0.15_1 [pfSense] php56-sockets: 5.6.29 [pfSense] php56-filter: 5.6.29 [pfSense] voucher: 0.1_2 [pfSense] filterdns: 1.0_15 [pfSense] isc-dhcp43-client: 4.3.5 [pfSense] php56-pfSense-module: 0.32 [pfSense] links: 2.13,1 [pfSense] php56-sysvsem: 5.6.29 [pfSense] php56-readline: 5.6.29 [pfSense] filterlog: 0.1_5 [pfSense] scponly: 4.8.20110526_2 [pfSense] qstats: 0.2 [pfSense] php56-posix: 5.6.29 [pfSense] igmpproxy: 0.1_5,1 [pfSense] php56-openssl: 5.6.29 [pfSense] php56-gettext: 5.6.29 [pfSense] php-xdebug: 2.4.1_1 [pfSense] xinetd: 2.3.15_2 [pfSense] bsnmp-regex: 0.6_1 [pfSense] php56-ldap: 5.6.29 [pfSense] openldap-client: 2.4.44 [pfSense] uclcmd: 0.1_1 [pfSense] libucl: 0.8.0 [pfSense] wrapalixresetbutton: 0.0.7 [pfSense] php56-pcntl: 5.6.29 [pfSense] rate: 0.9_1 [pfSense] php56-session: 5.6.29 [pfSense] php56-sysvmsg: 5.6.29 [pfSense] php56-shmop: 5.6.29 [pfSense] php56-xmlreader: 5.6.29 [pfSense] bsnmp-ucd: 0.4.2 [pfSense] dhcpleases6: 0.1_2 [pfSense] pear-Crypt_CHAP: 1.5.0 [pfSense] dmidecode: 3.0 [pfSense] php56-sqlite3: 5.6.29 [pfSense] relayd: 5.5.20140810_2 [pfSense] cpustats: 0.1_1 [pfSense] php-suhosin: 0.9.38_3 [pfSense] php56-bcmath: 5.6.29 [pfSense] Number of packages to be installed: 149 The process will require 449 MiB more space. 125 MiB to be downloaded. Proceed with this action? [y/N]: yNo dice ?
Already more than 15min in this state. No I/O. CPU 25%, Load ~ 1.4, Mem. normal 14%. What's going on ?ADD: I aborted after 30min. Ctrl-C => signal received, cleaning up. Back at prompt.
ADD:
PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 53727 root 1 103 0 47888K 14804K CPU1 1 2:39 100.00% pkg 27942 root 1 23 0 264M 29224K piperd 2 0:00 1.76% php-fpm 47711 root 1 52 20 13048K 2384K wait 2 0:42 0.20% sh .... -
Might be that some pkg file is partially corrupted? Can you try (im assuming the name of the pool is zroot):
zpool list zpool scrub zroot zpool status -v zrootKeep running the last command until its done scanning the disk.
-
[2.4.0-BETA][root@apu2b2.localdomain]/root: zpool list NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT zroot 12.9G 470M 12.4G - 6% 3% 1.00x ONLINE - [2.4.0-BETA][root@apu2b2.localdomain]/root: zpool scrub zroot [2.4.0-BETA][root@apu2b2.localdomain]/root: [2.4.0-BETA][root@apu2b2.localdomain]/root: zpool status -v zroot pool: zroot state: ONLINE scan: scrub repaired 0 in 0h0m with 0 errors on Tue Jan 17 20:55:02 2017 config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 ada0p3 ONLINE 0 0 0 errors: No known data errors -
Try logging what its doing, might show that it fails to read (part of) some file? After a minute or two check the logfile possibly showing lots of commands repeating/hanging at the end of the log.. The try and find what if any file it was busy with at that time.?.
truss -o /root/truss_pkg_upgrade.log pkg upgradeNot sure if this is the best way, but it did help me find some problem files once..
-
[2.4.0-BETA][root@apu2b2.localdomain]/root: truss -o /root/truss_pkg_upgrade.log pkg upgrade Updating pfSense-core repository catalogue... Fetching meta.txz: 100% 940 B 0.9kB/s 00:01 Fetching packagesite.txz: 100% 2 KiB 1.8kB/s 00:01 Processing entries: 100% pfSense-core repository update completed. 7 packages processed. Updating pfSense repository catalogue... Fetching meta.txz: 100% 940 B 0.9kB/s 00:01 Fetching packagesite.txz: 100% 120 KiB 122.6kB/s 00:01 Processing entries: 100% pfSense repository update completed. 441 packages processed. Checking for upgrades (0 candidates): 100% Processing candidates (0 candidates): 100% Checking integrity... done (0 conflicting) Your packages are up to date. [2.4.0-BETA][root@apu2b2.localdomain]/root:A lot of "No such file or directory" in the log file i.e.
[2.4.0-BETA][root@apu2b2.localdomain]/root: tail -40 truss_pkg_upgrade.log access("/var/db/pkg/local.sqlite-journal",F_OK) ERR#2 'No such file or directory' fstat(12,{ mode=-rw-r--r-- ,inode=20540,size=109568,blksize=109568 }) = 0 (0x0) pread(0xc,0x7fffffffd900,0x10,0x18) = 16 (0x10) fstat(12,{ mode=-rw-r--r-- ,inode=20540,size=109568,blksize=109568 }) = 0 (0x0) access("/var/db/pkg/local.sqlite-wal",F_OK) ERR#2 'No such file or directory' fstat(12,{ mode=-rw-r--r-- ,inode=20540,size=109568,blksize=109568 }) = 0 (0x0) fcntl(12,F_SETLK,0x7fffffffd8d0) = 0 (0x0) stat("/var/db/pkg/local.sqlite",{ mode=-rw-r--r-- ,inode=20540,size=109568,blksize=109568 }) = 0 (0x0) getpid() = 86979 (0x153c3) stat("/var/db/pkg/local.sqlite",{ mode=-rw-r--r-- ,inode=20540,size=109568,blksize=109568 }) = 0 (0x0) openat(AT_FDCWD,"/var/db/pkg/local.sqlite-journal",O_RDWR|O_CREAT|O_CLOEXEC,0644) = 17 (0x11) fstat(17,{ mode=-rw-r--r-- ,inode=21580,size=0,blksize=131072 }) = 0 (0x0) geteuid() = 0 (0x0) fchown(17,0,0) = 0 (0x0) fstat(17,{ mode=-rw-r--r-- ,inode=21580,size=0,blksize=131072 }) = 0 (0x0) pwrite(0x11,0x8031d7908,0x200,0x0) = 512 (0x200) pwrite(0x11,0x7fffffffd914,0x4,0x200) = 4 (0x4) pwrite(0x11,0x803319b08,0x400,0x204) = 1024 (0x400) pwrite(0x11,0x7fffffffd914,0x4,0x604) = 4 (0x4) fcntl(12,F_SETLK,0x7fffffffd8c0) = 0 (0x0) fcntl(12,F_SETLK,0x7fffffffd8c0) = 0 (0x0) pwrite(0x11,0x7fffffffd8a4,0x4,0x608) = 4 (0x4) pwrite(0x11,0x803325d08,0x400,0x60c) = 1024 (0x400) pwrite(0x11,0x7fffffffd8a4,0x4,0xa0c) = 4 (0x4) pread(0x11,0x7fffffffd898,0x8,0xc00) = 0 (0x0) pwrite(0x11,0x7fffffffd888,0xc,0x0) = 12 (0xc) fsync(0x11) = 0 (0x0) openat(AT_FDCWD,"/var/db/pkg",O_CLOEXEC,00) = 18 (0x12) fsync(0x12) = 0 (0x0) close(18) = 0 (0x0) pwrite(0xc,0x803325d08,0x400,0x0) = 1024 (0x400) pwrite(0xc,0x803319b08,0x400,0x10400) = 1024 (0x400) fsync(0xc) = 0 (0x0) close(17) = 0 (0x0) unlink("/var/db/pkg/local.sqlite-journal") = 0 (0x0) fcntl(12,F_SETLK,0x7fffffffd840) = 0 (0x0) fcntl(12,F_SETLK,0x7fffffffd840) = 0 (0x0) fcntl(12,F_SETLK,0x7fffffffd860) = 0 (0x0) exit(0x0) process exit, rval = 0 [2.4.0-BETA][root@apu2b2.localdomain]/root:Thanks again for pointing the methods. And I concluded that the database is missing.
So reinstall is mandatory I think. -
Hm but the command 'pkg upgrade' itself completed correctly right, what was the one that failed.? I dont have those -journal and -wal files myself either might be normal..
Was the failure with installing the pfsense package parts? Can you try?:
truss -o /root/truss_pkg_install.log pkg install pfSense pfSense-base pfSense-default-config pfSense-kernel-pfSense pfSense-rc pfSense-repo pfSense-upgrade -
After 2min, aborted. 16MB file.
[2.4.0-BETA][root@apu2b2.localdomain]/root: tail -40 truss_pkg_install.log mkdir("/var/cache",0777) ERR#17 'File exists' mkdir("/var/cache/pkg",0777) ERR#17 'File exists' statfs("/var/cache/pkg",0x7fffffffd798) ERR#2 'No such file or directory' mkdir("/var",0777) ERR#17 'File exists' mkdir("/var/cache",0777) ERR#17 'File exists' mkdir("/var/cache/pkg",0777) ERR#17 'File exists' statfs("/var/cache/pkg",0x7fffffffd798) ERR#2 'No such file or directory' mkdir("/var",0777) ERR#17 'File exists' mkdir("/var/cache",0777) ERR#17 'File exists' mkdir("/var/cache/pkg",0777) ERR#17 'File exists' statfs("/var/cache/pkg",0x7fffffffd798) ERR#2 'No such file or directory' mkdir("/var",0777) ERR#17 'File exists' mkdir("/var/cache",0777) ERR#17 'File exists' mkdir("/var/cache/pkg",0777) ERR#17 'File exists' statfs("/var/cache/pkg",0x7fffffffd798) ERR#2 'No such file or directory' mkdir("/var",0777) ERR#17 'File exists' mkdir("/var/cache",0777) ERR#17 'File exists' mkdir("/var/cache/pkg",0777) ERR#17 'File exists' statfs("/var/cache/pkg",0x7fffffffd798) ERR#2 'No such file or directory' mkdir("/var",0777) ERR#17 'File exists' mkdir("/var/cache",0777) ERR#17 'File exists' SIGNAL 2 (SIGINT) sigprocmask(SIG_SETMASK,{ SIGINT },0x0) = 0 (0x0) write(2,"pkg: ",5) = 5 (0x5) write(2,"\nsignal received, cleaning up",29) = 29 (0x1d) write(2,"\n",1) = 1 (0x1) unlink("pÙÿÿ") ERR#2 'No such file or directory' rename("pÙÿÿ-pkgtemp","pÙÿÿ") ERR#2 'No such file or directory' unlink("pÙÿÿ-journal") ERR#2 'No such file or directory' unlink("pÙÿÿ") ERR#2 'No such file or directory' rename("pÙÿÿ-pkgtemp","pÙÿÿ") ERR#2 'No such file or directory' unlink("pÙÿÿ-journal") ERR#2 'No such file or directory' unlink("pÙÿÿ") ERR#2 'No such file or directory' rename("pÙÿÿ-pkgtemp","pÙÿÿ") ERR#2 'No such file or directory' unlink("pÙÿÿ-journal") ERR#2 'No such file or directory' unlink("pÙÿÿ") ERR#2 'No such file or directory' rename("pÙÿÿ-pkgtemp","pÙÿÿ") ERR#2 'No such file or directory' unlink("pÙÿÿ-journal") ERR#2 'No such file or directory' exit(0x1) process exit, rval = 1 [2.4.0-BETA][root@apu2b2.localdomain]/root:Circular ?
[2.4.0-BETA][root@apu2b2.localdomain]/var/cache: ls -l total 1 lrwxr-xr-x 1 root wheel 24 Jan 16 16:51 pkg -> ../../root/var/cache/pkg [2.4.0-BETA][root@apu2b2.localdomain]/var/cache: -
So the "SIGNAL 2 (SIGINT)" is probably from you pressing Ctrl+C, and before that its looping over the same 4 commands..
What happens right before it enters that 'loop'?
Can you check if /var/cache/pkg/ exists, and all files seem 'valid' no 0 byte files or something? Or just throw the whole pkg folder in there away or rename it.?. -
Can you disable memory disks if you have those enabled? For me on a regular full install it isn't a 'linked' folder..
[2.4.0-BETA][root@pfSe.localdomain]/var/cache: ls -l total 105 drwxr-xr-x 2 root wheel 16 Jan 18 22:44 pkg -
I do not have RAM-disk enabled.
[2.4.0-BETA][root@apu2b2.localdomain]/: find / -name pkg /var/cache/pkg /var/db/pkg /var/db/pkg/pkg /etc/pkg /usr/share/keys/pkg /usr/local/pkg /usr/local/sbin/pkg /usr/local/etc/pkg /usr/local/share/pfSense/pkg /usr/local/share/pfSense/keys/pkg /usr/sbin/pkgAnd this is all that is there for "db pkg":
[2.4.0-BETA][root@apu2b2.localdomain]/: ls -l /var/db/pkg/pkg lrwxr-xr-x 1 root wheel 21 Jan 16 16:51 /var/db/pkg/pkg -> ../../root/var/db/pkg [2.4.0-BETA][root@apu2b2.localdomain]/: ls -l /var/db/pkg/ total 483 -rw-r--r-- 1 root wheel 109568 Jan 18 22:25 local.sqlite -rw-r--r-- 1 root wheel 246 Jan 18 21:59 pfSense-core.meta -rw-r--r-- 1 root wheel 246 Jan 18 16:55 pfSense.meta lrwxr-xr-x 1 root wheel 21 Jan 16 16:51 pkg -> ../../root/var/db/pkg -rw-r--r-- 1 root wheel 70656 Jan 18 22:26 repo-pfSense-core.sqlite -rw-r--r-- 1 root wheel 832512 Jan 18 22:26 repo-pfSense.sqlite [2.4.0-BETA][root@apu2b2.localdomain]/: -
My vote go's to getting rid of those linked pkg folders..
Mine:[2.4.0-BETA][root@pfSe.localdomain]/var/cache: ls -l /var/db/pkg total 3667 -rw-r--r-- 1 root wheel 6195200 Jan 18 22:44 local.sqlite -rw-r--r-- 1 root wheel 246 Jan 18 21:59 pfSense-core.meta -rw-r--r-- 1 root wheel 246 Jan 18 16:55 pfSense.meta -rw-r--r-- 1 root wheel 70656 Jan 18 22:42 repo-pfSense-core.sqlite -rw-r--r-- 1 root wheel 832512 Jan 18 22:42 repo-pfSense.sqlite [2.4.0-BETA][root@pfSe.localdomain]/var/cache: ls -l total 105 drwxr-xr-x 2 root wheel 16 Jan 18 22:44 pkg drwxr-xr-x 2 root wheel 474 Jan 18 19:43 pkg X [2.4.0-BETA][root@pfSe.localdomain]/var/cache:The place that "pkg -> ../../root/var/db/pkg" points to doesnt seem to exist.. I would try and delete it. And also from /var/cache/
pkg should recreate the cache folder itself when it downloads new stuff..
