ERROR: It was not possible to identify which pfSense kernel is installed
-
So what's up with this pkg db corruption? Only happens with RAM disks?
-
Until a week or so ago there were some corner cases like ZFS where the difference in how/when the disk was mounted stopped the code from working as expected.
On current snaps it should be OK all-around.
-
OK, thanks. ;)
-
Actually you could reinstall the packages to get back to a good state:
[2.4.0-BETA][root@apu2b2.localdomain]/root: pkg install pfSense pfSense-base pfSense-default-config pfSense-kernel-pfSense pfSense-rc pfSense-repo pfSense-upgrade Updating pfSense-core repository catalogue... Fetching meta.txz: 100% 940 B 0.9kB/s 00:01 Fetching packagesite.txz: 100% 2 KiB 1.8kB/s 00:01 Processing entries: 100% pfSense-core repository update completed. 7 packages processed. Updating pfSense repository catalogue... Fetching meta.txz: 100% 940 B 0.9kB/s 00:01 Fetching packagesite.txz: 100% 120 KiB 122.6kB/s 00:01 Processing entries: 100% pfSense repository update completed. 441 packages processed. The following 149 package(s) will be affected (of 0 checked): New packages to be INSTALLED: pfSense: 2.4.0.b.20170116.1330 [pfSense] pfSense-base: 2.4.0.b.20170116.1536 [pfSense-core] pfSense-default-config: 2.4.0.b.20170116.1536 [pfSense-core] pfSense-kernel-pfSense: 2.4.0.b.20170116.1536 [pfSense-core] pfSense-rc: 2.4.0.b.20170116.1536 [pfSense-core] pfSense-repo: 2.4.0.b.20170116.1330 [pfSense] pfSense-upgrade: 0.12 [pfSense] libxml2: 2.9.4 [pfSense] clog: 1.0.1 [pfSense] dpinger: 2.0 [pfSense] php56-pdo_sqlite: 5.6.29 [pfSense] php56-pdo: 5.6.29 [pfSense] php56: 5.6.29 [pfSense] pcre: 8.39_1 [pfSense] sqlite3: 3.15.1_1 [pfSense] readline: 6.3.8 [pfSense] indexinfo: 0.2.6 [pfSense] expiretable: 0.6_1 [pfSense] libltdl: 2.4.6 [pfSense] smartmontools: 6.5_1 [pfSense] php56-mbstring: 5.6.29 [pfSense] oniguruma5: 5.9.6_1 [pfSense] ipmitool: 1.8.17_1 [pfSense] php56-json: 5.6.29 [pfSense] php56-sysvshm: 5.6.29 [pfSense] pecl-radius: 1.4.0.b1 [pfSense] php56-bz2: 5.6.29 [pfSense] isc-dhcp43-server: 4.3.5 [pfSense] beep: 1.0_1 [pfSense] pear-Net_Growl: 2.7.0 [pfSense] pear: 1.10.1 [pfSense] php56-zlib: 5.6.29 [pfSense] php56-xml: 5.6.29 [pfSense] bwi-firmware-kmod: 3.130.20 [pfSense] ca_root_nss: 3.28.1 [pfSense] mpd5: 5.8 [pfSense] php56-hash: 5.6.29 [pfSense] mobile-broadband-provider-info: 20151214 [pfSense] bind-tools: 9.11.0P2 [pfSense] idnkit: 1.0_6 [pfSense] json-c: 0.12.1 [pfSense] libedit: 3.1.20150325_2,1 [pfSense] php56-mcrypt: 5.6.29 [pfSense] libmcrypt: 2.5.8_3 [pfSense] pear-XML_RPC2: 1.1.3 [pfSense] php56-curl: 5.6.29 [pfSense] curl: 7.52.1_1 [pfSense] libnghttp2: 1.18.0 [pfSense] pear-Cache_Lite: 1.7.16,1 [pfSense] pear-HTTP_Request2: 2.3.0,1 [pfSense] pear-Net_URL2: 2.2.1 [pfSense] pear-Mail: 1.3.0,1 [pfSense] pear-Net_SMTP: 1.7.2 [pfSense] pear-Net_Socket: 1.0.14 [pfSense] check_reload_status: 0.0.8 [pfSense] libevent2: 2.0.22_1 [pfSense] ataidle: 2.7.2 [pfSense] php56-tokenizer: 5.6.29 [pfSense] minicron: 0.0.2 [pfSense] dhcp6: 20080615.1 [pfSense] php56-ctype: 5.6.29 [pfSense] openvpn: 2.4.0 [pfSense] lzo2: 2.09 [pfSense] liblz4: 131 [pfSense] ntp: 4.2.8p9_1 [pfSense] libwww: 5.4.0_6 [pfSense] gettext-runtime: 0.19.8.1_1 [pfSense] pecl-zmq: 1.1.3_2 [pfSense] libzmq4: 4.1.5 [pfSense] norm: 1.5r6 [pfSense] libsodium: 1.0.11_1 [pfSense] ssh_tunnel_shell: 0.1_1 [pfSense] php56-simplexml: 5.6.29 [pfSense] cpdup: 1.18 [pfSense] pear-Net_IPv6: 1.3.0.b2_2 [pfSense] miniupnpd: 1.9.20160113,1 [pfSense] pecl-ssh2: 0.13 [pfSense] libssh2: 1.8.0,2 [pfSense] sshlockout_pf: 0.0.2 [pfSense] pfSense-Status_Monitoring: 1.6.0_2 [pfSense] pecl-rrd: 1.1.3_4 [pfSense] rrdtool: 1.6.0_1 [pfSense] perl5: 5.24.1.r4_1 [pfSense] glib: 2.46.2_4 [pfSense] python27: 2.7.13_1 [pfSense] libffi: 3.2.1 [pfSense] libiconv: 1.14_10 [pfSense] pear-Auth_RADIUS: 1.1.0 [pfSense] radvd: 1.9.1 [pfSense] libdaemon: 0.14_1 [pfSense] unbound: 1.6.0 [pfSense] expat: 2.2.0_1 [pfSense] ldns: 1.6.17_5 [pfSense] isc-dhcp43-relay: 4.3.5 [pfSense] dhcpleases: 0.3_1 [pfSense] php56-opcache: 5.6.29 [pfSense] php56-dom: 5.6.29 [pfSense] gogoc: 1.2_1 [pfSense] wol: 0.7.1_3 [pfSense] php56-xmlwriter: 5.6.29 [pfSense] choparp: 20150613 [pfSense] nginx: 1.10.2_3,2 [pfSense] luajit: 2.0.4_1 [pfSense] dnsmasq: 2.76,1 [pfSense] nettle: 3.3 [pfSense] gmp: 5.1.3_3 [pfSense] libidn: 1.33_1 [pfSense] pftop: 0.7_7 [pfSense] strongswan: 5.5.1 [pfSense] vstr: 1.0.15_1 [pfSense] php56-sockets: 5.6.29 [pfSense] php56-filter: 5.6.29 [pfSense] voucher: 0.1_2 [pfSense] filterdns: 1.0_15 [pfSense] isc-dhcp43-client: 4.3.5 [pfSense] php56-pfSense-module: 0.32 [pfSense] links: 2.13,1 [pfSense] php56-sysvsem: 5.6.29 [pfSense] php56-readline: 5.6.29 [pfSense] filterlog: 0.1_5 [pfSense] scponly: 4.8.20110526_2 [pfSense] qstats: 0.2 [pfSense] php56-posix: 5.6.29 [pfSense] igmpproxy: 0.1_5,1 [pfSense] php56-openssl: 5.6.29 [pfSense] php56-gettext: 5.6.29 [pfSense] php-xdebug: 2.4.1_1 [pfSense] xinetd: 2.3.15_2 [pfSense] bsnmp-regex: 0.6_1 [pfSense] php56-ldap: 5.6.29 [pfSense] openldap-client: 2.4.44 [pfSense] uclcmd: 0.1_1 [pfSense] libucl: 0.8.0 [pfSense] wrapalixresetbutton: 0.0.7 [pfSense] php56-pcntl: 5.6.29 [pfSense] rate: 0.9_1 [pfSense] php56-session: 5.6.29 [pfSense] php56-sysvmsg: 5.6.29 [pfSense] php56-shmop: 5.6.29 [pfSense] php56-xmlreader: 5.6.29 [pfSense] bsnmp-ucd: 0.4.2 [pfSense] dhcpleases6: 0.1_2 [pfSense] pear-Crypt_CHAP: 1.5.0 [pfSense] dmidecode: 3.0 [pfSense] php56-sqlite3: 5.6.29 [pfSense] relayd: 5.5.20140810_2 [pfSense] cpustats: 0.1_1 [pfSense] php-suhosin: 0.9.38_3 [pfSense] php56-bcmath: 5.6.29 [pfSense] Number of packages to be installed: 149 The process will require 449 MiB more space. 125 MiB to be downloaded. Proceed with this action? [y/N]: yNo dice ?
Already more than 15min in this state. No I/O. CPU 25%, Load ~ 1.4, Mem. normal 14%. What's going on ?ADD: I aborted after 30min. Ctrl-C => signal received, cleaning up. Back at prompt.
ADD:
PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 53727 root 1 103 0 47888K 14804K CPU1 1 2:39 100.00% pkg 27942 root 1 23 0 264M 29224K piperd 2 0:00 1.76% php-fpm 47711 root 1 52 20 13048K 2384K wait 2 0:42 0.20% sh .... -
Might be that some pkg file is partially corrupted? Can you try (im assuming the name of the pool is zroot):
zpool list zpool scrub zroot zpool status -v zrootKeep running the last command until its done scanning the disk.
-
[2.4.0-BETA][root@apu2b2.localdomain]/root: zpool list NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT zroot 12.9G 470M 12.4G - 6% 3% 1.00x ONLINE - [2.4.0-BETA][root@apu2b2.localdomain]/root: zpool scrub zroot [2.4.0-BETA][root@apu2b2.localdomain]/root: [2.4.0-BETA][root@apu2b2.localdomain]/root: zpool status -v zroot pool: zroot state: ONLINE scan: scrub repaired 0 in 0h0m with 0 errors on Tue Jan 17 20:55:02 2017 config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 ada0p3 ONLINE 0 0 0 errors: No known data errors -
Try logging what its doing, might show that it fails to read (part of) some file? After a minute or two check the logfile possibly showing lots of commands repeating/hanging at the end of the log.. The try and find what if any file it was busy with at that time.?.
truss -o /root/truss_pkg_upgrade.log pkg upgradeNot sure if this is the best way, but it did help me find some problem files once..
-
[2.4.0-BETA][root@apu2b2.localdomain]/root: truss -o /root/truss_pkg_upgrade.log pkg upgrade Updating pfSense-core repository catalogue... Fetching meta.txz: 100% 940 B 0.9kB/s 00:01 Fetching packagesite.txz: 100% 2 KiB 1.8kB/s 00:01 Processing entries: 100% pfSense-core repository update completed. 7 packages processed. Updating pfSense repository catalogue... Fetching meta.txz: 100% 940 B 0.9kB/s 00:01 Fetching packagesite.txz: 100% 120 KiB 122.6kB/s 00:01 Processing entries: 100% pfSense repository update completed. 441 packages processed. Checking for upgrades (0 candidates): 100% Processing candidates (0 candidates): 100% Checking integrity... done (0 conflicting) Your packages are up to date. [2.4.0-BETA][root@apu2b2.localdomain]/root:A lot of "No such file or directory" in the log file i.e.
[2.4.0-BETA][root@apu2b2.localdomain]/root: tail -40 truss_pkg_upgrade.log access("/var/db/pkg/local.sqlite-journal",F_OK) ERR#2 'No such file or directory' fstat(12,{ mode=-rw-r--r-- ,inode=20540,size=109568,blksize=109568 }) = 0 (0x0) pread(0xc,0x7fffffffd900,0x10,0x18) = 16 (0x10) fstat(12,{ mode=-rw-r--r-- ,inode=20540,size=109568,blksize=109568 }) = 0 (0x0) access("/var/db/pkg/local.sqlite-wal",F_OK) ERR#2 'No such file or directory' fstat(12,{ mode=-rw-r--r-- ,inode=20540,size=109568,blksize=109568 }) = 0 (0x0) fcntl(12,F_SETLK,0x7fffffffd8d0) = 0 (0x0) stat("/var/db/pkg/local.sqlite",{ mode=-rw-r--r-- ,inode=20540,size=109568,blksize=109568 }) = 0 (0x0) getpid() = 86979 (0x153c3) stat("/var/db/pkg/local.sqlite",{ mode=-rw-r--r-- ,inode=20540,size=109568,blksize=109568 }) = 0 (0x0) openat(AT_FDCWD,"/var/db/pkg/local.sqlite-journal",O_RDWR|O_CREAT|O_CLOEXEC,0644) = 17 (0x11) fstat(17,{ mode=-rw-r--r-- ,inode=21580,size=0,blksize=131072 }) = 0 (0x0) geteuid() = 0 (0x0) fchown(17,0,0) = 0 (0x0) fstat(17,{ mode=-rw-r--r-- ,inode=21580,size=0,blksize=131072 }) = 0 (0x0) pwrite(0x11,0x8031d7908,0x200,0x0) = 512 (0x200) pwrite(0x11,0x7fffffffd914,0x4,0x200) = 4 (0x4) pwrite(0x11,0x803319b08,0x400,0x204) = 1024 (0x400) pwrite(0x11,0x7fffffffd914,0x4,0x604) = 4 (0x4) fcntl(12,F_SETLK,0x7fffffffd8c0) = 0 (0x0) fcntl(12,F_SETLK,0x7fffffffd8c0) = 0 (0x0) pwrite(0x11,0x7fffffffd8a4,0x4,0x608) = 4 (0x4) pwrite(0x11,0x803325d08,0x400,0x60c) = 1024 (0x400) pwrite(0x11,0x7fffffffd8a4,0x4,0xa0c) = 4 (0x4) pread(0x11,0x7fffffffd898,0x8,0xc00) = 0 (0x0) pwrite(0x11,0x7fffffffd888,0xc,0x0) = 12 (0xc) fsync(0x11) = 0 (0x0) openat(AT_FDCWD,"/var/db/pkg",O_CLOEXEC,00) = 18 (0x12) fsync(0x12) = 0 (0x0) close(18) = 0 (0x0) pwrite(0xc,0x803325d08,0x400,0x0) = 1024 (0x400) pwrite(0xc,0x803319b08,0x400,0x10400) = 1024 (0x400) fsync(0xc) = 0 (0x0) close(17) = 0 (0x0) unlink("/var/db/pkg/local.sqlite-journal") = 0 (0x0) fcntl(12,F_SETLK,0x7fffffffd840) = 0 (0x0) fcntl(12,F_SETLK,0x7fffffffd840) = 0 (0x0) fcntl(12,F_SETLK,0x7fffffffd860) = 0 (0x0) exit(0x0) process exit, rval = 0 [2.4.0-BETA][root@apu2b2.localdomain]/root:Thanks again for pointing the methods. And I concluded that the database is missing.
So reinstall is mandatory I think. -
Hm but the command 'pkg upgrade' itself completed correctly right, what was the one that failed.? I dont have those -journal and -wal files myself either might be normal..
Was the failure with installing the pfsense package parts? Can you try?:
truss -o /root/truss_pkg_install.log pkg install pfSense pfSense-base pfSense-default-config pfSense-kernel-pfSense pfSense-rc pfSense-repo pfSense-upgrade -
After 2min, aborted. 16MB file.
[2.4.0-BETA][root@apu2b2.localdomain]/root: tail -40 truss_pkg_install.log mkdir("/var/cache",0777) ERR#17 'File exists' mkdir("/var/cache/pkg",0777) ERR#17 'File exists' statfs("/var/cache/pkg",0x7fffffffd798) ERR#2 'No such file or directory' mkdir("/var",0777) ERR#17 'File exists' mkdir("/var/cache",0777) ERR#17 'File exists' mkdir("/var/cache/pkg",0777) ERR#17 'File exists' statfs("/var/cache/pkg",0x7fffffffd798) ERR#2 'No such file or directory' mkdir("/var",0777) ERR#17 'File exists' mkdir("/var/cache",0777) ERR#17 'File exists' mkdir("/var/cache/pkg",0777) ERR#17 'File exists' statfs("/var/cache/pkg",0x7fffffffd798) ERR#2 'No such file or directory' mkdir("/var",0777) ERR#17 'File exists' mkdir("/var/cache",0777) ERR#17 'File exists' mkdir("/var/cache/pkg",0777) ERR#17 'File exists' statfs("/var/cache/pkg",0x7fffffffd798) ERR#2 'No such file or directory' mkdir("/var",0777) ERR#17 'File exists' mkdir("/var/cache",0777) ERR#17 'File exists' mkdir("/var/cache/pkg",0777) ERR#17 'File exists' statfs("/var/cache/pkg",0x7fffffffd798) ERR#2 'No such file or directory' mkdir("/var",0777) ERR#17 'File exists' mkdir("/var/cache",0777) ERR#17 'File exists' SIGNAL 2 (SIGINT) sigprocmask(SIG_SETMASK,{ SIGINT },0x0) = 0 (0x0) write(2,"pkg: ",5) = 5 (0x5) write(2,"\nsignal received, cleaning up",29) = 29 (0x1d) write(2,"\n",1) = 1 (0x1) unlink("pÙÿÿ") ERR#2 'No such file or directory' rename("pÙÿÿ-pkgtemp","pÙÿÿ") ERR#2 'No such file or directory' unlink("pÙÿÿ-journal") ERR#2 'No such file or directory' unlink("pÙÿÿ") ERR#2 'No such file or directory' rename("pÙÿÿ-pkgtemp","pÙÿÿ") ERR#2 'No such file or directory' unlink("pÙÿÿ-journal") ERR#2 'No such file or directory' unlink("pÙÿÿ") ERR#2 'No such file or directory' rename("pÙÿÿ-pkgtemp","pÙÿÿ") ERR#2 'No such file or directory' unlink("pÙÿÿ-journal") ERR#2 'No such file or directory' unlink("pÙÿÿ") ERR#2 'No such file or directory' rename("pÙÿÿ-pkgtemp","pÙÿÿ") ERR#2 'No such file or directory' unlink("pÙÿÿ-journal") ERR#2 'No such file or directory' exit(0x1) process exit, rval = 1 [2.4.0-BETA][root@apu2b2.localdomain]/root:Circular ?
[2.4.0-BETA][root@apu2b2.localdomain]/var/cache: ls -l total 1 lrwxr-xr-x 1 root wheel 24 Jan 16 16:51 pkg -> ../../root/var/cache/pkg [2.4.0-BETA][root@apu2b2.localdomain]/var/cache: -
So the "SIGNAL 2 (SIGINT)" is probably from you pressing Ctrl+C, and before that its looping over the same 4 commands..
What happens right before it enters that 'loop'?
Can you check if /var/cache/pkg/ exists, and all files seem 'valid' no 0 byte files or something? Or just throw the whole pkg folder in there away or rename it.?. -
Can you disable memory disks if you have those enabled? For me on a regular full install it isn't a 'linked' folder..
[2.4.0-BETA][root@pfSe.localdomain]/var/cache: ls -l total 105 drwxr-xr-x 2 root wheel 16 Jan 18 22:44 pkg -
I do not have RAM-disk enabled.
[2.4.0-BETA][root@apu2b2.localdomain]/: find / -name pkg /var/cache/pkg /var/db/pkg /var/db/pkg/pkg /etc/pkg /usr/share/keys/pkg /usr/local/pkg /usr/local/sbin/pkg /usr/local/etc/pkg /usr/local/share/pfSense/pkg /usr/local/share/pfSense/keys/pkg /usr/sbin/pkgAnd this is all that is there for "db pkg":
[2.4.0-BETA][root@apu2b2.localdomain]/: ls -l /var/db/pkg/pkg lrwxr-xr-x 1 root wheel 21 Jan 16 16:51 /var/db/pkg/pkg -> ../../root/var/db/pkg [2.4.0-BETA][root@apu2b2.localdomain]/: ls -l /var/db/pkg/ total 483 -rw-r--r-- 1 root wheel 109568 Jan 18 22:25 local.sqlite -rw-r--r-- 1 root wheel 246 Jan 18 21:59 pfSense-core.meta -rw-r--r-- 1 root wheel 246 Jan 18 16:55 pfSense.meta lrwxr-xr-x 1 root wheel 21 Jan 16 16:51 pkg -> ../../root/var/db/pkg -rw-r--r-- 1 root wheel 70656 Jan 18 22:26 repo-pfSense-core.sqlite -rw-r--r-- 1 root wheel 832512 Jan 18 22:26 repo-pfSense.sqlite [2.4.0-BETA][root@apu2b2.localdomain]/: -
My vote go's to getting rid of those linked pkg folders..
Mine:[2.4.0-BETA][root@pfSe.localdomain]/var/cache: ls -l /var/db/pkg total 3667 -rw-r--r-- 1 root wheel 6195200 Jan 18 22:44 local.sqlite -rw-r--r-- 1 root wheel 246 Jan 18 21:59 pfSense-core.meta -rw-r--r-- 1 root wheel 246 Jan 18 16:55 pfSense.meta -rw-r--r-- 1 root wheel 70656 Jan 18 22:42 repo-pfSense-core.sqlite -rw-r--r-- 1 root wheel 832512 Jan 18 22:42 repo-pfSense.sqlite [2.4.0-BETA][root@pfSe.localdomain]/var/cache: ls -l total 105 drwxr-xr-x 2 root wheel 16 Jan 18 22:44 pkg drwxr-xr-x 2 root wheel 474 Jan 18 19:43 pkg X [2.4.0-BETA][root@pfSe.localdomain]/var/cache:The place that "pkg -> ../../root/var/db/pkg" points to doesnt seem to exist.. I would try and delete it. And also from /var/cache/
pkg should recreate the cache folder itself when it downloads new stuff.. -
Beautyfull !
Yes, the removal of the link-files freed up the update & recreation processes.
Thanks PiBa.... pfSense 2.4.0-BETA amd64 Wed Jan 18 14:23:45 CST 2017 Bootup complete FreeBSD/amd64 (apu2b2.localdomain) (ttyu0) *** Welcome to pfSense 2.4.0-BETA (amd64) on apu2b2 *** ...:) :) :)
-
Just freshly installed pfSense-CE-2.4.0-BETA-amd64-20170119-1521.iso on a KVM.
Same error:
pkg: No package(s) matching pfSense-kernel-*Issuing
pkg install pfSense pfSense-base pfSense-default-config pfSense-kernel-pfSense pfSense-rc pfSense-repo pfSense-upgradeseems to have fixed it.
Maybe related to changes introduced in Bug #7086?
zfs list NAME USED AVAIL REFER MOUNTPOINT zroot 592M 45.7G 96K /zroot zroot/ROOT 455M 45.7G 96K none zroot/ROOT/default 454M 45.7G 454M / zroot/tmp 336K 45.7G 336K /tmp zroot/var 137M 45.7G 137M /var -
Does this still happen on a current snapshot?
-
Nope, Renato fixed it. I also reinstalled my primary system today, all fine.
-
S Sergei_Shablovsky referenced this topic on
-
S Sergei_Shablovsky referenced this topic on
-
S Sergei_Shablovsky referenced this topic on
-
S Sergei_Shablovsky referenced this topic on
-
S Sergei_Shablovsky referenced this topic on
-
S Sergei_Shablovsky referenced this topic on
-
S Sergei_Shablovsky referenced this topic on
Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.