Kernel Panic in 2.7.0
-
@pfsss Do you have the backtrace from that crash?
-
@stephenw10 yes
db:0:kdb.enter.default> bt Tracing pid 9 tid 100125 td 0xfffffe006b90f900 kdb_enter() at kdb_enter+0x32/frame 0xfffffe006af05810 vpanic() at vpanic+0x183/frame 0xfffffe006af05860 panic() at panic+0x43/frame 0xfffffe006af058c0 trap_fatal() at trap_fatal+0x409/frame 0xfffffe006af05920 trap_pfault() at trap_pfault+0x4f/frame 0xfffffe006af05980 calltrap() at calltrap+0x8/frame 0xfffffe006af05980 --- trap 0xc, rip = 0xffffffff80cd2552, rsp = 0xfffffe006af05a50, rbp = 0xfffffe006af05a60 --- _mtx_trylock_flags_() at _mtx_trylock_flags_+0x32/frame 0xfffffe006af05a60 pmap_ts_referenced() at pmap_ts_referenced+0x5d5/frame 0xfffffe006af05af0 vm_pageout_worker() at vm_pageout_worker+0xb21/frame 0xfffffe006af05eb0 vm_pageout() at vm_pageout+0x1c7/frame 0xfffffe006af05ef0 fork_exit() at fork_exit+0x7d/frame 0xfffffe006af05f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe006af05f30 --- trap 0xb96e72dc, rip = 0xffffffffffffffff, rsp = 0xffffffffffffffff, rbp = 0xfdf9fbf3f7e6efcc --- -
I forked this to a new topic because that's a completely different crash.
Did it just start happening? Was it stable in 2.6? Any reason you are not running 2.7.2?
-
@stephenw10 I installed 2.7.0 in this x86 machine and not tested in 2.6.0 . this happens stochastic, sometimes few hours.
-
@stephenw10 I'd be like to update 2.7.2 but when I check for updates shows this
Version 2.7.0-RELEASE (amd64)
built on Wed Jun 28 03:53:34 UTC 2023
FreeBSD 14.0-CURRENTThe system is on the latest version.
Version information updated at Thu Jan 4 21:58:57 +08 2024 -
Ok, so it has always crashed in 2.7.0 on that hardware?
The crash itself could well be a bad RAM. Do the crashes all look the same? Same backtrace?
You can install 2.7.2 directly. Otherwise make sure latest version is set in System > Update. Try running
pfSense-upgradeat the CLI and see wha error is shown. -
@stephenw10 the web ui upgrade is as follow
no button to confirm upgrade
-
@stephenw10 here are some crash bt
one crash:
db:0:kdb.enter.default> bt
Tracing pid 9 tid 100125 td 0xfffffe006b90f900
kdb_enter() at kdb_enter+0x32/frame 0xfffffe006af05810
vpanic() at vpanic+0x183/frame 0xfffffe006af05860
panic() at panic+0x43/frame 0xfffffe006af058c0
trap_fatal() at trap_fatal+0x409/frame 0xfffffe006af05920
trap_pfault() at trap_pfault+0x4f/frame 0xfffffe006af05980
calltrap() at calltrap+0x8/frame 0xfffffe006af05980
--- trap 0xc, rip = 0xffffffff80cd2552, rsp = 0xfffffe006af05a50, rbp = 0xfffffe006af05a60 ---
mtx_trylock_flags() at mtx_trylock_flags+0x32/frame 0xfffffe006af05a60
pmap_ts_referenced() at pmap_ts_referenced+0x5d5/frame 0xfffffe006af05af0
vm_pageout_worker() at vm_pageout_worker+0xb21/frame 0xfffffe006af05eb0
vm_pageout() at vm_pageout+0x1c7/frame 0xfffffe006af05ef0
fork_exit() at fork_exit+0x7d/frame 0xfffffe006af05f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe006af05f30
--- trap 0xb96e72dc, rip = 0xffffffffffffffff, rsp = 0xffffffffffffffff, rbp = 0xfdf9fbf3f7e6efcc ---
the other one
db:0:kdb.enter.default> bt
Tracing pid 17611 tid 100279 td 0xfffffe00a14413a0
kdb_enter() at kdb_enter+0x32/frame 0xfffffe00a04c4070
vpanic() at vpanic+0x183/frame 0xfffffe00a04c40c0
panic() at panic+0x43/frame 0xfffffe00a04c4120
vm_fault() at vm_fault+0x1539/frame 0xfffffe00a04c4230
vm_fault_trap() at vm_fault_trap+0xb1/frame 0xfffffe00a04c4280
trap_pfault() at trap_pfault+0x1f2/frame 0xfffffe00a04c42e0
calltrap() at calltrap+0x8/frame 0xfffffe00a04c42e0
--- trap 0xc, rip = 0xffffffff811616d7, rsp = 0xfffffe00a04c43b0, rbp = 0xfffffe00a04c43b0 ---
_vm_page_pqstate_commit_dequeue() at _vm_page_pqstate_commit_dequeue+0x47/frame 0xfffffe00a04c43b0
vm_page_pqstate_commit_dequeue() at vm_page_pqstate_commit_dequeue+0x7b/frame 0xfffffe00a04c43f0
vm_page_alloc_domain_after() at vm_page_alloc_domain_after+0x242/frame 0xfffffe00a04c4460
vm_page_alloc() at vm_page_alloc+0x64/frame 0xfffffe00a04c44c0
vm_fault_allocate() at vm_fault_allocate+0x1d4/frame 0xfffffe00a04c4530
vm_fault() at vm_fault+0x360/frame 0xfffffe00a04c4640
vm_fault_trap() at vm_fault_trap+0x6d/frame 0xfffffe00a04c4690
trap_pfault() at trap_pfault+0x1f2/frame 0xfffffe00a04c46f0
calltrap() at calltrap+0x8/frame 0xfffffe00a04c46f0
--- trap 0xc, rip = 0xffffffff81297056, rsp = 0xfffffe00a04c47c0, rbp = 0xfffffe00a04c47c0 ---
copyout_nosmap_erms() at copyout_nosmap_erms+0x156/frame 0xfffffe00a04c47c0
elf64_load_sections() at elf64_load_sections+0x265/frame 0xfffffe00a04c4860
elf64_load_file() at elf64_load_file+0x24f/frame 0xfffffe00a04c48c0
elf64_load_interp() at elf64_load_interp+0xff/frame 0xfffffe00a04c4910
exec_elf64_imgact() at exec_elf64_imgact+0xe32/frame 0xfffffe00a04c4a10
kern_execve() at kern_execve+0x658/frame 0xfffffe00a04c4d80
sys_execve() at sys_execve+0x58/frame 0xfffffe00a04c4e00
amd64_syscall() at amd64_syscall+0x109/frame 0xfffffe00a04c4f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00a04c4f30
--- syscall (59, FreeBSD ELF64, execve), rip = 0x8268e4ffa, rsp = 0x820b57978, rbp = 0x820b579f0 ---
db:0:kdb.enter.default> psthe third one
db:0:kdb.enter.default> bt
Tracing pid 84979 tid 100259 td 0xfffffe00a15a9720
kdb_enter() at kdb_enter+0x32/frame 0xfffffe009fcb0990
vpanic() at vpanic+0x183/frame 0xfffffe009fcb09e0
panic() at panic+0x43/frame 0xfffffe009fcb0a40
trap_fatal() at trap_fatal+0x409/frame 0xfffffe009fcb0aa0
trap_pfault() at trap_pfault+0x4f/frame 0xfffffe009fcb0b00
calltrap() at calltrap+0x8/frame 0xfffffe009fcb0b00
--- trap 0xc, rip = 0xffffffff811616d7, rsp = 0xfffffe009fcb0bd0, rbp = 0xfffffe009fcb0bd0 ---
_vm_page_pqstate_commit_dequeue() at _vm_page_pqstate_commit_dequeue+0x47/frame 0xfffffe009fcb0bd0
vm_pqbatch_process_page() at vm_pqbatch_process_page+0xaf/frame 0xfffffe009fcb0c10
vm_page_pqbatch_submit() at vm_page_pqbatch_submit+0x20f/frame 0xfffffe009fcb0c60
vm_page_free_prep() at vm_page_free_prep+0x9c/frame 0xfffffe009fcb0c80
vm_page_free_toq() at vm_page_free_toq+0x12/frame 0xfffffe009fcb0cb0
vm_object_terminate() at vm_object_terminate+0x12a/frame 0xfffffe009fcb0cd0
vm_object_deallocate() at vm_object_deallocate+0x1a8/frame 0xfffffe009fcb0d10
vm_map_process_deferred() at vm_map_process_deferred+0x92/frame 0xfffffe009fcb0d30
vm_map_remove() at vm_map_remove+0x100/frame 0xfffffe009fcb0d60
vmspace_exit() at vmspace_exit+0xaa/frame 0xfffffe009fcb0d90
exit1() at exit1+0x56c/frame 0xfffffe009fcb0df0
sys_exit() at sys_exit+0xd/frame 0xfffffe009fcb0e00
amd64_syscall() at amd64_syscall+0x109/frame 0xfffffe009fcb0f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe009fcb0f30
--- syscall (1, FreeBSD ELF64, exit), rip = 0x358e7bf9e99a, rsp = 0x358e77c4b518, rbp = 0x358e77c4b530 ---
db:0:kdb.enter.default> ps -
@stephenw10 when I run the command ```
pfsense-upgrade~~ERROR: It was not possible to determine pkg remote version >>> Updating repositories metadata... Updating pfSense-core repository catalogue... pkg-static: An error occured while fetching package pkg-static: An error occured while fetching package repository pfSense-core has no meta file, using default settings pkg-static: An error occured while fetching package pkg-static: An error occured while fetching package Unable to update repository pfSense-core Updating pfSense repository catalogue... pkg-static: An error occured while fetching package pkg-static: An error occured while fetching package repository pfSense has no meta file, using default settings pkg-static: An error occured while fetching package pkg-static: An error occured while fetching package Unable to update repository pfSense Error updating repositories! ERROR: It was not possible to determine pfSense-upgrade remote version ERROR: It was not possible to determine pfSense-upgrade remote version >>> Upgrading pfSense-upgrade... failed.~~ -
Ok, so those crashes all appear to be in memory management. I would run a few cycles of memtest to be sure it's not bad RAM.
Try running
pfSense-upgrade -dto get more debug output. -
@stephenw10 said in Kernel Panic in 2.7.0:
I would run a few cycles of memtest to be sure it's not bad RAM
thanks for replying. how to do memtest, is there a simple instruction to run memtest? thanks!
after running the command
pfsense-upgrade -dthe output information is as follows:
~~ERROR: It was not possible to determine pkg remote versionUpdating repositories metadata...
Updating pfSense-core repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense-core has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense
Error updating repositories!
ERROR: It was not possible to determine pfSense-upgrade remote version
ERROR: It was not possible to determine pfSense-upgrade remote version
Upgrading pfSense-upgrade...
Updating pfSense-core repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense-core has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense
Error updating repositories!~~ -
ERROR: It was not possible to determine pkg remote version >>> Updating repositories metadata... Updating pfSense-core repository catalogue... pkg-static: An error occured while fetching package pkg-static: An error occured while fetching package repository pfSense-core has no meta file, using default settings pkg-static: An error occured while fetching package pkg-static: An error occured while fetching package Unable to update repository pfSense-core Updating pfSense repository catalogue... pkg-static: An error occured while fetching package pkg-static: An error occured while fetching package repository pfSense has no meta file, using default settings pkg-static: An error occured while fetching package pkg-static: An error occured while fetching package Unable to update repository pfSense Error updating repositories! ERROR: It was not possible to determine pfSense-upgrade remote version ERROR: It was not possible to determine pfSense-upgrade remote version >>> Upgrading pfSense-upgrade... Updating pfSense-core repository catalogue... pkg-static: An error occured while fetching package pkg-static: An error occured while fetching package repository pfSense-core has no meta file, using default settings pkg-static: An error occured while fetching package pkg-static: An error occured while fetching package Unable to update repository pfSense-core Updating pfSense repository catalogue... pkg-static: An error occured while fetching package pkg-static: An error occured while fetching package repository pfSense has no meta file, using default settings pkg-static: An error occured while fetching package pkg-static: An error occured while fetching package Unable to update repository pfSense Error updating repositories! -
@pfsss
Hi,
From the command line are you able to ping outside sites by name, for example to www.yahoo.com, or some known site? Looks like you don't have access to internet.
Thanks, and best regards,
HeCSa. -
@hecsa hi
I ping yahoo.com usingping -c 5 yahoo.com, the output isPING6(56=40+8+8 bytes) 2***************4 --> 2001:4998:24:120d::1:1 16 bytes from 2001:4998:24:120d::1:1, icmp_seq=0 hlim=46 time=263.204 ms 16 bytes from 2001:4998:24:120d::1:1, icmp_seq=1 hlim=47 time=263.954 ms 16 bytes from 2001:4998:24:120d::1:1, icmp_seq=2 hlim=47 time=261.079 ms 16 bytes from 2001:4998:24:120d::1:1, icmp_seq=3 hlim=46 time=259.364 ms 16 bytes from 2001:4998:24:120d::1:1, icmp_seq=4 hlim=47 time=257.564 ms --- yahoo.com ping6 statistics --- 5 packets transmitted, 5 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 257.564/261.033/263.954/2.369 msit seems not the network problem~~
-
@pfsss
Hi,
Take a look at this thread, looks something like what you are experiencing: https://forum.netgate.com/topic/121783/solved-update-2-3-4-to-2-4-failed-unable-to-check-for-updates/3
Thanks, and best regards,
HeCSa. -
@hecsa I tried to
ls -al /var/pkg/cachethere is no such directory.. -
Try:
pkg-static -d updateThat should give a more useful error.
-
You probably need to run:
certctl rehash -
@stephenw10 hi
after running commandcertctl rehashandpng-static -d updateI get the following hintsNewer FreeBSD version for package xxd: To ignore this error set IGNORE_OSVERSION=yes - package: 1400094 - running kernel: 1400085 Ignore the mismatch and continue? [y/N]: pkg-static: repository pfSense contains packages for wrong OS version: FreeBSD:14:amd64 Processing entries... done Unable to update repository pfSense Error updating repositories!I think it is close to the right way
-
Yes, it must be seeing the pkgs to be able to throw that error so try to upgrade again.
