Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Kernel Panic in 2.7.0

    Scheduled Pinned Locked Moved General pfSense Questions
    29 Posts 4 Posters 3.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • stephenw10S
      stephenw10 Netgate Administrator
      last edited by

      Ok, so it has always crashed in 2.7.0 on that hardware?

      The crash itself could well be a bad RAM. Do the crashes all look the same? Same backtrace?

      You can install 2.7.2 directly. Otherwise make sure latest version is set in System > Update. Try running pfSense-upgrade at the CLI and see wha error is shown.

      P 3 Replies Last reply Reply Quote 0
      • P
        pfsss @stephenw10
        last edited by

        @stephenw10 the web ui upgrade is as follow59e5ff73-1435-4e8b-8f8e-48f97e9f9bb6-image.png

        no button to confirm upgrade

        1 Reply Last reply Reply Quote 0
        • P
          pfsss @stephenw10
          last edited by

          @stephenw10 here are some crash bt
          one crash:
          db:0:kdb.enter.default> bt
          Tracing pid 9 tid 100125 td 0xfffffe006b90f900
          kdb_enter() at kdb_enter+0x32/frame 0xfffffe006af05810
          vpanic() at vpanic+0x183/frame 0xfffffe006af05860
          panic() at panic+0x43/frame 0xfffffe006af058c0
          trap_fatal() at trap_fatal+0x409/frame 0xfffffe006af05920
          trap_pfault() at trap_pfault+0x4f/frame 0xfffffe006af05980
          calltrap() at calltrap+0x8/frame 0xfffffe006af05980
          --- trap 0xc, rip = 0xffffffff80cd2552, rsp = 0xfffffe006af05a50, rbp = 0xfffffe006af05a60 ---
          mtx_trylock_flags() at mtx_trylock_flags+0x32/frame 0xfffffe006af05a60
          pmap_ts_referenced() at pmap_ts_referenced+0x5d5/frame 0xfffffe006af05af0
          vm_pageout_worker() at vm_pageout_worker+0xb21/frame 0xfffffe006af05eb0
          vm_pageout() at vm_pageout+0x1c7/frame 0xfffffe006af05ef0
          fork_exit() at fork_exit+0x7d/frame 0xfffffe006af05f30
          fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe006af05f30
          --- trap 0xb96e72dc, rip = 0xffffffffffffffff, rsp = 0xffffffffffffffff, rbp = 0xfdf9fbf3f7e6efcc ---
          the other one
          db:0:kdb.enter.default> bt
          Tracing pid 17611 tid 100279 td 0xfffffe00a14413a0
          kdb_enter() at kdb_enter+0x32/frame 0xfffffe00a04c4070
          vpanic() at vpanic+0x183/frame 0xfffffe00a04c40c0
          panic() at panic+0x43/frame 0xfffffe00a04c4120
          vm_fault() at vm_fault+0x1539/frame 0xfffffe00a04c4230
          vm_fault_trap() at vm_fault_trap+0xb1/frame 0xfffffe00a04c4280
          trap_pfault() at trap_pfault+0x1f2/frame 0xfffffe00a04c42e0
          calltrap() at calltrap+0x8/frame 0xfffffe00a04c42e0
          --- trap 0xc, rip = 0xffffffff811616d7, rsp = 0xfffffe00a04c43b0, rbp = 0xfffffe00a04c43b0 ---
          _vm_page_pqstate_commit_dequeue() at _vm_page_pqstate_commit_dequeue+0x47/frame 0xfffffe00a04c43b0
          vm_page_pqstate_commit_dequeue() at vm_page_pqstate_commit_dequeue+0x7b/frame 0xfffffe00a04c43f0
          vm_page_alloc_domain_after() at vm_page_alloc_domain_after+0x242/frame 0xfffffe00a04c4460
          vm_page_alloc() at vm_page_alloc+0x64/frame 0xfffffe00a04c44c0
          vm_fault_allocate() at vm_fault_allocate+0x1d4/frame 0xfffffe00a04c4530
          vm_fault() at vm_fault+0x360/frame 0xfffffe00a04c4640
          vm_fault_trap() at vm_fault_trap+0x6d/frame 0xfffffe00a04c4690
          trap_pfault() at trap_pfault+0x1f2/frame 0xfffffe00a04c46f0
          calltrap() at calltrap+0x8/frame 0xfffffe00a04c46f0
          --- trap 0xc, rip = 0xffffffff81297056, rsp = 0xfffffe00a04c47c0, rbp = 0xfffffe00a04c47c0 ---
          copyout_nosmap_erms() at copyout_nosmap_erms+0x156/frame 0xfffffe00a04c47c0
          elf64_load_sections() at elf64_load_sections+0x265/frame 0xfffffe00a04c4860
          elf64_load_file() at elf64_load_file+0x24f/frame 0xfffffe00a04c48c0
          elf64_load_interp() at elf64_load_interp+0xff/frame 0xfffffe00a04c4910
          exec_elf64_imgact() at exec_elf64_imgact+0xe32/frame 0xfffffe00a04c4a10
          kern_execve() at kern_execve+0x658/frame 0xfffffe00a04c4d80
          sys_execve() at sys_execve+0x58/frame 0xfffffe00a04c4e00
          amd64_syscall() at amd64_syscall+0x109/frame 0xfffffe00a04c4f30
          fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00a04c4f30
          --- syscall (59, FreeBSD ELF64, execve), rip = 0x8268e4ffa, rsp = 0x820b57978, rbp = 0x820b579f0 ---
          db:0:kdb.enter.default> ps

          the third one
          db:0:kdb.enter.default> bt
          Tracing pid 84979 tid 100259 td 0xfffffe00a15a9720
          kdb_enter() at kdb_enter+0x32/frame 0xfffffe009fcb0990
          vpanic() at vpanic+0x183/frame 0xfffffe009fcb09e0
          panic() at panic+0x43/frame 0xfffffe009fcb0a40
          trap_fatal() at trap_fatal+0x409/frame 0xfffffe009fcb0aa0
          trap_pfault() at trap_pfault+0x4f/frame 0xfffffe009fcb0b00
          calltrap() at calltrap+0x8/frame 0xfffffe009fcb0b00
          --- trap 0xc, rip = 0xffffffff811616d7, rsp = 0xfffffe009fcb0bd0, rbp = 0xfffffe009fcb0bd0 ---
          _vm_page_pqstate_commit_dequeue() at _vm_page_pqstate_commit_dequeue+0x47/frame 0xfffffe009fcb0bd0
          vm_pqbatch_process_page() at vm_pqbatch_process_page+0xaf/frame 0xfffffe009fcb0c10
          vm_page_pqbatch_submit() at vm_page_pqbatch_submit+0x20f/frame 0xfffffe009fcb0c60
          vm_page_free_prep() at vm_page_free_prep+0x9c/frame 0xfffffe009fcb0c80
          vm_page_free_toq() at vm_page_free_toq+0x12/frame 0xfffffe009fcb0cb0
          vm_object_terminate() at vm_object_terminate+0x12a/frame 0xfffffe009fcb0cd0
          vm_object_deallocate() at vm_object_deallocate+0x1a8/frame 0xfffffe009fcb0d10
          vm_map_process_deferred() at vm_map_process_deferred+0x92/frame 0xfffffe009fcb0d30
          vm_map_remove() at vm_map_remove+0x100/frame 0xfffffe009fcb0d60
          vmspace_exit() at vmspace_exit+0xaa/frame 0xfffffe009fcb0d90
          exit1() at exit1+0x56c/frame 0xfffffe009fcb0df0
          sys_exit() at sys_exit+0xd/frame 0xfffffe009fcb0e00
          amd64_syscall() at amd64_syscall+0x109/frame 0xfffffe009fcb0f30
          fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe009fcb0f30
          --- syscall (1, FreeBSD ELF64, exit), rip = 0x358e7bf9e99a, rsp = 0x358e77c4b518, rbp = 0x358e77c4b530 ---
          db:0:kdb.enter.default> ps

          1 Reply Last reply Reply Quote 0
          • P
            pfsss @stephenw10
            last edited by

            @stephenw10 when I run the command ```
            pfsense-upgrade

            ~~ERROR: It was not possible to determine pkg remote version
>>> Updating repositories metadata...
Updating pfSense-core repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense-core has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense
Error updating repositories!
ERROR: It was not possible to determine pfSense-upgrade remote version
ERROR: It was not possible to determine pfSense-upgrade remote version
>>> Upgrading pfSense-upgrade... failed.~~
            1 Reply Last reply Reply Quote 0
            • stephenw10S
              stephenw10 Netgate Administrator
              last edited by

              Ok, so those crashes all appear to be in memory management. I would run a few cycles of memtest to be sure it's not bad RAM.

              Try running pfSense-upgrade -d to get more debug output.

              P 1 Reply Last reply Reply Quote 0
              • P
                pfsss @stephenw10
                last edited by

                @stephenw10 said in Kernel Panic in 2.7.0:

                I would run a few cycles of memtest to be sure it's not bad RAM

                thanks for replying. how to do memtest, is there a simple instruction to run memtest? thanks!

                after running the command pfsense-upgrade -d the output information is as follows:
                ~~ERROR: It was not possible to determine pkg remote version

                Updating repositories metadata...
                Updating pfSense-core repository catalogue...
                pkg-static: An error occured while fetching package
                pkg-static: An error occured while fetching package
                repository pfSense-core has no meta file, using default settings
                pkg-static: An error occured while fetching package
                pkg-static: An error occured while fetching package
                Unable to update repository pfSense-core
                Updating pfSense repository catalogue...
                pkg-static: An error occured while fetching package
                pkg-static: An error occured while fetching package
                repository pfSense has no meta file, using default settings
                pkg-static: An error occured while fetching package
                pkg-static: An error occured while fetching package
                Unable to update repository pfSense
                Error updating repositories!
                ERROR: It was not possible to determine pfSense-upgrade remote version
                ERROR: It was not possible to determine pfSense-upgrade remote version
                Upgrading pfSense-upgrade...
                Updating pfSense-core repository catalogue...
                pkg-static: An error occured while fetching package
                pkg-static: An error occured while fetching package
                repository pfSense-core has no meta file, using default settings
                pkg-static: An error occured while fetching package
                pkg-static: An error occured while fetching package
                Unable to update repository pfSense-core
                Updating pfSense repository catalogue...
                pkg-static: An error occured while fetching package
                pkg-static: An error occured while fetching package
                repository pfSense has no meta file, using default settings
                pkg-static: An error occured while fetching package
                pkg-static: An error occured while fetching package
                Unable to update repository pfSense
                Error updating repositories!~~

                P 1 Reply Last reply Reply Quote 0
                • P
                  pfsss @pfsss
                  last edited by

                  @pfsss

                  ERROR: It was not possible to determine pkg remote version
>>> Updating repositories metadata...
Updating pfSense-core repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense-core has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense
Error updating repositories!
ERROR: It was not possible to determine pfSense-upgrade remote version
ERROR: It was not possible to determine pfSense-upgrade remote version
>>> Upgrading pfSense-upgrade...
Updating pfSense-core repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense-core has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense
Error updating repositories!
                  hecsaH 1 Reply Last reply Reply Quote 0
                  • hecsaH
                    hecsa @pfsss
                    last edited by

                    @pfsss
                    Hi,
                    From the command line are you able to ping outside sites by name, for example to www.yahoo.com, or some known site? Looks like you don't have access to internet.
                    Thanks, and best regards,
                    HeCSa.

                    P 1 Reply Last reply Reply Quote 0
                    • P
                      pfsss @hecsa
                      last edited by

                      @hecsa hi
                      I ping yahoo.com using ping -c 5 yahoo.com , the output is

                      PING6(56=40+8+8 bytes) 2***************4 --> 2001:4998:24:120d::1:1
16 bytes from 2001:4998:24:120d::1:1, icmp_seq=0 hlim=46 time=263.204 ms
16 bytes from 2001:4998:24:120d::1:1, icmp_seq=1 hlim=47 time=263.954 ms
16 bytes from 2001:4998:24:120d::1:1, icmp_seq=2 hlim=47 time=261.079 ms
16 bytes from 2001:4998:24:120d::1:1, icmp_seq=3 hlim=46 time=259.364 ms
16 bytes from 2001:4998:24:120d::1:1, icmp_seq=4 hlim=47 time=257.564 ms

--- yahoo.com ping6 statistics ---
5 packets transmitted, 5 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 257.564/261.033/263.954/2.369 ms

                      it seems not the network problem~~

                      hecsaH 1 Reply Last reply Reply Quote 0
                      • hecsaH
                        hecsa @pfsss
                        last edited by hecsa

                        @pfsss
                        Hi,
                        Take a look at this thread, looks something like what you are experiencing: https://forum.netgate.com/topic/121783/solved-update-2-3-4-to-2-4-failed-unable-to-check-for-updates/3
                        Thanks, and best regards,
                        HeCSa.

                        P 1 Reply Last reply Reply Quote 0
                        • P
                          pfsss @hecsa
                          last edited by

                          @hecsa I tried to ls -al /var/pkg/cache there is no such directory..

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by

                            Try: pkg-static -d update

                            That should give a more useful error.

                            1 Reply Last reply Reply Quote 1
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              You probably need to run: certctl rehash

                              P H 2 Replies Last reply Reply Quote 1
                              • P
                                pfsss @stephenw10
                                last edited by

                                @stephenw10 hi
                                after running command certctl rehash and png-static -d update I get the following hints

                                Newer FreeBSD version for package xxd:
To ignore this error set IGNORE_OSVERSION=yes
- package: 1400094
- running kernel: 1400085
Ignore the mismatch and continue? [y/N]: pkg-static: repository pfSense contains packages for wrong OS version: FreeBSD:14:amd64
Processing entries... done
Unable to update repository pfSense
Error updating repositories!

                                I think it is close to the right way

                                1 Reply Last reply Reply Quote 0
                                • stephenw10S
                                  stephenw10 Netgate Administrator
                                  last edited by

                                  Yes, it must be seeing the pkgs to be able to throw that error so try to upgrade again.

                                  P 1 Reply Last reply Reply Quote 0
                                  • P
                                    pfsss @stephenw10
                                    last edited by

                                    @stephenw10
                                    ok, I'm upgrading the system now. I wonder whether upgrading the system affects the packages I've installed. I have a Wireguard package installed in the 2.7.0 version os, after upgrading, it should be ok or I will reconfigured it? thanks~

                                    1 Reply Last reply Reply Quote 0
                                    • stephenw10S
                                      stephenw10 Netgate Administrator
                                      last edited by

                                      Should be fine. The packages will be upgraded and the config retained.

                                      P 3 Replies Last reply Reply Quote 0
                                      • P
                                        pfsss @stephenw10
                                        last edited by

                                        @stephenw10
                                        as your previous replys, how to do memtest, is there a simple instruction to run memtest? thanks

                                        1 Reply Last reply Reply Quote 0
                                        • P
                                          pfsss @stephenw10
                                          last edited by

                                          @stephenw10 thanks to your guide, I've upgrade my os to 2.7.2 successfully! I'll watch whether the crash occurs again.

                                          1 Reply Last reply Reply Quote 1
                                          • P
                                            pfsss @stephenw10
                                            last edited by

                                            @stephenw10 sadly, the gateway crashes again few hours after upgrade. the crash information is

                                            Fatal trap 12: page fault while in kernel mode
cpuid = 2; apic id = 04
fault virtual address	= 0xfffffe00483bdc90
fault code		= supervisor write data, page not present
instruction pointer	= 0x20:0xffffffff81159b45
stack pointer	        = 0x28:0xfffffe00862abc08
frame pointer	        = 0x28:0xfffffe00862abc10
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 54985 (netstat)
rdi: ffffffff830dbf00 rsi: fffffe00083bdc20 rdx: 0000000000000001
rcx: 0000000005010018  r8: ffffffff830dbf28  r9: 0000000005ff0008
rax: fffffe0001611c50 rbx: fffffe00483bdc90 rbp: fffffe00862abc10
r10: fffffe00483bdc90 r11: fffffe00483bdc88 r12: ffffffff830dbf00
r13: fffffe00098fd400 r14: 0000000000000001 r15: 0000000000000006
trap number		= 12
panic: page fault
cpuid = 2
time = 1704523671
KDB: enter: panic

                                            bt is

                                            db:0:kdb.enter.default>  bt
Tracing pid 54985 tid 100246 td 0xfffffe006cc34020
kdb_enter() at kdb_enter+0x32/frame 0xfffffe00862ab8e0
vpanic() at vpanic+0x163/frame 0xfffffe00862aba10
panic() at panic+0x43/frame 0xfffffe00862aba70
trap_fatal() at trap_fatal+0x40c/frame 0xfffffe00862abad0
trap_pfault() at trap_pfault+0x4f/frame 0xfffffe00862abb30
calltrap() at calltrap+0x8/frame 0xfffffe00862abb30
--- trap 0xc, rip = 0xffffffff81159b45, rsp = 0xfffffe00862abc08, rbp = 0xfffffe00862abc10 ---
vm_pqbatch_process_page() at vm_pqbatch_process_page+0xb5/frame 0xfffffe00862abc10
vm_page_pqbatch_submit() at vm_page_pqbatch_submit+0x200/frame 0xfffffe00862abc60
vm_page_free_prep() at vm_page_free_prep+0x9c/frame 0xfffffe00862abc80
vm_page_free_toq() at vm_page_free_toq+0x12/frame 0xfffffe00862abcb0
vm_object_terminate() at vm_object_terminate+0x11a/frame 0xfffffe00862abcd0
vm_object_deallocate() at vm_object_deallocate+0x1af/frame 0xfffffe00862abd10
vm_map_process_deferred() at vm_map_process_deferred+0x92/frame 0xfffffe00862abd30
vm_map_remove() at vm_map_remove+0xff/frame 0xfffffe00862abd60
vmspace_exit() at vmspace_exit+0xab/frame 0xfffffe00862abd90
exit1() at exit1+0x53a/frame 0xfffffe00862abdf0
sys_exit() at sys_exit+0xd/frame 0xfffffe00862abe00
amd64_syscall() at amd64_syscall+0x109/frame 0xfffffe00862abf30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00862abf30
--- syscall (1, FreeBSD ELF64, exit), rip = 0x33c88f91140a, rsp = 0x33c88b1d4378, rbp = 0x33c88b1d4390 ---
db:0:kdb.enter.default>  ps
                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.