Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Kernel Panic in 2.7.0

    Scheduled Pinned Locked Moved General pfSense Questions
    29 Posts 4 Posters 3.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • P
      pfsss @stephenw10
      last edited by

      @stephenw10 here are some crash bt
      one crash:
      db:0:kdb.enter.default> bt
      Tracing pid 9 tid 100125 td 0xfffffe006b90f900
      kdb_enter() at kdb_enter+0x32/frame 0xfffffe006af05810
      vpanic() at vpanic+0x183/frame 0xfffffe006af05860
      panic() at panic+0x43/frame 0xfffffe006af058c0
      trap_fatal() at trap_fatal+0x409/frame 0xfffffe006af05920
      trap_pfault() at trap_pfault+0x4f/frame 0xfffffe006af05980
      calltrap() at calltrap+0x8/frame 0xfffffe006af05980
      --- trap 0xc, rip = 0xffffffff80cd2552, rsp = 0xfffffe006af05a50, rbp = 0xfffffe006af05a60 ---
      mtx_trylock_flags() at mtx_trylock_flags+0x32/frame 0xfffffe006af05a60
      pmap_ts_referenced() at pmap_ts_referenced+0x5d5/frame 0xfffffe006af05af0
      vm_pageout_worker() at vm_pageout_worker+0xb21/frame 0xfffffe006af05eb0
      vm_pageout() at vm_pageout+0x1c7/frame 0xfffffe006af05ef0
      fork_exit() at fork_exit+0x7d/frame 0xfffffe006af05f30
      fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe006af05f30
      --- trap 0xb96e72dc, rip = 0xffffffffffffffff, rsp = 0xffffffffffffffff, rbp = 0xfdf9fbf3f7e6efcc ---
      the other one
      db:0:kdb.enter.default> bt
      Tracing pid 17611 tid 100279 td 0xfffffe00a14413a0
      kdb_enter() at kdb_enter+0x32/frame 0xfffffe00a04c4070
      vpanic() at vpanic+0x183/frame 0xfffffe00a04c40c0
      panic() at panic+0x43/frame 0xfffffe00a04c4120
      vm_fault() at vm_fault+0x1539/frame 0xfffffe00a04c4230
      vm_fault_trap() at vm_fault_trap+0xb1/frame 0xfffffe00a04c4280
      trap_pfault() at trap_pfault+0x1f2/frame 0xfffffe00a04c42e0
      calltrap() at calltrap+0x8/frame 0xfffffe00a04c42e0
      --- trap 0xc, rip = 0xffffffff811616d7, rsp = 0xfffffe00a04c43b0, rbp = 0xfffffe00a04c43b0 ---
      _vm_page_pqstate_commit_dequeue() at _vm_page_pqstate_commit_dequeue+0x47/frame 0xfffffe00a04c43b0
      vm_page_pqstate_commit_dequeue() at vm_page_pqstate_commit_dequeue+0x7b/frame 0xfffffe00a04c43f0
      vm_page_alloc_domain_after() at vm_page_alloc_domain_after+0x242/frame 0xfffffe00a04c4460
      vm_page_alloc() at vm_page_alloc+0x64/frame 0xfffffe00a04c44c0
      vm_fault_allocate() at vm_fault_allocate+0x1d4/frame 0xfffffe00a04c4530
      vm_fault() at vm_fault+0x360/frame 0xfffffe00a04c4640
      vm_fault_trap() at vm_fault_trap+0x6d/frame 0xfffffe00a04c4690
      trap_pfault() at trap_pfault+0x1f2/frame 0xfffffe00a04c46f0
      calltrap() at calltrap+0x8/frame 0xfffffe00a04c46f0
      --- trap 0xc, rip = 0xffffffff81297056, rsp = 0xfffffe00a04c47c0, rbp = 0xfffffe00a04c47c0 ---
      copyout_nosmap_erms() at copyout_nosmap_erms+0x156/frame 0xfffffe00a04c47c0
      elf64_load_sections() at elf64_load_sections+0x265/frame 0xfffffe00a04c4860
      elf64_load_file() at elf64_load_file+0x24f/frame 0xfffffe00a04c48c0
      elf64_load_interp() at elf64_load_interp+0xff/frame 0xfffffe00a04c4910
      exec_elf64_imgact() at exec_elf64_imgact+0xe32/frame 0xfffffe00a04c4a10
      kern_execve() at kern_execve+0x658/frame 0xfffffe00a04c4d80
      sys_execve() at sys_execve+0x58/frame 0xfffffe00a04c4e00
      amd64_syscall() at amd64_syscall+0x109/frame 0xfffffe00a04c4f30
      fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00a04c4f30
      --- syscall (59, FreeBSD ELF64, execve), rip = 0x8268e4ffa, rsp = 0x820b57978, rbp = 0x820b579f0 ---
      db:0:kdb.enter.default> ps

      the third one
      db:0:kdb.enter.default> bt
      Tracing pid 84979 tid 100259 td 0xfffffe00a15a9720
      kdb_enter() at kdb_enter+0x32/frame 0xfffffe009fcb0990
      vpanic() at vpanic+0x183/frame 0xfffffe009fcb09e0
      panic() at panic+0x43/frame 0xfffffe009fcb0a40
      trap_fatal() at trap_fatal+0x409/frame 0xfffffe009fcb0aa0
      trap_pfault() at trap_pfault+0x4f/frame 0xfffffe009fcb0b00
      calltrap() at calltrap+0x8/frame 0xfffffe009fcb0b00
      --- trap 0xc, rip = 0xffffffff811616d7, rsp = 0xfffffe009fcb0bd0, rbp = 0xfffffe009fcb0bd0 ---
      _vm_page_pqstate_commit_dequeue() at _vm_page_pqstate_commit_dequeue+0x47/frame 0xfffffe009fcb0bd0
      vm_pqbatch_process_page() at vm_pqbatch_process_page+0xaf/frame 0xfffffe009fcb0c10
      vm_page_pqbatch_submit() at vm_page_pqbatch_submit+0x20f/frame 0xfffffe009fcb0c60
      vm_page_free_prep() at vm_page_free_prep+0x9c/frame 0xfffffe009fcb0c80
      vm_page_free_toq() at vm_page_free_toq+0x12/frame 0xfffffe009fcb0cb0
      vm_object_terminate() at vm_object_terminate+0x12a/frame 0xfffffe009fcb0cd0
      vm_object_deallocate() at vm_object_deallocate+0x1a8/frame 0xfffffe009fcb0d10
      vm_map_process_deferred() at vm_map_process_deferred+0x92/frame 0xfffffe009fcb0d30
      vm_map_remove() at vm_map_remove+0x100/frame 0xfffffe009fcb0d60
      vmspace_exit() at vmspace_exit+0xaa/frame 0xfffffe009fcb0d90
      exit1() at exit1+0x56c/frame 0xfffffe009fcb0df0
      sys_exit() at sys_exit+0xd/frame 0xfffffe009fcb0e00
      amd64_syscall() at amd64_syscall+0x109/frame 0xfffffe009fcb0f30
      fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe009fcb0f30
      --- syscall (1, FreeBSD ELF64, exit), rip = 0x358e7bf9e99a, rsp = 0x358e77c4b518, rbp = 0x358e77c4b530 ---
      db:0:kdb.enter.default> ps

      1 Reply Last reply Reply Quote 0
      • P
        pfsss @stephenw10
        last edited by

        @stephenw10 when I run the command ```
        pfsense-upgrade

        ~~ERROR: It was not possible to determine pkg remote version
>>> Updating repositories metadata...
Updating pfSense-core repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense-core has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense
Error updating repositories!
ERROR: It was not possible to determine pfSense-upgrade remote version
ERROR: It was not possible to determine pfSense-upgrade remote version
>>> Upgrading pfSense-upgrade... failed.~~
        1 Reply Last reply Reply Quote 0
        • stephenw10S
          stephenw10 Netgate Administrator
          last edited by

          Ok, so those crashes all appear to be in memory management. I would run a few cycles of memtest to be sure it's not bad RAM.

          Try running pfSense-upgrade -d to get more debug output.

          P 1 Reply Last reply Reply Quote 0
          • P
            pfsss @stephenw10
            last edited by

            @stephenw10 said in Kernel Panic in 2.7.0:

            I would run a few cycles of memtest to be sure it's not bad RAM

            thanks for replying. how to do memtest, is there a simple instruction to run memtest? thanks!

            after running the command pfsense-upgrade -d the output information is as follows:
            ~~ERROR: It was not possible to determine pkg remote version

            Updating repositories metadata...
            Updating pfSense-core repository catalogue...
            pkg-static: An error occured while fetching package
            pkg-static: An error occured while fetching package
            repository pfSense-core has no meta file, using default settings
            pkg-static: An error occured while fetching package
            pkg-static: An error occured while fetching package
            Unable to update repository pfSense-core
            Updating pfSense repository catalogue...
            pkg-static: An error occured while fetching package
            pkg-static: An error occured while fetching package
            repository pfSense has no meta file, using default settings
            pkg-static: An error occured while fetching package
            pkg-static: An error occured while fetching package
            Unable to update repository pfSense
            Error updating repositories!
            ERROR: It was not possible to determine pfSense-upgrade remote version
            ERROR: It was not possible to determine pfSense-upgrade remote version
            Upgrading pfSense-upgrade...
            Updating pfSense-core repository catalogue...
            pkg-static: An error occured while fetching package
            pkg-static: An error occured while fetching package
            repository pfSense-core has no meta file, using default settings
            pkg-static: An error occured while fetching package
            pkg-static: An error occured while fetching package
            Unable to update repository pfSense-core
            Updating pfSense repository catalogue...
            pkg-static: An error occured while fetching package
            pkg-static: An error occured while fetching package
            repository pfSense has no meta file, using default settings
            pkg-static: An error occured while fetching package
            pkg-static: An error occured while fetching package
            Unable to update repository pfSense
            Error updating repositories!~~

            P 1 Reply Last reply Reply Quote 0
            • P
              pfsss @pfsss
              last edited by

              @pfsss

              ERROR: It was not possible to determine pkg remote version
>>> Updating repositories metadata...
Updating pfSense-core repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense-core has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense
Error updating repositories!
ERROR: It was not possible to determine pfSense-upgrade remote version
ERROR: It was not possible to determine pfSense-upgrade remote version
>>> Upgrading pfSense-upgrade...
Updating pfSense-core repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense-core has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense
Error updating repositories!
              hecsaH 1 Reply Last reply Reply Quote 0
              • hecsaH
                hecsa @pfsss
                last edited by

                @pfsss
                Hi,
                From the command line are you able to ping outside sites by name, for example to www.yahoo.com, or some known site? Looks like you don't have access to internet.
                Thanks, and best regards,
                HeCSa.

                P 1 Reply Last reply Reply Quote 0
                • P
                  pfsss @hecsa
                  last edited by

                  @hecsa hi
                  I ping yahoo.com using ping -c 5 yahoo.com , the output is

                  PING6(56=40+8+8 bytes) 2***************4 --> 2001:4998:24:120d::1:1
16 bytes from 2001:4998:24:120d::1:1, icmp_seq=0 hlim=46 time=263.204 ms
16 bytes from 2001:4998:24:120d::1:1, icmp_seq=1 hlim=47 time=263.954 ms
16 bytes from 2001:4998:24:120d::1:1, icmp_seq=2 hlim=47 time=261.079 ms
16 bytes from 2001:4998:24:120d::1:1, icmp_seq=3 hlim=46 time=259.364 ms
16 bytes from 2001:4998:24:120d::1:1, icmp_seq=4 hlim=47 time=257.564 ms

--- yahoo.com ping6 statistics ---
5 packets transmitted, 5 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 257.564/261.033/263.954/2.369 ms

                  it seems not the network problem~~

                  hecsaH 1 Reply Last reply Reply Quote 0
                  • hecsaH
                    hecsa @pfsss
                    last edited by hecsa

                    @pfsss
                    Hi,
                    Take a look at this thread, looks something like what you are experiencing: https://forum.netgate.com/topic/121783/solved-update-2-3-4-to-2-4-failed-unable-to-check-for-updates/3
                    Thanks, and best regards,
                    HeCSa.

                    P 1 Reply Last reply Reply Quote 0
                    • P
                      pfsss @hecsa
                      last edited by

                      @hecsa I tried to ls -al /var/pkg/cache there is no such directory..

                      1 Reply Last reply Reply Quote 0
                      • stephenw10S
                        stephenw10 Netgate Administrator
                        last edited by

                        Try: pkg-static -d update

                        That should give a more useful error.

                        1 Reply Last reply Reply Quote 1
                        • stephenw10S
                          stephenw10 Netgate Administrator
                          last edited by

                          You probably need to run: certctl rehash

                          P H 2 Replies Last reply Reply Quote 1
                          • P
                            pfsss @stephenw10
                            last edited by

                            @stephenw10 hi
                            after running command certctl rehash and png-static -d update I get the following hints

                            Newer FreeBSD version for package xxd:
To ignore this error set IGNORE_OSVERSION=yes
- package: 1400094
- running kernel: 1400085
Ignore the mismatch and continue? [y/N]: pkg-static: repository pfSense contains packages for wrong OS version: FreeBSD:14:amd64
Processing entries... done
Unable to update repository pfSense
Error updating repositories!

                            I think it is close to the right way

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              Yes, it must be seeing the pkgs to be able to throw that error so try to upgrade again.

                              P 1 Reply Last reply Reply Quote 0
                              • P
                                pfsss @stephenw10
                                last edited by

                                @stephenw10
                                ok, I'm upgrading the system now. I wonder whether upgrading the system affects the packages I've installed. I have a Wireguard package installed in the 2.7.0 version os, after upgrading, it should be ok or I will reconfigured it? thanks~

                                1 Reply Last reply Reply Quote 0
                                • stephenw10S
                                  stephenw10 Netgate Administrator
                                  last edited by

                                  Should be fine. The packages will be upgraded and the config retained.

                                  P 3 Replies Last reply Reply Quote 0
                                  • P
                                    pfsss @stephenw10
                                    last edited by

                                    @stephenw10
                                    as your previous replys, how to do memtest, is there a simple instruction to run memtest? thanks

                                    1 Reply Last reply Reply Quote 0
                                    • P
                                      pfsss @stephenw10
                                      last edited by

                                      @stephenw10 thanks to your guide, I've upgrade my os to 2.7.2 successfully! I'll watch whether the crash occurs again.

                                      1 Reply Last reply Reply Quote 1
                                      • P
                                        pfsss @stephenw10
                                        last edited by

                                        @stephenw10 sadly, the gateway crashes again few hours after upgrade. the crash information is

                                        Fatal trap 12: page fault while in kernel mode
cpuid = 2; apic id = 04
fault virtual address	= 0xfffffe00483bdc90
fault code		= supervisor write data, page not present
instruction pointer	= 0x20:0xffffffff81159b45
stack pointer	        = 0x28:0xfffffe00862abc08
frame pointer	        = 0x28:0xfffffe00862abc10
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 54985 (netstat)
rdi: ffffffff830dbf00 rsi: fffffe00083bdc20 rdx: 0000000000000001
rcx: 0000000005010018  r8: ffffffff830dbf28  r9: 0000000005ff0008
rax: fffffe0001611c50 rbx: fffffe00483bdc90 rbp: fffffe00862abc10
r10: fffffe00483bdc90 r11: fffffe00483bdc88 r12: ffffffff830dbf00
r13: fffffe00098fd400 r14: 0000000000000001 r15: 0000000000000006
trap number		= 12
panic: page fault
cpuid = 2
time = 1704523671
KDB: enter: panic

                                        bt is

                                        db:0:kdb.enter.default>  bt
Tracing pid 54985 tid 100246 td 0xfffffe006cc34020
kdb_enter() at kdb_enter+0x32/frame 0xfffffe00862ab8e0
vpanic() at vpanic+0x163/frame 0xfffffe00862aba10
panic() at panic+0x43/frame 0xfffffe00862aba70
trap_fatal() at trap_fatal+0x40c/frame 0xfffffe00862abad0
trap_pfault() at trap_pfault+0x4f/frame 0xfffffe00862abb30
calltrap() at calltrap+0x8/frame 0xfffffe00862abb30
--- trap 0xc, rip = 0xffffffff81159b45, rsp = 0xfffffe00862abc08, rbp = 0xfffffe00862abc10 ---
vm_pqbatch_process_page() at vm_pqbatch_process_page+0xb5/frame 0xfffffe00862abc10
vm_page_pqbatch_submit() at vm_page_pqbatch_submit+0x200/frame 0xfffffe00862abc60
vm_page_free_prep() at vm_page_free_prep+0x9c/frame 0xfffffe00862abc80
vm_page_free_toq() at vm_page_free_toq+0x12/frame 0xfffffe00862abcb0
vm_object_terminate() at vm_object_terminate+0x11a/frame 0xfffffe00862abcd0
vm_object_deallocate() at vm_object_deallocate+0x1af/frame 0xfffffe00862abd10
vm_map_process_deferred() at vm_map_process_deferred+0x92/frame 0xfffffe00862abd30
vm_map_remove() at vm_map_remove+0xff/frame 0xfffffe00862abd60
vmspace_exit() at vmspace_exit+0xab/frame 0xfffffe00862abd90
exit1() at exit1+0x53a/frame 0xfffffe00862abdf0
sys_exit() at sys_exit+0xd/frame 0xfffffe00862abe00
amd64_syscall() at amd64_syscall+0x109/frame 0xfffffe00862abf30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00862abf30
--- syscall (1, FreeBSD ELF64, exit), rip = 0x33c88f91140a, rsp = 0x33c88b1d4378, rbp = 0x33c88b1d4390 ---
db:0:kdb.enter.default>  ps
                                        1 Reply Last reply Reply Quote 0
                                        • stephenw10S
                                          stephenw10 Netgate Administrator
                                          last edited by

                                          Hmm, same crash too. I would definitely run some passes through memtest to check the RAM.

                                          P 1 Reply Last reply Reply Quote 0
                                          • P
                                            pfsss @stephenw10
                                            last edited by

                                            @stephenw10 I've run a memtest to the memory, report this
                                            IMG_8772 Large.jpeg
                                            it is actually bad RAM!
                                            thanks for your guides!

                                            1 Reply Last reply Reply Quote 1
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.