Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Kernel Panic in 2.7.0

    Scheduled Pinned Locked Moved General pfSense Questions
    29 Posts 4 Posters 3.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • stephenw10S
      stephenw10 Netgate Administrator
      last edited by

      I forked this to a new topic because that's a completely different crash.

      Did it just start happening? Was it stable in 2.6? Any reason you are not running 2.7.2?

      P 2 Replies Last reply Reply Quote 0
      • P
        pfsss @stephenw10
        last edited by

        @stephenw10 I installed 2.7.0 in this x86 machine and not tested in 2.6.0 . this happens stochastic, sometimes few hours.

        1 Reply Last reply Reply Quote 0
        • P
          pfsss @stephenw10
          last edited by

          @stephenw10 I'd be like to update 2.7.2 but when I check for updates shows this
          Version 2.7.0-RELEASE (amd64)
          built on Wed Jun 28 03:53:34 UTC 2023
          FreeBSD 14.0-CURRENT

          The system is on the latest version.
          Version information updated at Thu Jan 4 21:58:57 +08 2024

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Ok, so it has always crashed in 2.7.0 on that hardware?

            The crash itself could well be a bad RAM. Do the crashes all look the same? Same backtrace?

            You can install 2.7.2 directly. Otherwise make sure latest version is set in System > Update. Try running pfSense-upgrade at the CLI and see wha error is shown.

            P 3 Replies Last reply Reply Quote 0
            • P
              pfsss @stephenw10
              last edited by

              @stephenw10 the web ui upgrade is as follow59e5ff73-1435-4e8b-8f8e-48f97e9f9bb6-image.png

              no button to confirm upgrade

              1 Reply Last reply Reply Quote 0
              • P
                pfsss @stephenw10
                last edited by

                @stephenw10 here are some crash bt
                one crash:
                db:0:kdb.enter.default> bt
                Tracing pid 9 tid 100125 td 0xfffffe006b90f900
                kdb_enter() at kdb_enter+0x32/frame 0xfffffe006af05810
                vpanic() at vpanic+0x183/frame 0xfffffe006af05860
                panic() at panic+0x43/frame 0xfffffe006af058c0
                trap_fatal() at trap_fatal+0x409/frame 0xfffffe006af05920
                trap_pfault() at trap_pfault+0x4f/frame 0xfffffe006af05980
                calltrap() at calltrap+0x8/frame 0xfffffe006af05980
                --- trap 0xc, rip = 0xffffffff80cd2552, rsp = 0xfffffe006af05a50, rbp = 0xfffffe006af05a60 ---
                mtx_trylock_flags() at mtx_trylock_flags+0x32/frame 0xfffffe006af05a60
                pmap_ts_referenced() at pmap_ts_referenced+0x5d5/frame 0xfffffe006af05af0
                vm_pageout_worker() at vm_pageout_worker+0xb21/frame 0xfffffe006af05eb0
                vm_pageout() at vm_pageout+0x1c7/frame 0xfffffe006af05ef0
                fork_exit() at fork_exit+0x7d/frame 0xfffffe006af05f30
                fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe006af05f30
                --- trap 0xb96e72dc, rip = 0xffffffffffffffff, rsp = 0xffffffffffffffff, rbp = 0xfdf9fbf3f7e6efcc ---
                the other one
                db:0:kdb.enter.default> bt
                Tracing pid 17611 tid 100279 td 0xfffffe00a14413a0
                kdb_enter() at kdb_enter+0x32/frame 0xfffffe00a04c4070
                vpanic() at vpanic+0x183/frame 0xfffffe00a04c40c0
                panic() at panic+0x43/frame 0xfffffe00a04c4120
                vm_fault() at vm_fault+0x1539/frame 0xfffffe00a04c4230
                vm_fault_trap() at vm_fault_trap+0xb1/frame 0xfffffe00a04c4280
                trap_pfault() at trap_pfault+0x1f2/frame 0xfffffe00a04c42e0
                calltrap() at calltrap+0x8/frame 0xfffffe00a04c42e0
                --- trap 0xc, rip = 0xffffffff811616d7, rsp = 0xfffffe00a04c43b0, rbp = 0xfffffe00a04c43b0 ---
                _vm_page_pqstate_commit_dequeue() at _vm_page_pqstate_commit_dequeue+0x47/frame 0xfffffe00a04c43b0
                vm_page_pqstate_commit_dequeue() at vm_page_pqstate_commit_dequeue+0x7b/frame 0xfffffe00a04c43f0
                vm_page_alloc_domain_after() at vm_page_alloc_domain_after+0x242/frame 0xfffffe00a04c4460
                vm_page_alloc() at vm_page_alloc+0x64/frame 0xfffffe00a04c44c0
                vm_fault_allocate() at vm_fault_allocate+0x1d4/frame 0xfffffe00a04c4530
                vm_fault() at vm_fault+0x360/frame 0xfffffe00a04c4640
                vm_fault_trap() at vm_fault_trap+0x6d/frame 0xfffffe00a04c4690
                trap_pfault() at trap_pfault+0x1f2/frame 0xfffffe00a04c46f0
                calltrap() at calltrap+0x8/frame 0xfffffe00a04c46f0
                --- trap 0xc, rip = 0xffffffff81297056, rsp = 0xfffffe00a04c47c0, rbp = 0xfffffe00a04c47c0 ---
                copyout_nosmap_erms() at copyout_nosmap_erms+0x156/frame 0xfffffe00a04c47c0
                elf64_load_sections() at elf64_load_sections+0x265/frame 0xfffffe00a04c4860
                elf64_load_file() at elf64_load_file+0x24f/frame 0xfffffe00a04c48c0
                elf64_load_interp() at elf64_load_interp+0xff/frame 0xfffffe00a04c4910
                exec_elf64_imgact() at exec_elf64_imgact+0xe32/frame 0xfffffe00a04c4a10
                kern_execve() at kern_execve+0x658/frame 0xfffffe00a04c4d80
                sys_execve() at sys_execve+0x58/frame 0xfffffe00a04c4e00
                amd64_syscall() at amd64_syscall+0x109/frame 0xfffffe00a04c4f30
                fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00a04c4f30
                --- syscall (59, FreeBSD ELF64, execve), rip = 0x8268e4ffa, rsp = 0x820b57978, rbp = 0x820b579f0 ---
                db:0:kdb.enter.default> ps

                the third one
                db:0:kdb.enter.default> bt
                Tracing pid 84979 tid 100259 td 0xfffffe00a15a9720
                kdb_enter() at kdb_enter+0x32/frame 0xfffffe009fcb0990
                vpanic() at vpanic+0x183/frame 0xfffffe009fcb09e0
                panic() at panic+0x43/frame 0xfffffe009fcb0a40
                trap_fatal() at trap_fatal+0x409/frame 0xfffffe009fcb0aa0
                trap_pfault() at trap_pfault+0x4f/frame 0xfffffe009fcb0b00
                calltrap() at calltrap+0x8/frame 0xfffffe009fcb0b00
                --- trap 0xc, rip = 0xffffffff811616d7, rsp = 0xfffffe009fcb0bd0, rbp = 0xfffffe009fcb0bd0 ---
                _vm_page_pqstate_commit_dequeue() at _vm_page_pqstate_commit_dequeue+0x47/frame 0xfffffe009fcb0bd0
                vm_pqbatch_process_page() at vm_pqbatch_process_page+0xaf/frame 0xfffffe009fcb0c10
                vm_page_pqbatch_submit() at vm_page_pqbatch_submit+0x20f/frame 0xfffffe009fcb0c60
                vm_page_free_prep() at vm_page_free_prep+0x9c/frame 0xfffffe009fcb0c80
                vm_page_free_toq() at vm_page_free_toq+0x12/frame 0xfffffe009fcb0cb0
                vm_object_terminate() at vm_object_terminate+0x12a/frame 0xfffffe009fcb0cd0
                vm_object_deallocate() at vm_object_deallocate+0x1a8/frame 0xfffffe009fcb0d10
                vm_map_process_deferred() at vm_map_process_deferred+0x92/frame 0xfffffe009fcb0d30
                vm_map_remove() at vm_map_remove+0x100/frame 0xfffffe009fcb0d60
                vmspace_exit() at vmspace_exit+0xaa/frame 0xfffffe009fcb0d90
                exit1() at exit1+0x56c/frame 0xfffffe009fcb0df0
                sys_exit() at sys_exit+0xd/frame 0xfffffe009fcb0e00
                amd64_syscall() at amd64_syscall+0x109/frame 0xfffffe009fcb0f30
                fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe009fcb0f30
                --- syscall (1, FreeBSD ELF64, exit), rip = 0x358e7bf9e99a, rsp = 0x358e77c4b518, rbp = 0x358e77c4b530 ---
                db:0:kdb.enter.default> ps

                1 Reply Last reply Reply Quote 0
                • P
                  pfsss @stephenw10
                  last edited by

                  @stephenw10 when I run the command ```
                  pfsense-upgrade

                  ~~ERROR: It was not possible to determine pkg remote version
>>> Updating repositories metadata...
Updating pfSense-core repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense-core has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense
Error updating repositories!
ERROR: It was not possible to determine pfSense-upgrade remote version
ERROR: It was not possible to determine pfSense-upgrade remote version
>>> Upgrading pfSense-upgrade... failed.~~
                  1 Reply Last reply Reply Quote 0
                  • stephenw10S
                    stephenw10 Netgate Administrator
                    last edited by

                    Ok, so those crashes all appear to be in memory management. I would run a few cycles of memtest to be sure it's not bad RAM.

                    Try running pfSense-upgrade -d to get more debug output.

                    P 1 Reply Last reply Reply Quote 0
                    • P
                      pfsss @stephenw10
                      last edited by

                      @stephenw10 said in Kernel Panic in 2.7.0:

                      I would run a few cycles of memtest to be sure it's not bad RAM

                      thanks for replying. how to do memtest, is there a simple instruction to run memtest? thanks!

                      after running the command pfsense-upgrade -d the output information is as follows:
                      ~~ERROR: It was not possible to determine pkg remote version

                      Updating repositories metadata...
                      Updating pfSense-core repository catalogue...
                      pkg-static: An error occured while fetching package
                      pkg-static: An error occured while fetching package
                      repository pfSense-core has no meta file, using default settings
                      pkg-static: An error occured while fetching package
                      pkg-static: An error occured while fetching package
                      Unable to update repository pfSense-core
                      Updating pfSense repository catalogue...
                      pkg-static: An error occured while fetching package
                      pkg-static: An error occured while fetching package
                      repository pfSense has no meta file, using default settings
                      pkg-static: An error occured while fetching package
                      pkg-static: An error occured while fetching package
                      Unable to update repository pfSense
                      Error updating repositories!
                      ERROR: It was not possible to determine pfSense-upgrade remote version
                      ERROR: It was not possible to determine pfSense-upgrade remote version
                      Upgrading pfSense-upgrade...
                      Updating pfSense-core repository catalogue...
                      pkg-static: An error occured while fetching package
                      pkg-static: An error occured while fetching package
                      repository pfSense-core has no meta file, using default settings
                      pkg-static: An error occured while fetching package
                      pkg-static: An error occured while fetching package
                      Unable to update repository pfSense-core
                      Updating pfSense repository catalogue...
                      pkg-static: An error occured while fetching package
                      pkg-static: An error occured while fetching package
                      repository pfSense has no meta file, using default settings
                      pkg-static: An error occured while fetching package
                      pkg-static: An error occured while fetching package
                      Unable to update repository pfSense
                      Error updating repositories!~~

                      P 1 Reply Last reply Reply Quote 0
                      • P
                        pfsss @pfsss
                        last edited by

                        @pfsss

                        ERROR: It was not possible to determine pkg remote version
>>> Updating repositories metadata...
Updating pfSense-core repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense-core has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense
Error updating repositories!
ERROR: It was not possible to determine pfSense-upgrade remote version
ERROR: It was not possible to determine pfSense-upgrade remote version
>>> Upgrading pfSense-upgrade...
Updating pfSense-core repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense-core has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
repository pfSense has no meta file, using default settings
pkg-static: An error occured while fetching package
pkg-static: An error occured while fetching package
Unable to update repository pfSense
Error updating repositories!
                        hecsaH 1 Reply Last reply Reply Quote 0
                        • hecsaH
                          hecsa @pfsss
                          last edited by

                          @pfsss
                          Hi,
                          From the command line are you able to ping outside sites by name, for example to www.yahoo.com, or some known site? Looks like you don't have access to internet.
                          Thanks, and best regards,
                          HeCSa.

                          P 1 Reply Last reply Reply Quote 0
                          • P
                            pfsss @hecsa
                            last edited by

                            @hecsa hi
                            I ping yahoo.com using ping -c 5 yahoo.com , the output is

                            PING6(56=40+8+8 bytes) 2***************4 --> 2001:4998:24:120d::1:1
16 bytes from 2001:4998:24:120d::1:1, icmp_seq=0 hlim=46 time=263.204 ms
16 bytes from 2001:4998:24:120d::1:1, icmp_seq=1 hlim=47 time=263.954 ms
16 bytes from 2001:4998:24:120d::1:1, icmp_seq=2 hlim=47 time=261.079 ms
16 bytes from 2001:4998:24:120d::1:1, icmp_seq=3 hlim=46 time=259.364 ms
16 bytes from 2001:4998:24:120d::1:1, icmp_seq=4 hlim=47 time=257.564 ms

--- yahoo.com ping6 statistics ---
5 packets transmitted, 5 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 257.564/261.033/263.954/2.369 ms

                            it seems not the network problem~~

                            hecsaH 1 Reply Last reply Reply Quote 0
                            • hecsaH
                              hecsa @pfsss
                              last edited by hecsa

                              @pfsss
                              Hi,
                              Take a look at this thread, looks something like what you are experiencing: https://forum.netgate.com/topic/121783/solved-update-2-3-4-to-2-4-failed-unable-to-check-for-updates/3
                              Thanks, and best regards,
                              HeCSa.

                              P 1 Reply Last reply Reply Quote 0
                              • P
                                pfsss @hecsa
                                last edited by

                                @hecsa I tried to ls -al /var/pkg/cache there is no such directory..

                                1 Reply Last reply Reply Quote 0
                                • stephenw10S
                                  stephenw10 Netgate Administrator
                                  last edited by

                                  Try: pkg-static -d update

                                  That should give a more useful error.

                                  1 Reply Last reply Reply Quote 1
                                  • stephenw10S
                                    stephenw10 Netgate Administrator
                                    last edited by

                                    You probably need to run: certctl rehash

                                    P H 2 Replies Last reply Reply Quote 1
                                    • P
                                      pfsss @stephenw10
                                      last edited by

                                      @stephenw10 hi
                                      after running command certctl rehash and png-static -d update I get the following hints

                                      Newer FreeBSD version for package xxd:
To ignore this error set IGNORE_OSVERSION=yes
- package: 1400094
- running kernel: 1400085
Ignore the mismatch and continue? [y/N]: pkg-static: repository pfSense contains packages for wrong OS version: FreeBSD:14:amd64
Processing entries... done
Unable to update repository pfSense
Error updating repositories!

                                      I think it is close to the right way

                                      1 Reply Last reply Reply Quote 0
                                      • stephenw10S
                                        stephenw10 Netgate Administrator
                                        last edited by

                                        Yes, it must be seeing the pkgs to be able to throw that error so try to upgrade again.

                                        P 1 Reply Last reply Reply Quote 0
                                        • P
                                          pfsss @stephenw10
                                          last edited by

                                          @stephenw10
                                          ok, I'm upgrading the system now. I wonder whether upgrading the system affects the packages I've installed. I have a Wireguard package installed in the 2.7.0 version os, after upgrading, it should be ok or I will reconfigured it? thanks~

                                          1 Reply Last reply Reply Quote 0
                                          • stephenw10S
                                            stephenw10 Netgate Administrator
                                            last edited by

                                            Should be fine. The packages will be upgraded and the config retained.

                                            P 3 Replies Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.