Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    25.11: Fatal trap 12 on reboot

    Scheduled Pinned Locked Moved Plus 25.11 Development Snapshots
    24 Posts 2 Posters 2.5k Views 2 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • w0wW Offline
      w0w @stephenw10
      last edited by

      @stephenw10
      Different hardware, one of them were running on the procmox 9 and looks like a bit different backtraces too. Yes I have a couple of backtraces from both firewalls. Currently running 25.07.1 and no traps observed. Can I upload it somewhere?

      1 Reply Last reply Reply Quote 0
      • stephenw10S Offline
        stephenw10 Netgate Administrator
        last edited by

        Sure you can upload them here: https://nc.netgate.com/nextcloud/s/Ym2qxrrr7cpstgw

        w0wW 1 Reply Last reply Reply Quote 0
        • w0wW Offline
          w0w @stephenw10
          last edited by

          @stephenw10
          Uploaded primary_secondary.zip

          1 Reply Last reply Reply Quote 1
          • stephenw10S Offline
            stephenw10 Netgate Administrator
            last edited by

            Great I see that.

            Were these VMs previously running 25.07.1 OK?

            w0wW 1 Reply Last reply Reply Quote 0
            • w0wW Offline
              w0w @stephenw10
              last edited by

              @stephenw10
              Yep

              1 Reply Last reply Reply Quote 0
              • stephenw10S Offline
                stephenw10 Netgate Administrator
                last edited by

                Are you able to test using the debug kernel in 25.11?

                https://docs.netgate.com/pfsense/en/latest/troubleshooting/debug-kernel.html

                Those crashes are not very informative unfortunately. The output from the debug kernel should give us more.

                w0wW 1 Reply Last reply Reply Quote 1
                • w0wW Offline
                  w0w @stephenw10
                  last edited by

                  @stephenw10
                  I’ve uploaded dumps from the primary firewall (primary.zip). I’ll check tomorrow whether the secondary firewall crashes as well.

                  1 Reply Last reply Reply Quote 0
                  • stephenw10S Offline
                    stephenw10 Netgate Administrator
                    last edited by

                    Thanks.

                    That's interesting. It looks like a different bug to me. Those are almost identical backtraces in the IPv6 stack. We are looking at it....

                    w0wW 1 Reply Last reply Reply Quote 1
                    • w0wW Offline
                      w0w @stephenw10
                      last edited by

                      @stephenw10
                      Uploaded secondary.zip, these look similar to me.
                      A bit off-topic.
                      Please note that both during system startup and when enabling CARP maintenance mode, there is a flood of CARP events. After the initial VIP reconfiguration at boot, it sometimes fails to complete (on the primary, almost consistently—likely due to higher latency). When PPPoE comes up, the configuration process starts over, even though PPPoE has no VIPs configured, and then it stops with nothing configured on LAN and I need to disable/enable CARP to make work it again. Is this the intended behavior? I understand this may be a separate issue and this is not related to 25.11, but it should be evident in the dumps.

                      1 Reply Last reply Reply Quote 0
                      • stephenw10S Offline
                        stephenw10 Netgate Administrator
                        last edited by

                        Yup this new panic you;re seeing is specific to the debug kernel. We are fixing that so we will then be able to see the reboot issue.

                        w0wW 1 Reply Last reply Reply Quote 1
                        • w0wW Offline
                          w0w @stephenw10
                          last edited by

                          @stephenw10
                          I am very grateful for your help and support!

                          1 Reply Last reply Reply Quote 1
                          • stephenw10S Offline
                            stephenw10 Netgate Administrator
                            last edited by

                            No worries. I'm happy you're able to test an early dev snaphot. Finding these issues earlier makes it much easier for us.

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S Offline
                              stephenw10 Netgate Administrator
                              last edited by

                              Ok are you able to try a custom debug kernel here?

                              https://nc.netgate.com/nextcloud/s/Rj5F5K2qb4ps72R

                              If so rename the /boot/kernel folder to a backup then extract that .tgz into /boot and reboot into the new kernel.

                              w0wW 1 Reply Last reply Reply Quote 0
                              • w0wW Offline
                                w0w @stephenw10
                                last edited by

                                @stephenw10 said in 25.11: Fatal trap 12 on reboot:

                                rename the /boot/kernel folder

                                You mean /boot/kernel.debug?

                                1 Reply Last reply Reply Quote 0
                                • stephenw10S Offline
                                  stephenw10 Netgate Administrator
                                  last edited by

                                  No I mean the kernel folder in boot. That tgz contains a complete replacement for that with the new debug kernel and all the modules for it.

                                  1 Reply Last reply Reply Quote 0
                                  • stephenw10S Offline
                                    stephenw10 Netgate Administrator
                                    last edited by

                                    So for example, assuming you uploaded the file via the GUI:

                                    [25.11-DEVELOPMENT][admin@4860.stevew.lan]/root: mv /tmp/kernel_nd6_ifa_alloc.tgz /boot
                                    [25.11-DEVELOPMENT][admin@4860.stevew.lan]/root: mv /boot/kernel /boot/kernel_old
                                    [25.11-DEVELOPMENT][admin@4860.stevew.lan]/root: tar -xzf /boot/ker
                                    [25.11-DEVELOPMENT][admin@4860.stevew.lan]/root: cd /boot
                                    [25.11-DEVELOPMENT][admin@4860.stevew.lan]/boot: tar -xzf /boot/kernel_nd6_ifa_alloc.tgz 
                                    

                                    Then reboot and you should have:

                                    [25.11-DEVELOPMENT][admin@4860.stevew.lan]/root: uname -a
                                    FreeBSD 4860.stevew.lan 15.0-PRERELEASE FreeBSD 15.0-PRERELEASE #0 mjg-nd6-ifa_alloc-n256484-038e2217543a: Mon Sep 29 15:06:18 UTC 2025     root@pfsense-build-01.netgate.com:/usr/obj/usr/home/mjg/repos/FreeBSD-src/amd64.amd64/sys/pfSense-DEBUG amd64
                                    
                                    w0wW 1 Reply Last reply Reply Quote 1
                                    • w0wW Offline
                                      w0w @stephenw10
                                      last edited by

                                      @stephenw10

                                      Uploaded primary29.zip

                                      1 Reply Last reply Reply Quote 0
                                      • stephenw10S Offline
                                        stephenw10 Netgate Administrator
                                        last edited by

                                        Nice! That looks like something we can work with. Thanks!

                                        w0wW 1 Reply Last reply Reply Quote 0
                                        • w0wW Offline
                                          w0w @stephenw10
                                          last edited by

                                          @stephenw10
                                          Just for your information, the secondary firewall has exactly the same dump, and that’s good, I suppose.

                                          1 Reply Last reply Reply Quote 0
                                          • stephenw10S Offline
                                            stephenw10 Netgate Administrator
                                            last edited by

                                            Yes it's exactly what you want to see in a crash report. If you get one. 😉

                                            Here is a test kernel that should have that fixed. Same procedure as before;

                                            https://nc.netgate.com/nextcloud/s/Sri33J8L5tZZQ9K

                                            1 Reply Last reply Reply Quote 1
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.