Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    25.11: Fatal trap 12 on reboot

    Scheduled Pinned Locked Moved Plus 25.11 Development Snapshots
    24 Posts 2 Posters 2.5k Views 2 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • w0wW Offline
      w0w @stephenw10
      last edited by

      @stephenw10
      Uploaded primary_secondary.zip

      1 Reply Last reply Reply Quote 1
      • stephenw10S Offline
        stephenw10 Netgate Administrator
        last edited by

        Great I see that.

        Were these VMs previously running 25.07.1 OK?

        w0wW 1 Reply Last reply Reply Quote 0
        • w0wW Offline
          w0w @stephenw10
          last edited by

          @stephenw10
          Yep

          1 Reply Last reply Reply Quote 0
          • stephenw10S Offline
            stephenw10 Netgate Administrator
            last edited by

            Are you able to test using the debug kernel in 25.11?

            https://docs.netgate.com/pfsense/en/latest/troubleshooting/debug-kernel.html

            Those crashes are not very informative unfortunately. The output from the debug kernel should give us more.

            w0wW 1 Reply Last reply Reply Quote 1
            • w0wW Offline
              w0w @stephenw10
              last edited by

              @stephenw10
              I’ve uploaded dumps from the primary firewall (primary.zip). I’ll check tomorrow whether the secondary firewall crashes as well.

              1 Reply Last reply Reply Quote 0
              • stephenw10S Offline
                stephenw10 Netgate Administrator
                last edited by

                Thanks.

                That's interesting. It looks like a different bug to me. Those are almost identical backtraces in the IPv6 stack. We are looking at it....

                w0wW 1 Reply Last reply Reply Quote 1
                • w0wW Offline
                  w0w @stephenw10
                  last edited by

                  @stephenw10
                  Uploaded secondary.zip, these look similar to me.
                  A bit off-topic.
                  Please note that both during system startup and when enabling CARP maintenance mode, there is a flood of CARP events. After the initial VIP reconfiguration at boot, it sometimes fails to complete (on the primary, almost consistently—likely due to higher latency). When PPPoE comes up, the configuration process starts over, even though PPPoE has no VIPs configured, and then it stops with nothing configured on LAN and I need to disable/enable CARP to make work it again. Is this the intended behavior? I understand this may be a separate issue and this is not related to 25.11, but it should be evident in the dumps.

                  1 Reply Last reply Reply Quote 0
                  • stephenw10S Offline
                    stephenw10 Netgate Administrator
                    last edited by

                    Yup this new panic you;re seeing is specific to the debug kernel. We are fixing that so we will then be able to see the reboot issue.

                    w0wW 1 Reply Last reply Reply Quote 1
                    • w0wW Offline
                      w0w @stephenw10
                      last edited by

                      @stephenw10
                      I am very grateful for your help and support!

                      1 Reply Last reply Reply Quote 1
                      • stephenw10S Offline
                        stephenw10 Netgate Administrator
                        last edited by

                        No worries. I'm happy you're able to test an early dev snaphot. Finding these issues earlier makes it much easier for us.

                        1 Reply Last reply Reply Quote 0
                        • stephenw10S Offline
                          stephenw10 Netgate Administrator
                          last edited by

                          Ok are you able to try a custom debug kernel here?

                          https://nc.netgate.com/nextcloud/s/Rj5F5K2qb4ps72R

                          If so rename the /boot/kernel folder to a backup then extract that .tgz into /boot and reboot into the new kernel.

                          w0wW 1 Reply Last reply Reply Quote 0
                          • w0wW Offline
                            w0w @stephenw10
                            last edited by

                            @stephenw10 said in 25.11: Fatal trap 12 on reboot:

                            rename the /boot/kernel folder

                            You mean /boot/kernel.debug?

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S Offline
                              stephenw10 Netgate Administrator
                              last edited by

                              No I mean the kernel folder in boot. That tgz contains a complete replacement for that with the new debug kernel and all the modules for it.

                              1 Reply Last reply Reply Quote 0
                              • stephenw10S Offline
                                stephenw10 Netgate Administrator
                                last edited by

                                So for example, assuming you uploaded the file via the GUI:

                                [25.11-DEVELOPMENT][admin@4860.stevew.lan]/root: mv /tmp/kernel_nd6_ifa_alloc.tgz /boot
                                [25.11-DEVELOPMENT][admin@4860.stevew.lan]/root: mv /boot/kernel /boot/kernel_old
                                [25.11-DEVELOPMENT][admin@4860.stevew.lan]/root: tar -xzf /boot/ker
                                [25.11-DEVELOPMENT][admin@4860.stevew.lan]/root: cd /boot
                                [25.11-DEVELOPMENT][admin@4860.stevew.lan]/boot: tar -xzf /boot/kernel_nd6_ifa_alloc.tgz 
                                

                                Then reboot and you should have:

                                [25.11-DEVELOPMENT][admin@4860.stevew.lan]/root: uname -a
                                FreeBSD 4860.stevew.lan 15.0-PRERELEASE FreeBSD 15.0-PRERELEASE #0 mjg-nd6-ifa_alloc-n256484-038e2217543a: Mon Sep 29 15:06:18 UTC 2025     root@pfsense-build-01.netgate.com:/usr/obj/usr/home/mjg/repos/FreeBSD-src/amd64.amd64/sys/pfSense-DEBUG amd64
                                
                                w0wW 1 Reply Last reply Reply Quote 1
                                • w0wW Offline
                                  w0w @stephenw10
                                  last edited by

                                  @stephenw10

                                  Uploaded primary29.zip

                                  1 Reply Last reply Reply Quote 0
                                  • stephenw10S Offline
                                    stephenw10 Netgate Administrator
                                    last edited by

                                    Nice! That looks like something we can work with. Thanks!

                                    w0wW 1 Reply Last reply Reply Quote 0
                                    • w0wW Offline
                                      w0w @stephenw10
                                      last edited by

                                      @stephenw10
                                      Just for your information, the secondary firewall has exactly the same dump, and that’s good, I suppose.

                                      1 Reply Last reply Reply Quote 0
                                      • stephenw10S Offline
                                        stephenw10 Netgate Administrator
                                        last edited by

                                        Yes it's exactly what you want to see in a crash report. If you get one. 😉

                                        Here is a test kernel that should have that fixed. Same procedure as before;

                                        https://nc.netgate.com/nextcloud/s/Sri33J8L5tZZQ9K

                                        1 Reply Last reply Reply Quote 1
                                        • w0wW Offline
                                          w0w
                                          last edited by

                                          So far, so good, no crashes, thanks.

                                          w0wW 1 Reply Last reply Reply Quote 1
                                          • w0wW Offline
                                            w0w @w0w
                                            last edited by w0w

                                            @stephenw10
                                            I’ve opened another thread about the Virtual IP (VIP) issue I mentioned earlier.
                                            https://forum.netgate.com/topic/198976/incomplete-vip-configuration-on-boot-causing-carp-failure-since-25.07-beta

                                            1 Reply Last reply Reply Quote 1
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.