Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    PfSense 2.5 Crash report

    2.5 Development Snapshots (Retired)
    6
    17
    773
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • W
      woodchip
      last edited by

      Random crash occurs at random times. System reboots and is active again. Been having these since trying 2.5.

      info.0 textdump.tar.0

      1 Reply Last reply Reply Quote 0
      • kiokomanK
        kiokoman LAYER 8
        last edited by

        the crash was caused by ntpd, that is strange
        is always the same process that cause the crash?
        if the process change it's probably a faulty memory ... otherwise ... idk never saw that crash

        ̿' ̿'\̵͇̿̿\з=(◕_◕)=ε/̵͇̿̿/'̿'̿ ̿
        Please do not use chat/PM to ask for help
        we must focus on silencing this @guest character. we must make up lies and alter the copyrights !
        Don't forget to Upvote with the 👍 button for any post you find to be helpful.

        1 Reply Last reply Reply Quote 0
        • W
          woodchip
          last edited by

          I'll get back to you on that, cant find the other crash reports.
          Are the old ones stored on the pfsense box somewhere?

          1 Reply Last reply Reply Quote 0
          • kiokomanK
            kiokoman LAYER 8
            last edited by

            nope, afaik they are deleted / overwritten

            ̿' ̿'\̵͇̿̿\з=(◕_◕)=ε/̵͇̿̿/'̿'̿ ̿
            Please do not use chat/PM to ask for help
            we must focus on silencing this @guest character. we must make up lies and alter the copyrights !
            Don't forget to Upvote with the 👍 button for any post you find to be helpful.

            1 Reply Last reply Reply Quote 1
            • jimpJ
              jimp Rebel Alliance Developer Netgate
              last edited by 

              db:0:kdb.enter.default>  bt
Tracing pid 8411 tid 100736 td 0xfffff800099ea000
rtfree() at rtfree+0x6f/frame 0xfffffe004cc4e6b0
in_pcbladdr() at in_pcbladdr+0x411/frame 0xfffffe004cc4e750
in_pcbconnect_setup() at in_pcbconnect_setup+0x2e5/frame 0xfffffe004cc4e810
in_pcbconnect_mbuf() at in_pcbconnect_mbuf+0x3f/frame 0xfffffe004cc4e850
udp_connect() at udp_connect+0x16c/frame 0xfffffe004cc4e8a0
soconnect() at soconnect+0xad/frame 0xfffffe004cc4e8e0
kern_connectat() at kern_connectat+0xe4/frame 0xfffffe004cc4e940
sys_connect() at sys_connect+0x77/frame 0xfffffe004cc4e980
amd64_syscall() at amd64_syscall+0x369/frame 0xfffffe004cc4eab0
fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe004cc4eab0

              It crashed when trying to free a route. It's possible that is hardware but it's also possible it's due to changes in 2.5.0. We enabled RADIX_MPATH for multipath routing, and also enabled multipath in FRR. If you are using anything that might be touching that code, it's possible that the crash is due to some instability there. We have not seen any crashes here but we also have not extensively tested multipath.

              Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

              Need help fast? Netgate Global Support!

              Do not Chat/PM for help!

              W 1 Reply Last reply Reply Quote 1
              • M
                mi8088
                last edited by

                I had random crashes as well after running 2.5.0 - still don't know what caused them, but I got them to go away.

                "The important setting is "IPv6 Configuration Type" for the WAN interface. It this is set to DHCP6, as it was by default, the firewalls crash regularly. If it is set to "None", there are no crashes (or at least they are so infrequent that I haven't seen them yet). Also, as described above, DHCP6 causes a lot of log entries and blocks updates."

                See here for more info

                W 1 Reply Last reply Reply Quote 0
                • yon 0Y
                  yon 0
                  last edited by

                  Fatal trap 12: page fault while in kernel mode
                  cpuid = 0; apic id = 10
                  fault virtual address = 0x70
                  fault code = supervisor read data, page not present
                  instruction pointer = 0x20:0xffffffff80e7d451
                  stack pointer = 0x28:0xfffffe0095aa03c0
                  frame pointer = 0x28:0xfffffe0095aa04c0
                  code segment = base 0x0, limit 0xfffff, type 0x1b
                  = DPL 0, pres 1, long 1, def32 0, gran 1
                  processor eflags = interrupt enabled, resume, IOPL = 0
                  current process = 7007 (bgpd)
                  version.txt0600007113567634117 7546 ustarrootwheelFreeBSD 12.0-RELEASE-p10 fe839027797(RELENG_2_5) pfSense

                  1 Reply Last reply Reply Quote 0
                  • W
                    woodchip
                    last edited by

                    I've had another, looks pretty much the same kind of, with my limited knowledge.
                    In the mean time between crashes I have updated to a newer version.

                    info(1).0 textdump.tar(1).0

                    1 Reply Last reply Reply Quote 0
                    • W
                      woodchip @mi8088
                      last edited by

                      @mi8088 This setting is and has been set to none thru all the crashes. So does not seem related.

                      1 Reply Last reply Reply Quote 0
                      • W
                        woodchip @jimp
                        last edited by woodchip

                        @jimp I do not know for certain that I am using this. Any way to check?
                        I see rtfree again in the dump.
                        I have uploaded a new crash report.

                        1 Reply Last reply Reply Quote 0
                        • yon 0Y
                          yon 0
                          last edited by

                          <118>>>> Removing unnecessary packages... done.
                          <118>>>> Cleanup pkg cache... done.
                          <118> Starting package OpenBGPD...done.
                          <118> Starting /usr/local/etc/rc.d/bgpd.sh...done.
                          <118>pfSense 2.5.0-DEVELOPMENT amd64 Tue Dec 03 01:48:44 EST 2019
                          <118>Bootup complete
                          <6>gif4: link state changed to DOWN
                          <6>gif4: link state changed to UP
                          <6>gre0: link state changed to DOWN
                          <6>gre0: link state changed to UP
                          <6>ovpnc9: link state changed to UP
                          <6>ovpnc7: link state changed to UP
                          <6>ovpnc3: link state changed to UP
                          <3>rn_delete: Orphaned Mask 0xfffff8008935af40 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8008a218960 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8008a4f8160 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff80086b5d400 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8009bdb9e80 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8008af86e00 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff80522579380 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff800854b8420 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff805a6e20700 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8005f47d600 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8042a919f20 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8009c951a00 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff805a692b0c0 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8010454e480 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8041c01dec0 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff80106e552c0 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff80086c537a0 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff80480d99280 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff801068f87e0 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8008546e4e0 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff80088e3e780 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff80107cb7bc0 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff80087f0c540 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8010adb4420 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8010ad90f80 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8010adbbf00 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8010c6ac4c0 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8010c6ac2c0 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8010acdbb40 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff804235c2020 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8011f167e00 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8011f1689e0 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8010ad9b820 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff805e29b5780 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff800863be2e0 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8008a534e40 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff804f3fce800 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff801207a6e40 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff80087015860 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8012134d660 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff80121f6a3e0 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff801224227c0 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff80087e48000 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff805a64b7f80 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff801229b2c40 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff80088842280 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff801229b6500 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff80122a38160 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff801229b2640 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff805482ff4a0 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff800880759e0 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff80453886c40 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff80480033c00 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8041e6e7240 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff80522707a40 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff80122908da0 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff801229af700 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff80122908880 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff80418ceb7c0 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8041e7420e0 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff80480033e40 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8010ad1c980 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff80522707b80 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff800896eeb60 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8008864b680 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8010adba3c0 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8010add04e0 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff801212147a0 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff80086c1d9a0 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8025b513460 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8008512b420 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8010e61a320 at 0
                          <3>rn_delete: Orphaned Mask 0xfffff8025bbb1900 at 0
                          <3>rn_delete: couldn't find our annotation

                          Fatal trap 12: page fault while in kernel mode
                          cpuid = 2; apic id = 12
                          fault virtual address = 0x70
                          fault code = supervisor read data, page not present
                          instruction pointer = 0x20:0xffffffff80e7d451
                          stack pointer = 0x28:0xfffffe0095c683c0
                          frame pointer = 0x28:0xfffffe0095c684c0
                          code segment = base 0x0, limit 0xfffff, type 0x1b
                          = DPL 0, pres 1, long 1, def32 0, gran 1
                          processor eflags = interrupt enabled, resume, IOPL = 0
                          current process = 52869 (bgpd)
                          version.txt0600007113571735606 7546 ustarrootwheelFreeBSD 12.0-RELEASE-p10 fe839027797(RELENG_2_5) pfSense

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by stephenw10

                            Are you using FRR there? It looks like you still have OpenBGP running.

                            You should switch to FRR at this point.

                            yon 0Y 1 Reply Last reply Reply Quote 0
                            • yon 0Y
                              yon 0 @stephenw10
                              last edited by

                              @stephenw10 said in PfSense 2.5 Crash report:

                              Are you using FRR there? It looks like you still have OpenBGP running.

                              You should switch to FRR at this point.

                              i have to using openbgpd, so frr will system down.

                              https://forum.netgate.com/topic/147637/frr-with-get-two-bgp-full-route-tables-from-neighbors-with-openvpn-update-source/16

                              1 Reply Last reply Reply Quote 0
                              • yon 0Y
                                yon 0
                                last edited by

                                pkg install pfSense-pkg-Quagga_OSPF
                                Updating pfSense-core repository catalogue...
                                pfSense-core repository is up to date.
                                Updating pfSense repository catalogue...
                                pfSense repository is up to date.
                                All repositories are up to date.
                                Checking integrity... done (0 conflicting)
                                The following 2 package(s) will be affected (of 0 checked):

                                New packages to be INSTALLED:
                                pfSense-pkg-Quagga_OSPF: 0.6.21_4 [pfSense]
                                quagga: 1.2.4_7 [pfSense]

                                Number of packages to be installed: 2

                                The process will require 8 MiB more space.

                                Proceed with this action? [y/N]: y
                                [1/2] Installing quagga-1.2.4_7...
                                ===> Creating groups.
                                Using existing group 'quagga'.
                                ===> Creating users
                                Using existing user 'quagga'.
                                [1/2] Extracting quagga-1.2.4_7: 57%
                                pkg: Fail to extract /usr/local/lib/libospf.a from package: Lzma library error: Corrupted input data
                                [1/2] Extracting quagga-1.2.4_7: 100%

                                1 Reply Last reply Reply Quote 0
                                • yon 0Y
                                  yon 0
                                  last edited by

                                  <6>ovpnc3: link state changed to DOWN
                                  <6>ovpnc9: link state changed to DOWN
                                  <6>ovpnc3: link state changed to UP
                                  <6>ovpnc9: link state changed to UP
                                  <3>rn_delete: Orphaned Mask 0xfffff802586875e0 at 0
                                  <3>rn_delete: Orphaned Mask 0xfffff80259b72d40 at 0
                                  <3>rn_delete: Orphaned Mask 0xfffff801db1d3240 at 0
                                  <3>rn_delete: Orphaned Mask 0xfffff80259a5a180 at 0
                                  <3>rn_delete: Orphaned Mask 0xfffff8025291f840 at 0
                                  <3>rn_delete: Orphaned Mask 0xfffff80252827840 at 0
                                  <6>gif4: link state changed to DOWN
                                  <6>gif4: link state changed to UP
                                  <3>rn_delete: Orphaned Mask 0xfffff8025a41ce00 at 0
                                  <3>rn_delete: Orphaned Mask 0xfffff80259bdbc00 at 0
                                  <3>rn_delete: Orphaned Mask 0xfffff802599090a0 at 0
                                  <3>rn_delete: couldn't find our annotation

                                  <3>rn_delete: couldn't find our annotation
                                  <3>rn_delete: Orphaned Mask 0xfffff8056525a020 at 0
                                  <3>rn_delete: Orphaned Mask 0xfffff80565bf23c0 at 0
                                  <3>rn_delete: couldn't find our annotation
                                  <3>rn_delete: Orphaned Mask 0xfffff80565e9f820 at 0
                                  <3>rn_delete: couldn't find our annotation
                                  <3>rn_delete: Orphaned Mask 0xfffff802557bfbc0 at 0
                                  <3>rn_delete: Orphaned Mask 0xfffff802ba13f0a0 at 0
                                  <3>rn_delete: couldn't find our annotation
                                  <3>rn_delete: couldn't find our annotation
                                  <3>rn_delete: Orphaned Mask 0xfffff80566387460 at 0
                                  <3>rn_delete: Orphaned Mask 0xfffff80568e73aa0 at 0
                                  <3>rn_delete: Orphaned Mask 0xfffff8025a3dd680 at 0
                                  <3>rn_delete: couldn't find our annotation
                                  <3>rn_delete: Orphaned Mask 0xfffff80565bf1100 at 0
                                  <3>rn_delete: Orphaned Mask 0xfffff80259ea4760 at 0
                                  <3>rn_delete: Orphaned Mask 0xfffff80255150260 at 0
                                  <3>rn_delete: Orphaned Mask 0xfffff80566e7c040 at 0
                                  <3>rn_delete: Orphaned Mask 0xfffff80568610820 at 0
                                  <3>rn_delete: Orphaned Mask 0xfffff80595b31b60 at 0
                                  <3>rn_delete: Orphaned Mask 0xfffff80042723be0 at 0
                                  <3>rn_delete: Orphaned Mask 0xfffff80568e73a40 at 0
                                  <3>rn_delete: couldn't find our annotation
                                  <3>rn_delete: Orphaned Mask 0xfffff802bade9b00 at 0
                                  <3>rn_delete: couldn't find our annotation

                                  <3>rn_delete: Orphaned Mask 0xfffff8025a3cc4e0 at 0
                                  <3>rn_delete: couldn't find our annotation
                                  <3>rn_delete: Orphaned Mask 0xfffff8021ef44420 at 0

                                  <3>rn_delete: Orphaned Mask 0xfffff805695f5500 at 0

                                  <3>rn_delete: Orphaned Mask 0xfffff8056b2d2cc0 at 0
                                  <3>rn_delete: couldn't find our annotation
                                  <3>rn_delete: couldn't find our annotation
                                  <6>ovpnc7: link state changed to DOWN
                                  <6>ovpnc7: link state changed to UP
                                  <3>rn_delete: Orphaned Mask 0xfffff8025199b940 at 0
                                  <3>rn_delete: couldn't find our annotation

                                  Fatal trap 12: page fault while in kernel mode
                                  cpuid = 0; apic id = 10
                                  fault virtual address = 0x70
                                  fault code = supervisor read data, page not present
                                  instruction pointer = 0x20:0xffffffff80e7d451
                                  stack pointer = 0x28:0xfffffe0095cd43c0
                                  frame pointer = 0x28:0xfffffe0095cd44c0
                                  code segment = base 0x0, limit 0xfffff, type 0x1b
                                  = DPL 0, pres 1, long 1, def32 0, gran 1
                                  processor eflags = interrupt enabled, resume, IOPL = 0
                                  current process = 55611 (bgpd)
                                  version.txt0600007113572410756 7543 ustarrootwheelFreeBSD 12.0-RELEASE-p10 fe839027797(RELENG_2_5) pfSense

                                  1 Reply Last reply Reply Quote 0
                                  • yon 0Y
                                    yon 0
                                    last edited by

                                    <118>Starting syslog...done.
                                    <118>Starting Secure Shell Services...done.
                                    <118>Setting up interfaces microcode...done.
                                    <118>Removed leftover dhcp6c lock file: /tmp/dhcp6c_pppoe0_lock
                                    <118>Removed leftover dhcp6c lock file: /tmp/dhcp6c_pppoe2_lock
                                    <118>Configuring loopback interface...done.
                                    <118>Creating wireless clone interfaces...done.
                                    <118>Configuring LAGG interfaces...
                                    <6>lagg0: IPv6 addresses on igb0 have been removed before adding it as a member to prevent IPv6 address scope violation.
                                    <6>lagg0: link state changed to DOWN
                                    <6>lagg0: IPv6 addresses on igb1 have been removed before adding it as a member to prevent IPv6 address scope violation.
                                    <118>done.
                                    <118>route: writing to routing socket: Network is unreachable
                                    <118>route: route has not been found
                                    <118>pfSense 2.5.0-DEVELOPMENT amd64 Tue Dec 10 17:22:41 EST 2019
                                    <118>Bootup complete
                                    <6>ovpnc7: link state changed to UP
                                    <6>ovpnc3: link state changed to UP
                                    <6>ovpnc7: link state changed to DOWN
                                    <6>ovpnc9: link state changed to UP
                                    <6>ovpnc7: link state changed to UP
                                    <3>rn_delete: Orphaned Mask 0xfffff8055cb12080 at 0
                                    <3>rn_delete: Orphaned Mask 0xfffff802d00985e0 at 0
                                    <3>rn_delete: Orphaned Mask 0xfffff8058f3bc6a0 at 0
                                    <3>rn_delete: Orphaned Mask 0xfffff8049b4891a0 at 0
                                    <3>rn_delete: Orphaned Mask 0xfffff802d04c7f00 at 0
                                    <3>rn_delete: Orphaned Mask 0xfffff802d0f55860 at 0
                                    <3>rn_delete: couldn't find our annotation

                                    Fatal trap 12: page fault while in kernel mode
                                    cpuid = 1; apic id = 11
                                    fault virtual address = 0x70
                                    fault code = supervisor read data, page not present
                                    instruction pointer = 0x20:0xffffffff80e7d451
                                    stack pointer = 0x28:0xfffffe00915553c0
                                    frame pointer = 0x28:0xfffffe00915554c0
                                    code segment = base 0x0, limit 0xfffff, type 0x1b
                                    = DPL 0, pres 1, long 1, def32 0, gran 1
                                    processor eflags = interrupt enabled, resume, IOPL = 0
                                    current process = 37390 (bgpd)
                                    version.txt0600007113574107055 7540 ustarrootwheelFreeBSD 12.0-RELEASE-p10 fe839027797(RELENG_2_5) pfSense

                                    1 Reply Last reply Reply Quote 0
                                    • stephenw10S
                                      stephenw10 Netgate Administrator
                                      last edited by

                                      This crash in bgpd seems unrelated to the ntpd issue this thread was started for. You should open a new thread for it.

                                      Steve

                                      1 Reply Last reply Reply Quote 0
                                      • First post
                                        Last post