Crash report 2.5.0-RC
-
2.5.0-RC (amd64)
built on Tue Feb 16 03:07:30 EST 2021
FreeBSD 12.2-STABLECrash report begins. Anonymous machine information: amd64 12.2-STABLE FreeBSD 12.2-STABLE d48fb226319(devel-12) pfSense Crash report details: No PHP errors found. Filename: /var/crash/info.0 Dump header from device: /dev/label/swap0 Architecture: amd64 Architecture Version: 4 Dump Length: 157696 Blocksize: 512 Compression: none Dumptime: Tue Feb 16 12:55:15 2021 Hostname: face.xiaoyu.net Magic: FreeBSD Text Dump Version String: FreeBSD 12.2-STABLE d48fb226319(devel-12) pfSense Panic String: page fault Dump Parity: 3488812088 Bounds: 0 Dump Status: good Filename: /var/crash/textdump.tar.0 ddb.txt06000014000014012740263 7071 ustarrootwheeldb:0:kdb.enter.default> run lockinfo db:1:lockinfo> show locks No such command; use "help" to list available commands db:1:lockinfo> show alllocks No such command; use "help" to list available commands db:1:lockinfo> show lockedvnods Locked vnodes db:0:kdb.enter.default> show pcpu cpuid = 3 dynamic pcpu = 0xfffffe0080bf2380 curthread = 0xfffff800045b1740: pid 12 tid 100042 "swi1: netisr 3" curpcb = 0xfffff800045b1ce0 fpcurthread = none idlethread = 0xfffff800045dd740: tid 100006 "idle: cpu3" curpmap = 0xffffffff8368d5a8 tssp = 0xffffffff83717758 commontssp = 0xffffffff83717758 rsp0 = 0xfffffe0000502c40 kcr3 = 0xffffffffffffffff ucr3 = 0xffffffffffffffff scr3 = 0x0 gs32p = 0xffffffff8371df70 ldt = 0xffffffff8371dfb0 tss = 0xffffffff8371dfa0 tlb gen = 5196 curvnet = 0xfffff8000408aa80 db:0:kdb.enter.default> bt Tracing pid 12 tid 100042 td 0xfffff800045b1740 kdb_enter() at kdb_enter+0x37/frame 0xfffffe0000501f90 vpanic() at vpanic+0x197/frame 0xfffffe0000501fe0 panic() at panic+0x43/frame 0xfffffe0000502040 trap_fatal() at trap_fatal+0x391/frame 0xfffffe00005020a0 trap_pfault() at trap_pfault+0x4f/frame 0xfffffe00005020f0 trap() at trap+0x286/frame 0xfffffe0000502200 calltrap() at calltrap+0x8/frame 0xfffffe0000502200 --- trap 0xc, rip = 0xffffffff80d7e767, rsp = 0xfffffe00005022d0, rbp = 0xfffffe0000502350 --- __mtx_lock_sleep() at __mtx_lock_sleep+0xd7/frame 0xfffffe0000502350 100027 D - 0xfffff800045cd900 [softirq_2] 100028 D - 0xfffff800045cd700 [softirq_3] 100029 D - 0xfffff800045cd500 [if_config_tqg_0] 100030 D - 0xfffff800045cd300 [if_io_tqg_0] 100031 D - 0xfffff800045cd100 [if_io_tqg_1] 100032 D - 0xfffff800045cee00 [if_io_tqg_2] 100033 D - 0xfffff800045cec00 [if_io_tqg_3] 100047 D - 0xfffff800045cf200 [firmware taskq] 100052 D - 0xfffff8000457fc00 [crypto_0] 100053 D - 0xfffff8000457fc00 [crypto_1] 100054 D - 0xfffff8000457fc00 [crypto_2] 100055 D - 0xfffff8000457fc00 [crypto_3] 100118 D - 0xfffff800050b4000 [mca taskq] 100128 D - 0xfffff80020f61d00 [acpi_task_0] 100129 D - 0xfffff80020f61d00 [acpi_task_1] 100130 D - 0xfffff80020f61d00 [acpi_task_2] 100131 D - 0xfffff80004580900 [CAM taskq] 100133 D - 0xfffff80020f62600 [mlx4] db:0:kdb.enter.default> alltrace Tracing command nice pid 26634 tid 100345 td 0xfffff8011ef3a000 (CPU 2) cpustop_handler() at cpustop_handler+0x27/frame 0xfffffe0003f4be00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe0003f4be10 trap() at trap+0x44/frame 0xfffffe0003f4bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0003f4bf20 --- trap 0x13, rip = 0xffffffff813725b1, rsp = 0xfffffe009a2d6620, rbp = 0xfffffe009a2d6690 --- pmap_enter_quick_locked() at pmap_enter_quick_locked+0x231/frame 0xfffffe009a2d6690 pmap_enter_object() at pmap_enter_object+0x218/frame 0xfffffe009a2d6700 vm_map_pmap_enter() at vm_map_pmap_enter+0x2c9/frame 0xfffffe009a2d6780 vm_map_insert() at vm_map_insert+0x43a/frame 0xfffffe009a2d6810 vm_map_fixed() at vm_map_fixed+0x138/frame 0xfffffe009a2d6870 vm_mmap_object() at vm_mmap_object+0x1c2/frame 0xfffffe009a2d68e0 vn_mmap() at vn_mmap+0x14f/frame 0xfffffe009a2d6960 kern_mmap_fpcheck() at kern_mmap_fpcheck+0x4b0/frame 0xfffffe009a2d6a20 sys_mmap() at sys_mmap+0x23/frame 0xfffffe009a2d6a40 amd64_syscall() at amd64_syscall+0x387/frame 0xfffffe009a2d6b70 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe009a2d6b70 --- syscall (477, FreeBSD ELF64, sys_mmap), rip = 0x800221b5a, rsp = 0x7fffffffd178, rbp = 0x7fffffffd2f0 --- Tracing command netstat pid 26245 tid 100179 td 0xfffff80020f02740 sched_switch() at sched_switch+0x630/frame 0xfffffe00845e1af0 mi_switch() at mi_switch+0xd4/frame 0xfffffe00845e1b20 critical_exit_preempt() at critical_exit_preempt+0x55/frame 0xfffffe00845e1b40 ipi_bitmap_handler() at ipi_bitmap_handler+0x8f/frame 0xfffffe00845e1b70 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0xb3/frame 0xfffffe00845e1b70 --- interrupt, rip = 0x8002a309e, rsp = 0x7fffffffe090, rbp = 0x7fffffffe1b0 --- pcib1: <ACPI PCI-PCI bridge> irq 18 at device 2.0 on pci0 pci1: <ACPI PCI bus> on pcib1 igb0: <Intel(R) PRO/1000 PCI-Express Network Driver> mem 0xff480000-0xff4fffff,0xff504000-0xff507fff irq 18 at device 0.0 on pci1 igb0: Using 1024 TX descriptors and 1024 RX descriptors igb0: Using 4 RX queues 4 TX queues igb0: Using MSI-X interrupts with 5 vectors <6>igb0: Ethernet address: a0:36:9f:83:99:40 <6>igb0: netmap queues/slots: TX 4/1024, RX 4/1024 igb1: <Intel(R) PRO/1000 PCI-Express Network Driver> mem 0xff380000-0xff3fffff,0xff500000-0xff503fff irq 19 at device 0.1 on pci1 igb1: Using 1024 TX descriptors and 1024 RX descriptors igb1: Using 4 RX queues 4 TX queues igb1: Using MSI-X interrupts with 5 vectors <6>igb1: Ethernet address: a0:36:9f:83:99:41 <6>igb1: netmap queues/slots: TX 4/1024, RX 4/1024 [textdump.tar.0](/assets/uploads/files/1613480696829-textdump.tar.0) <6>wg1: link state changed to UP <118>netstat: sysctl: net.route.0.0.dump.0: Cannot allocate memory <118>netstat: sysctl: net.route.0.0.dump.0: Cannot allocate memory <118>done. <118>Setting up DNSs... <118>Synchronizing user settings...done. <118>Starting webConfigurator...done. <118>Configuring CRON...done. <118>Starting NTP Server...done. <118>Starting DHCP service...done. <118>Starting DHCPv6 service...done. <6>ovpnc12: link state changed to UP <6>ovpnc9: link state changed to UP <6>ovpnc9: link state changed to DOWN <118>Configuring firewall.. <6>ovpnc7: link state changed to UP <118>....done. <118>Configuring WireGuard Tunnels... <6>wg1: link state changed to DOWN Fatal trap 12: page fault while in kernel mode cpuid = 3; apic id = 13 fault virtual address = 0x410 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80d7e767 stack pointer = 0x0:0xfffffe00005022d0 frame pointer = 0x0:0xfffffe0000502350 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 12 (swi1: netisr 3) trap number = 12 panic: page fault cpuid = 3 time = 1613480115 KDB: enter: panic panic.txt0600001214012740263 7130 ustarrootwheelpage faultversion.txt0600006214012740263 7530 ustarrootwheelFreeBSD 12.2-STABLE d48fb226319(devel-12) pfSense
-
-
cpustop_handler() at cpustop_handler+0x27/frame 0xfffffe0003f4be00 ipi_nmi_handler() at ipi_nmi_handler+0x39/frame 0xfffffe0003f4be10 trap() at trap+0x44/frame 0xfffffe0003f4bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe0003f4bf20
Usually NMI (Non-maskable interrupt) calls indicate hardware problems. It's not always 100% but that's the top suspect.
-
Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 11 fault virtual address = 0x410 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80d7e767 stack pointer = 0x0:0xfffffe00004cb090 frame pointer = 0x0:0xfffffe00004cb110 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 0 (if_io_tqg_1) trap number = 12 panic: page fault cpuid = 1 time = 1613830597 KDB: enter: panic panic.txt0600001214014214705 7127 ustarrootwheelpage faultversion.txt0600006214014214705 7527 ustarrootwheelFreeBSD 12.2-STABLE d48fb226319(devel-12) pfSense
-
https://forum.netgate.com/topic/160491/cannot-allocate-memory/3
-
Feb 21 18:16:58 kernel matchaddr failed Feb 21 18:16:57 kernel matchaddr failed Feb 21 18:16:56 kernel matchaddr failed
-
I don't believe those "matchaddr failed" entries have to do with any crashes. I think they're related to WireGuard though, as they appeared in my logs a lot as I was getting WireGuard set up and was frequently changing settings to get things working.
-
@virgiliomi
I think multiple factors may cause the crash, I think it involves FRR and wireguard.
i have report to wireguard admin.