System Crash - 2.7: Panic String: deadlres_td_sleep_q: possible deadlock detected for 0xfffffe00dbe38c80 (php-fpm)
-
Looking for a little advice. I'm new to pfSense (strike 1) as I've mostly used a tangled alternative. Recently picked up a new white label FW box from Amazon. I got it from Amazon but it is an Alibaba retailer. Anyway, the box has I226-V ports (strike 2) to I had to go with the 2.7 dev build and that is really where the advice request comes from.
As I'm working with the box and becoming familiar with pfSense I've seen the box crash a few times. The first couple of times I thought it was me having to redo things as I learn the interface but this last time I took time to review the crash log (truncated excerpt below).
It looks to me like the crash was in FreeBSD and kinda the gist of the post. When these types of crashed are recorded, is it something the Netgate dev teams want to review? And if there is any separation between GA vs Dev releases and/or OS vs pfSense as the reported core of the issue?
Appreciate the feedback.
Crash report begins. Anonymous machine information: amd64 14.0-CURRENT FreeBSD 14.0-CURRENT #0 devel-main-n255825-17d2b04a49e: Fri Jan 20 03:25:13 UTC 2023 root@freebsd:/var/jenkins/workspace/pfSense-CE-snapshots-master-main/obj/amd64/TycEh1bZ/var/jenkins/workspace/pfSense-CE-snapshots-master-main/sources/FreeBSD-src-dev Crash report details: No PHP errors found. Filename: /var/crash/info.0 Dump header from device: /dev/label/swap0 Architecture: amd64 Architecture Version: 4 Dump Length: 157696 Blocksize: 512 Compression: none Dumptime: 2023-01-20 10:02:19 -0600 Hostname: xxxxxxxxxxxx Magic: FreeBSD Text Dump Version String: FreeBSD 14.0-CURRENT #0 devel-main-n255825-17d2b04a49e: Fri Jan 20 03:25:13 UTC 2023 root@freebsd:/var/jenkins/workspace/pfSense-CE-snapshots-master-main/obj/amd64/TycEh1bZ/var/jenkins/w Panic String: deadlres_td_sleep_q: possible deadlock detected for 0xfffffe00dbe38c80 (php-fpm), blocked for 1801288 ticks Dump Parity: 2061020004 Bounds: 0 Dump Status: good Filename: /var/crash/textdump.tar.0 ddb.txt���������������������������������������������������������������������������������������������0600����0�������0�������140000������14362535413� 7100� �����������������������������������������������������������������������������������������������������ustar���root����������������������������wheel������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������db:0:kdb.enter.default> show registers cs 0x20 ds 0x3b es 0x3b fs 0x13 gs 0x1b ss 0x28 rax 0x12 rcx 0x1 rdx 0xffffffff81558d03 rbx 0x100 rsp 0xfffffe00c8543e00 rbp 0xfffffe00c8543e00 rsi 0 rdi 0xffffffff8354b6a0 cnputs_mtx r8 0 r9 0x1c200001 r10 0xffffffff83182f48 vt_conswindow r11 0x20 r12 0x1b7c00 r13 0xfffffe00dbe38c80 r14 0xfffffe00c8543e90 r15 0xfffffe00ddbfeac0 rip 0xffffffff80dc8dc2 kdb_enter+0x32 rflags 0x86 kdb_enter+0x32: movq $0,0x27cda43(%rip) db:0:kdb.enter.default> run lockinfo db:1:lockinfo> show locks exclusive sleep mutex process lock (process lock) r = 0 (0xfffffe00def8e128) locked @ /var/jenkins/workspace/pfSense-CE-snapshots-master-main/sources/FreeBSD-src-devel-main/sys/kern/kern_clock.c:273 shared sx allproc (allproc) r = 0 (0xffffffff83002b80) locked @ /var/jenkins/workspace/pfSense-CE-snapshots-master-main/sources/FreeBSD-src-devel-main/sys/kern/kern_clock.c:263 db:1:lockinfo> show alllocks Process 75614 (dpinger) thread 0xfffffe00dd8a0720 (100478) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xfffff80027f74900) locked @ /var/jenkins/workspace/pfSense-CE-snapshots-master-main/sources/FreeBSD-src-devel-main/sys/kern/uipc_socket.c:3955 Process 75088 (dpinger) thread 0xfffffe00dd926000 (100474) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xfffff80025a9a540) locked @ /var/jenkins/workspace/pfSense-CE-snapshots-master-main/sources/FreeBSD-src-devel-main/sys/kern/uipc_socket.c:3955 Process 74660 (dpinger) thread 0xfffffe00dd8a8900 (100470) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xfffff80117c7b540) locked @ /var/jenkins/workspace/pfSense-CE-snapshots-master-main/sources/FreeBSD-src-devel-main/sys/kern/uipc_socket.c:3955 Process 74542 (dpinger) thread 0xfffffe00dd8ed560 (100466) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xfffff8002786d180) locked @ /var/jenkins/workspace/pfSense-CE-snapshots-master-main/sources/FreeBSD-src-devel-main/sys/kern/uipc_socket.c:3955 Process 41810 (php-fpm) thread 0xfffffe00dd8c2560 (100294) exclusive sx ifnet_detach_sx (ifnet_detach_sx) r = 0 (0xffffffff8365af48) locked @ /var/jenkins/workspace/pfSense-CE-snapshots-master-main/sources/FreeBSD-src-devel-main/sys/net/if.c:3113 Process 0 (kernel) thread 0xfffffe00ddbfeac0 (100110) exclusive sleep mutex process lock (process lock) r = 0 (0xfffffe00def8e128) locked @ /var/jenkins/workspace/pfSense-CE-snapshots-master-main/sources/FreeBSD-src-devel-main/sys/kern/kern_clock.c:273 shared sx allproc (allproc) r = 0 (0xffffffff83002b80) locked @ /var/jenkins/workspace/pfSense-CE-snapshots-master-main/sources/FreeBSD-src-devel-main/sys/kern/kern_clock.c:263 db:1:lockinfo> show lockedvnods Locked vnodes db:0:kdb.enter.default> show pcpu cpuid = 2 dynamic pcpu = 0xfffffe009ed3d600 curthread = 0xfffffe00ddbfeac0: pid 0 tid 100110 critnest 1 "deadlkres" curpcb = 0xfffffe00ddbfefe0 fpcurthread = none idlethread = 0xfffffe002210e560: tid 100005 "idle: cpu2" self = 0xffffffff84612000 curpmap = 0xffffffff8354a950 tssp = 0xffffffff84612384 rsp0 = 0xfffffe00c8544000 kcr3 = 0xffffffffffffffff ucr3 = 0xffffffffffffffff scr3 = 0x0 gs32p = 0xffffffff84612404 ldt = 0xffffffff84612444 tss = 0xffffffff84612434 curvnet = 0 spin locks held: db:0:kdb.enter.default> bt Tracing pid 0 tid 100110 td 0xfffffe00ddbfeac0 kdb_enter() at kdb_enter+0x32/frame 0xfffffe00c8543e00 vpanic() at vpanic+0x182/frame 0xfffffe00c8543e50 panic() at panic+0x43/frame 0xfffffe00c8543eb0 deadlkres() at deadlkres+0x351/frame 0xfffffe00c8543ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00c8543f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00c8543f30 --- trap 0xc, rip = 0x185c02df178a, rsp = 0x185bff9d7e48, rbp = 0x185bff9d7e60 --- db:0:kdb.enter.default> ps pid ppid pgrp uid state wmesg wchan cmd 78949 11771 566 0 S nanslp 0xffffffff8356a020 sleep 36316 566 566 0 S accept 0xfffff800251b11d0 php-fpm 75614 1 75614 0 Ss (threaded) dpinger 100329 S uwait 0xfffff802b49fcd00 dpinger 100478 S sbwait 0xfffff80027f7494c dpinger 100479 S nanslp 0xffffffff8356a023 dpinger 100480 S nanslp 0xffffffff8356a021 dpinger 100481 S accept 0xfffff80025a9ad10 dpinger 75088 1 75088 0 Ss (threaded) dpinger 100352 S uwait 0xfffff8002721a500 dpinger 100474 S sbwait 0xfffff80025a9a58c dpinger 100475 S nanslp 0xffffffff8356a023 dpinger 100476 S nanslp 0xffffffff8356a022 dpinger 100477 S accept 0xfffff80027f74590 dpinger 74660 1 74660 0 Ss (threaded) dpinger 100148 S uwait 0xfffff80001bb1480 dpinger 100470 S sbwait 0xfffff80117c7b58c dpinger 100471 S nanslp 0xffffffff8356a023 dpinger 100472 S nanslp 0xffffffff8356a021 dpinger 100473 S accept 0xfffff80027f7e1d0 dpinger 74542 1 74542 0 Ss (threaded) dpinger 100455 S uwait 0xfffff8002721a000 dpinger 100466 S sbwait 0xfffff8002786d1cc dpinger 100467 S nanslp 0xffffffff8356a022 dpinger 100468 S nanslp 0xffffffff8356a022 dpinger 100469 S accept 0xfffff800258b51d0 dpinger 15922 566 566 0 D ifnet_d 0xffffffff8365af48 php-fpm 11771 1 566 0 S wait 0xfffffe00dd8eaac0 sh 280 6 98975 0 S piperd 0xfffffe00defa86c0 sh 6 98975 98975 0 S wait 0xfffffe00df381560 sh 99886 98975 98975 0 S (threaded) sshg-blocker 100296 S piperd 0xfffffe00defef3e8 sshg-blocker 100413 S nanslp 0xffffffff8356a021 sshg-blocker 99429 98975 98975 0 S piperd 0xfffffe00ddb9f110 sshg-parser 99261 98975 98975 0 S piperd 0xfffffe00df27c888 cat 98975 22844 98975 0 Ss wait 0xfffffe00def7e558 sh 31944 1 31944 59 Ss (threaded) unbound 100404 S kqread 0xfffff800275f8400 unbound 100410 S kqread 0xfffff800254ae800 unbound 100411 S kqread 0xfffff801174bb800 unbound 100412 S kqread 0xfffff80117717500 unbound 56113 1 56113 0 Ss kqread 0xfffff80025367a00 dhcpleases 22463 1 22463 0 Ss select 0xfffff80117cce4c0 openvpn 70129 1 70129 0 Ss select 0xfffff8002721bac0 openvpn 52888 52324 52888 0 S+ ttyin 0xfffff8002725f8b0 sh 52324 50534 52324 0 S+ wait 0xfffffe00dd8bd568 sh 52154 1 52154 0 Ss+ ttyin 0xfffff80027261cb0 getty 52004 1 52004 0 Ss+ ttyin 0xfffff800272600b0 getty 51760 1 51760 0 Ss+ ttyin 0xfffff800272604b0 getty 51422 1 51422 0 Ss+ ttyin 0xfffff800272608b0 getty 51308 1 51308 0 Ss+ ttyin 0xfffff80027260cb0 getty 51132 1 51132 0 Ss+ ttyin 0xfffff8002725f0b0 getty 50786 1 50786 0 Ss+ ttyin 0xfffff8002725f4b0 getty 50534 1 50534 0 Ss+ wait 0xfffffe00221d5570 login 41810 566 566 0 D tun_con 0xfffff80022ff49a8 php-fpm 37879 1 37879 136 Ss select 0xfffff8027cbdf3c0 dhcpd 88352 1 88352 0 Ss (threaded) filterdns 100330 S usem 0xfffff80117ccd580 filterdns 100331 S usem 0xfffff800221c6180 filterdns 100332 S usem 0xfffff80117ccd480 filterdns 100333 S usem 0xfffff80117ccd380 filterdns 100334 S usem 0xfffff800221c6e80 filterdns 100335 S usem 0xfffff800221c6a80 filterdns 100336 S usem 0xfffff800221c6b80 filterdns 100337 S usem 0xfffff8002721aa80 192.168.111.128.192 100338 S usem 0xfffff8002721c580 10.0.0.0 100339 S usem 0xfffff800221c6d80 172.16.0.0 100340 S usem 0xfffff8002721c680 192.168.0.0 100341 S usem 0xfffff80001bb2b00 192.168.128.0 100342 S usem 0xfffff8002721ab80 192.168.192.0 100343 S usem 0xfffff80001bb2300 192.168.200.0 100344 S usem 0xfffff80001bb3800 merge-thread 25722 25399 25399 0 S nanslp 0xffffffff8356a020 minicron 25399 1 25399 0 Ss wait 0xfffffe00deff2010 minicron 25055 24619 24619 0 S nanslp 0xffffffff8356a021 minicron 24619 1 24619 0 Ss wait 0xfffffe00def80ac0 minicron 24441 23895 23895 0 S nanslp 0xffffffff8356a021 minicron 23895 1 23895 0 Ss wait 0xfffffe00df382ac0 minicron 23849 23662 23662 0 S nanslp 0xffffffff8356a020 minicron 23662 1 23662 0 Ss wait 0xfffffe00def8e558 minicron 22844 1 22844 0 Ss select 0xfffff800221c5040 syslogd 14026 13676 13676 0 S kqread 0xfffff80025f23000 nginx 13726 13676 13676 0 S kqread 0xfffff8012318ba00 nginx 13676 1 13676 0 Ss pause 0xfffffe00def8fb68 nginx 86274 1 86274 0 Ss (threaded) ntpd 100155 S select 0xfffff80117ccd640 ntpd 64574 1 64574 0 Ss nanslp 0xffffffff8356a023 cron 83062 1 83062 0 Ss bpf 0xfffff800220b7600 filterlog 81115 1 81115 0 Ss select 0xfffff800221c63c0 openvpn 63817 1 63817 0 Ss select 0xfffff800221c6f40 openvpn 48077 1 48077 0 Ss select 0xfffff80117ccd940 openvpn 31329 1 31329 0 Ss select 0xfffff80001bb1bc0 openvpn 1150 1 1150 0 Ss select 0xfffff80117ccdf40 devd 607 605 605 0 S kqread 0xfffff800020a6b00 check_reload_status 605 1 605 0 Ss kqread 0xfffff800020a8500 check_reload_status 568 566 566 0 D (threaded) php-fpm 100154 D ifnet_d 0xffffffff8365af48 php-fpm 567 566 566 0 S (threaded) php-fpm 100160 S accept 0xfffff800251b11d0 php-fpm 566 1 566 0 Ss kqread 0xfffff801175bfb00 php-fpm 22 0 0 0 DL aldslp 0xffffffff8354b370 [ALQ Daemon] 21 0 0 0 DL vlruwt 0xfffffe00def80010 [vnlru] 20 0 0 0 DL syncer 0xffffffff8365aca8 [syncer] 19 0 0 0 DL (threaded) [bufdaemon] 100126 D psleep 0xffffffff83659ce0 [bufdaemon] 100128 D - 0xffffffff8300b0c0 [bufspacedaemon-0] 100129 D - 0xffffffff83013380 [bufspacedaemon-1] 100130 D - 0xffffffff8301b640 [bufspacedaemon-2] 100131 D - 0xffffffff83023900 [bufspacedaemon-3] 100132 D - 0xffffffff8302bbc0 [bufspacedaemon-4] 100133 D - 0xffffffff83033e80 [bufspacedaemon-5] 100134 D - 0xffffffff8303c140 [bufspacedaemon-6] 100143 D sdflush 0xfffff8002766dce8 [/ worker] 18 0 0 0 DL psleep 0xffffffff83682050 [vmdaemon] 9 0 0 0 DL (threaded) [pagedaemon] 100124 D psleep 0xffffffff836764b8 [dom0] 100127 D launds 0xffffffff836764c4 [laundry: dom0] 100135 D umarcl 0xffffffff811ca770 [uma] 17 0 0 0 DL tzpoll 0xffffffff83272f58 [acpi_thermal] 8 0 0 0 DL - 0xffffffff8339d088 [rand_harvestq] 7 0 0 0 DL pftm 0xffffffff8103f7f0 [pf purge] 5 0 0 0 DL - 0xffffffff8368aa38 [busdma] 16 0 0 0 DL (threaded) [usb] 100067 D - 0xfffffe00c7fa0440 [usbus0] 100068 D - 0xfffffe00c7fa0498 [usbus0] 100069 D - 0xfffffe00c7fa04f0 [usbus0] 100070 D - 0xfffffe00c7fa0548 [usbus0] 100071 D - 0xfffffe00c7fa05a0 [usbus0] 4 0 0 0 DL (threaded) [cam] 100064 D - 0xffffffff83265d40 [doneq0] 100065 D - 0xffffffff83265cc0 [async] 100122 D - 0xffffffff83265b90 [scanner] 15 0 0 0 DL seqstat 0xfffff80001cd9888 [sequencer 00] 3 0 0 0 DL (threaded) [crypto] 100057 D crypto_ 0xffffffff83673a00 [crypto] 100058 D crypto_ 0xfffff800018afab0 [crypto returns 0] 100059 D crypto_ 0xfffff800018afb00 [crypto returns 1] 100060 D crypto_ 0xfffff800018afb50 [crypto returns 2] 100061 D crypto_ 0xfffff800018afba0 [crypto returns 3] 14 0 0 0 DL (threaded) [geom] 100050 D - 0xffffffff835495c8 [g_event] 100051 D - 0xffffffff835495d0 [g_up] 100052 D - 0xffffffff835495d8 [g_down] 13 0 0 0 DL (threaded) [ng_queue] 100045 D sleep 0xffffffff831de030 [ng_queue0] 100046 D sleep 0xffffffff831de030 [ng_queue1] 100047 D sleep 0xffffffff831de030 [ng_queue2] 100048 D sleep 0xffffffff831de030 [ng_queue3] 2 0 0 0 WL (threaded) [clock] 100041 I [clock (0)] 100042 I [clock (1)] 100043 I [clock (2)] 100044 I [clock (3)] 12 0 0 0 WL (threaded) [intr] 100020 I [swi5: fast taskq] 100023 I [swi6: task queue] 100026 I [swi6: Giant taskq] 100037 I [swi1: netisr 0] 100038 I [swi1: netisr 1] 100039 I [swi1: netisr 2] 100040 I [swi1: netisr 3] 100066 I [irq128: xhci0] 100072 I [irq129: ahci0] 100074 I [irq160: hdac0] 100075 I [irq7: ppc0] 100076 I [swi0: uart] 100077 I [irq1: atkbd0] 100113 I [swi1: pf send] 100114 I [swi1: pfsync] 11 0 0 0 RL (threaded) [idle] 100003 Run CPU 0 [idle: cpu0] 100004 Run CPU 1 [idle: cpu1] 100005 CanRun [idle: cpu2] 100006 Run CPU 3 [idle: cpu3] 1 0 1 0 SLs wait 0xfffffe002210b010 [init] 10 0 0 0 DL audit_w 0xffffffff83673f98 [audit] 0 0 0 0 RLs (threaded) [kernel] 100000 D swapin 0xffffffff83549b70 [swapper] 100007 D - 0xfffff80001239e00 [if_io_tqg_0] 100008 D - 0xfffff80001239d00 [if_io_tqg_1] 100009 D - 0xfffff80001239c00 [if_io_tqg_2] 100010 D - 0xfffff80001239b00 [if_io_tqg_3] 100011 D - 0xfffff80001239a00 [if_config_tqg_0] 100012 D - 0xfffff80001239900 [softirq_0] 100013 D - 0xfffff80001239800 [softirq_1] 100014 D - 0xfffff80001239700 [softirq_2] 100015 D - 0xfffff80001239600 [softirq_3] 100016 D - 0xfffff80001881a00 [linuxkpi_irq_wq] 100017 D - 0xfffff80001881900 [inm_free taskq] 100018 D - 0xfffff80001881800 [in6m_free taskq] 100019 D - 0xfffff80001881700 [thread taskq] 100021 D - 0xfffff80001881500 [pci_hp taskq] 100022 D - 0xfffff80001881400 [kqueue_ctx taskq] 100024 D - 0xfffff80001881200 [aiod_kick taskq] 100025 D - 0xfffff80001881100 [deferred_unmount ta] 100027 D - 0xfffff80001880e00 [linuxkpi_short_wq_0] 100028 D - 0xfffff80001880e00 [linuxkpi_short_wq_1] 100029 D - 0xfffff80001880e00 [linuxkpi_short_wq_2] 100030 D - 0xfffff80001880e00 [linuxkpi_short_wq_3] 100031 D - 0xfffff80001880e00 [linuxkpi_short_wq_4] 100032 D - 0xfffff80001880d00 [linuxkpi_long_wq_0] 100033 D - 0xfffff80001880d00 [linuxkpi_long_wq_1] 100034 D - 0xfffff80001880d00 [linuxkpi_long_wq_2] 100035 D - 0xfffff80001880d00 [linuxkpi_long_wq_3] 100036 D - 0xfffff80001880d00 [linuxkpi_long_wq_4] 100049 D - 0xfffff80001880c00 [firmware taskq] 100053 D - 0xfffff8000187fb00 [crypto_0] 100054 D - 0xfffff8000187fb00 [crypto_1] 100055 D - 0xfffff8000187fb00 [crypto_2] 100056 D - 0xfffff8000187fb00 [crypto_3] 100062 D - 0xfffff8000187fa00 [t4_rst_thr] 100110 Run CPU 2 [deadlkres] 100111 D - 0xfffff80001880600 [ip_mroute_tskq task] 100115 D - 0xfffff80001882100 [mca taskq] 100118 D - 0xfffff80001880a00 [acpi_task_0] 100119 D - 0xfffff80001880a00 [acpi_task_1] 100120 D - 0xfffff80001880a00 [acpi_task_2] 100121 D - 0xfffff8000187f300 [CAM taskq] 100123 D - 0xfffff80027246b00 [mlx4] db:0:kdb.enter.default> alltrace Tracing command kernel pid 0 tid 100000 td 0xffffffff8354a0e0 sched_switch() at sched_switch+0x582/frame 0xffffffff844ffe10 mi_switch() at mi_switch+0x157/frame 0xffffffff844ffe30 sleepq_switch() at sleepq_switch+0x107/frame 0xffffffff844ffe70 sleepq_timedwait() at sleepq_timedwait+0x4b/frame 0xffffffff844ffeb0 _sleep() at _sleep+0x2d6/frame 0xffffffff844fff40 swapper() at swapper+0x75/frame 0xffffffff844fffa0 mi_startup() at mi_startup+0x30e/frame 0xffffffff844ffff0 btext() at btext+0x23 Tracing command kernel pid 0 tid 100007 td 0xfffffe002210d720 sched_switch() at sched_switch+0x582/frame 0xfffffe001d9efde0 mi_switch() at mi_switch+0x157/frame 0xfffffe001d9efe00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe001d9efe40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe001d9efec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe001d9efef0 fork_exit() at fork_exit+0x80/frame 0xfffffe001d9eff30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe001d9eff30 --- trap 0x7c892650, rip = 0xc7e9a07d8b4c0000, rsp = 0x4800000102840f02, rbp = 0x78b417024448904 --- Tracing command kernel pid 0 tid 100008 td 0xfffffe002210d000 sched_switch() at sched_switch+0x582/frame 0xfffffe001d9eade0 mi_switch() at mi_switch+0x157/frame 0xfffffe001d9eae00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe001d9eae40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe001d9eaec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe001d9eaef0 fork_exit() at fork_exit+0x80/frame 0xfffffe001d9eaf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe001d9eaf30 --- trap 0xb0810966, rip = 0xe9831c74ce2b412a, rsp = 0x30000014482c626, rbp = 0x8b48184889485845 --- Tracing command kernel pid 0 tid 100009 td 0xfffffe0022194c80 sched_switch() at sched_switch+0x582/frame 0xfffffe00c7cb2de0 mi_switch() at mi_switch+0x157/frame 0xfffffe00c7cb2e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00c7cb2e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00c7cb2ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00c7cb2ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00c7cb2f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00c7cb2f30 --- trap 0xed7ddcf1, rip = 0x19af627195cc7ab7, rsp = 0xfc28688d41c71353, rbp = 0xeb2f0454daa3674c --- Tracing command kernel pid 0 tid 100010 td 0xfffffe0022194560 sched_switch() at sched_switch+0x582/frame 0xfffffe00c7cadde0 mi_switch() at mi_switch+0x157/frame 0xfffffe00c7cade00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00c7cade40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00c7cadec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00c7cadef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00c7cadf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00c7cadf30 --- trap 0x6891aea0, rip = 0xd3030df4b01bcbc5, rsp = 0xf21b2cec9103eadd, rbp = 0xcb2afaa699be5f8f --- Tracing command kernel pid 0 tid 100011 td 0xfffffe0022193e40 sched_switch() at sched_switch+0x582/frame 0xfffffe00c7ca8de0 mi_switch() at mi_switch+0x157/frame 0xfffffe00c7ca8e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00c7ca8e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00c7ca8ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00c7ca8ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00c7ca8f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00c7ca8f30 --- trap 0x8e89bf05, rip = 0x5e60985114320cf4, rsp = 0xb3943ff7273116bd, rbp = 0x6677db98a04657f --- Tracing command kernel pid 0 tid 100012 td 0xfffffe0022193720 sched_switch() at sched_switch+0x582/frame 0xfffffe00c7ca3de0 mi_switch() at mi_switch+0x157/frame 0xfffffe00c7ca3e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00c7ca3e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00c7ca3ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00c7ca3ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00c7ca3f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00c7ca3f30 --- trap 0x9c401023, rip = 0x746c3e3eaa9b83d1, rsp = 0xa1422d2135e7046b, rbp = 0x34f74f29b89457ef --- Tracing command kernel pid 0 tid 100013 td 0xfffffe0022193000 sched_switch() at sched_switch+0x582/frame 0xfffffe00c7c9ede0 mi_switch() at mi_switch+0x157/frame 0xfffffe00c7c9ee00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00c7c9ee40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00c7c9eec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00c7c9eef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00c7c9ef30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00c7c9ef30 --- trap 0x3e463e46, rip = 0xeb4ab9de1cf716a5, rsp = 0xf0554dba3665c1d9, rbp = 0xe0f0aaa23e07174d --- Tracing command kernel pid 0 tid 100014 td 0xfffffe0022111740 sched_switch() at sched_switch+0x582/frame 0xfffffe00c7c99de0 mi_switch() at mi_switch+0x157/frame 0xfffffe00c7c99e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00c7c99e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00c7c99ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00c7c99ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00c7c99f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00c7c99f30 --- trap 0x76585f12, rip = 0xa04905604f32db97, rsp = 0x56e11cb38816a15c, rbp = 0x1a60f51b2bec9603 --- Tracing command kernel pid 0 tid 100015 td 0xfffffe0022111020 sched_switch() at sched_switch+0x582/frame 0xfffffe00c7c94de0 mi_switch() at mi_switch+0x157/frame 0xfffffe00c7c94e00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00c7c94e40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00c7c94ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xf1/frame 0xfffffe00c7c94ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00c7c94f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00c7c94f30 --- trap 0x4d1f93e8, rip = 0x737aad8d10626bbc, rsp = 0xbd63a5a59429f73, rbp = 0xb23ad12217139b70 --- Tracing command kernel pid 0 tid 100016 td 0xfffffe0022110900 sched_switch() at sched_switch+0x582/frame 0xfffffe00c7c8fde0 mi_switch() at mi_switch+0x157/frame 0xfffffe00c7c8fe00 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00c7c8fe40 msleep_spin_sbt() at msleep_spin_sbt+0x154/frame 0xfffffe00c7c8fec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xf1/frame 0xfffffe00c7c8fef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00c7c8ff30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00c7c8ff30 --- trap 0xd13be0b7, rip = 0xa9df3d7a143046a4, rsp = 0xce3cad246b15e776, rbp = 0x11092085439d85ac --- Tracing command kernel pid 0 tid 100017 td 0xfffffe002219c720 sched_switch() at sched_switch+0x582/frame 0xfffffe00c7cdadd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00c7cdadf0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00c7cdae30 _sleep() at _sleep+0x2ef/frame 0xfffffe00c7cdaec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00c7cdaef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00c7cdaf30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00c7cdaf30 --- trap 0xcc867169, rip = 0x6a60ac51203238f4, rsp = 0x9f72b638e4ac4185, rbp = 0xc01c4c7f54b96535 --- Tracing command kernel pid 0 tid 100018 td 0xfffffe002219c000 sched_switch() at sched_switch+0x582/frame 0xfffffe00c7cd5dd0 mi_switch() at mi_switch+0x157/frame 0xfffffe00c7cd5df0 sleepq_switch() at sleepq_switch+0x107/frame 0xfffffe00c7cd5e30 _sleep() at _sleep+0x2ef/frame 0xfffffe00c7cd5ec0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb1/frame 0xfffffe00c7cd5ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe00c7cd5f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00c7cd5f30 --- trap 0x2690c9eb, rip = 0x8735b6b9d5a11390, rsp = 0xdbbfb8a77e96f2f5, rbp = 0x25025edca961461a --- -
@ac99 It would have been best to report this in the Development Section of the forum.
-
S stephenw10 moved this topic from General pfSense Questions on Jan 20, 2023, 8:58 PM
-
Moved to the 2.7 section.
That backtrace is not terribly helpful.
Do you see the same thing in every crash report?
Steve
-
Appreciate you moving the post to the proper location.
I didn't hold onto the prior traces figuring they have happened as a result of me becoming familiar with pfSense. The last crash was unique in that system wasn't really doing anything. Just a few connected VPN gateways configured but no load. That is what started me looking. Seeing FreeBSD reporting a deadlock around sleep was just as unexpected given all sleep functionality is disabled in the bios of the system.
I'll keep watching the system and if it happens again, I'll post all the logs for review. In addition to using the dev version of pfSense, the box itself, KingnovyPC 6 Port i226 2.5GbE LAN Fanless, is new enough to merit keeping an eye on.
Will keep watching and post the traces if it happens again. My takeaway here is that the team here would want to see it and not push it to FreeBSD just yet.
Thanks again.
-
The sleep it's referring to there is a sleeping thread rather than a system power state.
Yes comparing multiple crash reports is the next step here. If they are consistent it's probably a bug somewhere.
Steve
Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.