Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Crash dump with 2.3.4 on SG-2440 /var/crash/bounds

    Scheduled Pinned Locked Moved Hardware
    2 Posts 2 Posters 577 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • P
      paul0042
      last edited by

      We have had a couple of crashes recently but this one followed less than an hour after a restart.

      The pfSense is a Sg-2440 dump details pasted below.

      Does anyone have any ideas what may have caused this and what may stop it happening?

      All the Best,
      Paul.

      					Crash report begins.  Anonymous machine information:
      
      amd64
      10.3-RELEASE-p19
      FreeBSD 10.3-RELEASE-p19 #0 bbfdb9a1d(RELENG_2_3_4): Wed May  3 16:12:24 CDT 2017     root@factory23-amd64-builder:/builder/factory-234/tmp/obj/builder/factory-234/tmp/FreeBSD-src/sys/pfSense
      
      Crash report details:
      
      No PHP errors found.
      
      Filename: /var/crash/bounds
      1
      
      Filename: /var/crash/info.0
      Dump header from device /dev/label/swap0
        Architecture: amd64
        Architecture Version: 1
        Dump Length: 73216B (0 MB)
        Blocksize: 512
        Dumptime: Mon Jul 10 11:47:05 2017
        Hostname: pfsense0.??????????????.???????
        Magic: FreeBSD Text Dump
        Version String: FreeBSD 10.3-RELEASE-p19 #0 bbfdb9a1d(RELENG_2_3_4): Wed May  3 16:12:24 CDT 2017
          root@factory23-amd64-builder:/builder/factory-234/tmp/obj/builder/factory-234/tmp/FreeBSD-src/sys/pfSense  Panic String: sbflush_internal: cc 0 || mb 0xfffff80006f0bb00 || mbcnt 795648
        Dump Parity: 608319317
        Bounds: 0
        Dump Status: good
      
      Filename: /var/crash/info.last
      Dump header from device /dev/label/swap0
        Architecture: amd64
        Architecture Version: 1
        Dump Length: 73216B (0 MB)
        Blocksize: 512
        Dumptime: Mon Jul 10 11:47:05 2017
        Hostname: pfsense0.??????????????.???????
        Magic: FreeBSD Text Dump
        Version String: FreeBSD 10.3-RELEASE-p19 #0 bbfdb9a1d(RELENG_2_3_4): Wed May  3 16:12:24 CDT 2017
          root@factory23-amd64-builder:/builder/factory-234/tmp/obj/builder/factory-234/tmp/FreeBSD-src/sys/pfSense  Panic String: sbflush_internal: cc 0 || mb 0xfffff80006f0bb00 || mbcnt 795648
        Dump Parity: 608319317
        Bounds: 0
        Dump Status: good
      
      Filename: /var/crash/textdump.tar.0
      ddb.txt���������������������������������������������������������������������������������������������0600����0�������0�������140000������13130655451�  7075� �����������������������������������������������������������������������������������������������������ustar���root����������������������������wheel������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������db:0:kdb.enter.default>  run lockinfo
      db:1:lockinfo> show locks
      No such command
      db:1:locks>  show alllocks
      No such command
      db:1:alllocks>  show lockedvnods
      Locked vnodes
      db:0:kdb.enter.default>  show pcpu
      cpuid        = 1
      dynamic pcpu = 0xfffffe0099454100
      curthread    = 0xfffff80006951960: pid 97150 "dpinger"
      curpcb       = 0xfffffe0045b71cc0
      fpcurthread  = none
      idlethread   = 0xfffff800032a0960: tid 100004 "idle: cpu1"
      curpmap      = 0xfffff800032aad78
      tssp         = 0xffffffff821138f8
      commontssp   = 0xffffffff821138f8
      rsp0         = 0xfffffe0045b71cc0
      gs32p        = 0xffffffff82115350
      ldt          = 0xffffffff82115390
      tss          = 0xffffffff82115380
      db:0:kdb.enter.default>  bt
      Tracing pid 97150 tid 100281 td 0xfffff80006951960
      kdb_enter() at kdb_enter+0x3e/frame 0xfffffe0045b71780
      vpanic() at vpanic+0x146/frame 0xfffffe0045b717c0
      panic() at panic+0x43/frame 0xfffffe0045b71820
      sbflush_internal() at sbflush_internal+0x7b/frame 0xfffffe0045b71840
      sbdestroy() at sbdestroy+0x12/frame 0xfffffe0045b71860
      sofree() at sofree+0x18e/frame 0xfffffe0045b71890
      soclose() at soclose+0x34f/frame 0xfffffe0045b718d0
      _fdrop() at _fdrop+0x29/frame 0xfffffe0045b718f0
      closef() at closef+0x21e/frame 0xfffffe0045b71980
      fdescfree() at fdescfree+0x4f9/frame 0xfffffe0045b71a30
      exit1() at exit1+0x576/frame 0xfffffe0045b71ac0
      sys_sys_exit() at sys_sys_exit+0xe/frame 0xfffffe0045b71ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b71bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b71bf0
      --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x800b1c18a, rsp = 0x7fffdf9fa5a8, rbp = 0x7fffdf9fa5c0 ---
      db:0:kdb.enter.default>  ps
        pid  ppid  pgrp   uid   state   wmesg         wchan        cmd
      51704     1 51704  1002  Ss      select   0xfffff8012ffb4c40 dhcpd
      46353   293   293     0  S       accept   0xfffff800068e7066 php-fpm
      44672   293   293     0  R       CPU 255                     php-fpm
      42239 51491   293     0  S       nanslp   0xffffffff82001571 sleep
      40434 41872   293     0  S       nanslp   0xffffffff82001571 sleep
      59955     1 59948     0  S       kqread   0xfffff8013857e100 lighttpd_ls
      51491     1   293     0  S       wait     0xfffff80076f56000 sh
      43301 25520 25336   100  S       select   0xfffff8000677b6c0 pinger
      73253 25520 25336   100  S       select   0xfffff8004eca19c0 pinger
      41872     1   293     0  S       wait     0xfffff8000685e4f0 sh
      40758     1 40758     0  Ss      select   0xfffff80006778640 openvpn
      54114     1 54114    59  Ss      (threaded)                  unbound
      100346                   S       kqread   0xfffff80138423000 unbound
      100118                   S       kqread   0xfffff80076c64300 unbound
      29593 25520 25336   100  S       select   0xfffff8012fcad9c0 pinger
      98531     1 98531     0  Ds      biowr    0xfffffe001a433db0 dhcpleases
      97150     1 97150     0  REs     CPU 1                       dpinger
       2597     1  2597    65  Ss      select   0xfffff8004ee63d40 dhclient
      95568     1 95568     0  Ss      select   0xfffff800068a7ec0 dhclient
      53523 53254 53523     0  S+      ttyin    0xfffff8000607c0a8 sh
      53254 52866 53254     0  S+      wait     0xfffff800068ef9e0 sh
      53146 94455 53146     0  Ss      (threaded)                  sshlockout_pf
      100192                   S       nanslp   0xffffffff82001571 sshlockout_pf
      100087                   S       piperd   0xfffff80121d522e8 sshlockout_pf
      52866     1 52866     0  Ss+     wait     0xfffff8012ff2b9e0 login
      52641     1     1     0  S       nanslp   0xffffffff82001570 getty
      37029 36737 36737     0  S       nanslp   0xffffffff82001570 minicron
      36737     1 36737     0  Ss      wait     0xfffff8012ffd99e0 minicron
      36681 36302 36302     0  S       nanslp   0xffffffff82001571 minicron
      36302     1 36302     0  Ss      wait     0xfffff8013803d4f0 minicron
      36077 35618 35618     0  S       nanslp   0xffffffff82001571 minicron
      35618     1 35618     0  Ss      wait     0xfffff8012ffd8000 minicron
      30416 25520 25336   100  S       select   0xfffff800069477c0 pinger
      25520 25336 25336   100  S       kqread   0xfffff8012fe8c700 squid
      25336     1 25336   100  Ss      wait     0xfffff8012ff2d4f0 squid
      94455     1 94455     0  Ss      select   0xfffff800069471c0 syslogd
      75350     1 75350     0  Ss      select   0xfffff8012fcac5c0 powerd
      59905 59727 59905     0  Ss      (threaded)                  charon
      100170                   S       uwait    0xfffff8000691b280 charon
      100169                   S       uwait    0xfffff80006852300 charon
      100168                   S       uwait    0xfffff80006947b80 charon
      100167                   S       uwait    0xfffff80006945e80 charon
      100166                   S       uwait    0xfffff80006852c80 charon
      100165                   S       uwait    0xfffff8000677b480 charon
      100164                   S       uwait    0xfffff80006851500 charon
      100163                   S       uwait    0xfffff8000679ae00 charon
      100162                   S       select   0xfffff8012fcad0c0 charon
      100161                   S       select   0xfffff8012fcacf40 charon
      100160                   S       uwait    0xfffff8004ee63000 charon
      100159                   S       uwait    0xfffff80006778d00 charon
      100158                   S       uwait    0xfffff80121cfb600 charon
      100157                   S       uwait    0xfffff80121cfbc00 charon
      100156                   S       uwait    0xfffff80121cfbb00 charon
      100155                   S       uwait    0xfffff8004ee63280 charon
      100153                   S       sigwait  0xfffff80121d6b000 charon
      59727     1 59727     0  Ss      select   0xfffff800068a7940 starter
      48437     1 48437     0  Ss      (threaded)                  ntpd
      100096                   S       select   0xfffff8000691e940 ntpd
      47979     1 47979     0  Ss      nanslp   0xffffffff82001571 cron
      47204 46819 46819     0  S       kqread   0xfffff80121cccd00 nginx
      47104 46819 46819     0  S       kqread   0xfffff80121cf4600 nginx
      46819     1 46819     0  Ss      pause    0xfffff80006749a88 nginx
      28749     1 28749     0  Ss      bpf      0xfffff8004eccb800 filterlog
      27879     1 27879     0  Ss      select   0xfffff8000677b940 openvpn
      23330     1 23330    65  Ss      select   0xfffff8000691eb40 dhclient
      18134     1 18134     0  Ss      select   0xfffff8000691e040 dhclient
       8169     1  8169     0  Ss      (threaded)                  sshlockout_pf
      100108                   S       nanslp   0xffffffff82001570 sshlockout_pf
      100107                   S       uwait    0xfffff8000677b380 sshlockout_pf
       8092     1  8092     0  Ss      select   0xfffff8000679a940 sshd
        348     1   348     0  Ss      select   0xfffff80006947c40 devd
        335   333   333     0  S       kqread   0xfffff80006bb3700 check_reload_status
        333     1   333     0  Ss      kqread   0xfffff80006bb3600 check_reload_status
        293     1   293     0  Ss      kqread   0xfffff800067afd00 php-fpm
         53     0     0     0  DL      mdwait   0xfffff800066eb800 [md0]
         20     0     0     0  DL      syncer   0xffffffff82052508 [syncer]
         19     0     0     0  DL      vlruwt   0xfffff8000674b9e0 [vnlru]
         18     0     0     0  DL      (threaded)                  [bufdaemon]
      100083                   D       sdflush  0xfffff800066cbee8 [/ worker]
      100071                   D       psleep   0xffffffff82051704 [bufdaemon]
         17     0     0     0  DL      pgzero   0xffffffff8206283c [pagezero]
          9     0     0     0  DL      pollid   0xffffffff81fffe90 [idlepoll]
          8     0     0     0  DL      psleep   0xffffffff82061bc0 [vmdaemon]
          7     0     0     0  DL      (threaded)                  [pagedaemon]
      100075                   D       umarcl   0xffffffff82061540 [uma]
      100067                   D       psleep   0xffffffff82112c04 [pagedaemon]
          6     0     0     0  DL      waiting_ 0xffffffff821036c0 [sctp_iterator]
          5     0     0     0  DL      pftm     0xffffffff80d5db10 [pf purge]
         16     0     0     0  DL      (threaded)                  [usb]
      100056                   D       -        0xfffffe0000ac2e70 [usbus0]
      100055                   D       -        0xfffffe0000ac2e18 [usbus0]
      100054                   D       -        0xfffffe0000ac2dc0 [usbus0]
      100053                   D       -        0xfffffe0000ac2d68 [usbus0]
      100052                   D       -        0xfffffe0000ac2d10 [usbus0]
          4     0     0     0  DL      (threaded)                  [cam]
      100066                   D       -        0xffffffff81f360c8 [scanner]
      100019                   D       -        0xffffffff81f36280 [doneq0]
         15     0     0     0  DL      -        0xffffffff81f579c0 [rand_harvestq]
          3     0     0     0  DL      crypto_r 0xffffffff82060098 [crypto returns]
          2     0     0     0  DL      crypto_w 0xffffffff8205ff40 [crypto]
         14     0     0     0  DL      (threaded)                  [geom]
      100013                   D       -        0xffffffff820f7de8 [g_down]
      100012                   D       -        0xffffffff820f7de0 [g_up]
      100011                   D       -        0xffffffff820f7dd8 [g_event]
         13     0     0     0  DL      (threaded)                  [ng_queue]
      100010                   D       sleep    0xffffffff81ef46f8 [ng_queue1]
      100009                   D       sleep    0xffffffff81ef46f8 [ng_queue0]
         12     0     0     0  WL      (threaded)                  [intr]
      100074                   I                                   [swi1: netisr 1]
      100064                   I                                   [swi1: pfsync]
      100062                   I                                   [swi1: pf send]
      100059                   I                                   [swi0: uart uart]
      100058                   I                                   [irq269: ahci1]
      100057                   I                                   [irq268: ahci0]
      100051                   I                                   [irq22: ehci0]
      100050                   I                                   [irq267: igb3:link]
      100048                   I                                   [irq266: igb3:que 1]
      100046                   I                                   [irq265: igb3:que 0]
      100045                   I                                   [irq264: igb2:link]
      100043                   I                                   [irq263: igb2:que 1]
      100041                   I                                   [irq262: igb2:que 0]
      100040                   I                                   [irq261: igb1:link]
      100038                   I                                   [irq260: igb1:que 1]
      100036                   I                                   [irq259: igb1:que 0]
      100035                   I                                   [irq258: igb0:link]
      100033                   I                                   [irq257: igb0:que 1]
      100031                   I                                   [irq256: igb0:que 0]
      100028                   I                                   [irq16: ath0]
      100027                   I                                   [swi5: fast taskq]
      100025                   I                                   [swi6: Giant taskq]
      100023                   I                                   [swi6: task queue]
      100008                   I                                   [swi4: clock]
      100007                   I                                   [swi4: clock]
      100006                   I                                   [swi1: netisr 0]
      100005                   I                                   [swi3: vm]
         11     0     0     0  RL      (threaded)                  [idle]
      100004                   CanRun                              [idle: cpu1]
      100003                   CanRun                              [idle: cpu0]
          1     0     1     0  SLs     wait     0xfffff8000329e4f0 [init]
         10     0     0     0  DL      audit_wo 0xffffffff8210a160 [audit]
          0     0     0     0  DLs     (threaded)                  [kernel]
      100065                   D       -        0xfffff800032f1200 [CAM taskq]
      100060                   D       -        0xfffff8000608e300 [mca taskq]
      100049                   D       -        0xfffff8000348f300 [igb3 que]
      100047                   D       -        0xfffff8000348f600 [igb3 que]
      100044                   D       -        0xfffff8000601d900 [igb2 que]
      100042                   D       -        0xfffff8000601dc00 [igb2 que]
      100039                   D       -        0xfffff8000601f000 [igb1 que]
      100037                   D       -        0xfffff8000601f300 [igb1 que]
      100034                   D       -        0xfffff800032f2300 [igb0 que]
      100032                   D       -        0xfffff800032f2600 [igb0 que]
      100030                   D       -        0xfffff8000348cb00 [ath0 net80211 taskq]
      100029                   D       -        0xfffff8000348cd00 [ath0 taskq]
      100026                   D       -        0xfffff800032f0100 [thread]
      100024                   D       -        0xfffff800032f0600 [ffs_trim taskq]
      100022                   D       -        0xfffff800032f0d00 [acpi_task_2]
      100021                   D       -        0xfffff800032f0d00 [acpi_task_1]
      100020                   D       -        0xfffff800032f0d00 [acpi_task_0]
      100018                   D       -        0xfffff800032f1400 [kqueue taskq]
      100014                   D       -        0xfffff80003290900 [firmware taskq]
      100000                   D       swapin   0xffffffff820f7ee8 [swapper]
      db:0:kdb.enter.default>  alltrace
      
      Tracing command dhcpd pid 51704 tid 100082 td 0xfffff8000674d000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe00459a7700
      mi_switch() at mi_switch+0xe1/frame 0xfffffe00459a7740
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe00459a77c0
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe00459a77f0
      _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19e/frame 0xfffffe00459a7850
      seltdwait() at seltdwait+0xa4/frame 0xfffffe00459a7890
      kern_select() at kern_select+0x8fa/frame 0xfffffe00459a7a90
      sys_select() at sys_select+0x54/frame 0xfffffe00459a7ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe00459a7bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe00459a7bf0
      --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800d546aa, rsp = 0x7fffffffea18, rbp = 0x7fffffffea40 ---
      
      Tracing command php-fpm pid 46353 tid 100195 td 0xfffff8013855d000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b6c850
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b6c890
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b6c910
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045b6c940
      _sleep() at _sleep+0x27d/frame 0xfffffe0045b6c9c0
      kern_accept4() at kern_accept4+0x1b1/frame 0xfffffe0045b6ca70
      accept1() at accept1+0x63/frame 0xfffffe0045b6cad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b6cbf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b6cbf0
      --- syscall (30, FreeBSD ELF64, sys_accept), rip = 0x801d8101a, rsp = 0x7ffffffec6e8, rbp = 0x7ffffffec720 ---
      
      Tracing command php-fpm pid 44672 tid 100257 td 0xfffff80076b97960
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045a4ab60
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045a4aba0
      critical_exit() at critical_exit+0x7a/frame 0xfffffe0045a4abc0
      lapic_handle_timer() at lapic_handle_timer+0xae/frame 0xfffffe0045a4abf0
      Xtimerint() at Xtimerint+0x8c/frame 0xfffffe0045a4abf0
      --- interrupt, rip = 0x8009cfd23, rsp = 0x7fffffff3580, rbp = 0x7fffffff3580 ---
      
      Tracing command sleep pid 42239 tid 100311 td 0xfffff800069904b0
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045c2c8c0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045c2c900
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045c2c980
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045c2c9b0
      _sleep() at _sleep+0x238/frame 0xfffffe0045c2ca30
      kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe0045c2ca90
      sys_nanosleep() at sys_nanosleep+0x51/frame 0xfffffe0045c2cad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045c2cbf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045c2cbf0
      --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8008f1f7a, rsp = 0x7fffffffec98, rbp = 0x7fffffffecd0 ---
      
      Tracing command sleep pid 40434 tid 100174 td 0xfffff8012ff46960
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b038c0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b03900
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b03980
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045b039b0
      _sleep() at _sleep+0x238/frame 0xfffffe0045b03a30
      kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe0045b03a90
      sys_nanosleep() at sys_nanosleep+0x51/frame 0xfffffe0045b03ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b03bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b03bf0
      --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8008f1f7a, rsp = 0x7fffffffec88, rbp = 0x7fffffffecc0 ---
      
      Tracing command lighttpd_ls pid 59955 tid 100313 td 0xfffff800768e5960
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045c36670
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045c366b0
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045c36730
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045c36760
      _sleep() at _sleep+0x238/frame 0xfffffe0045c367e0
      kern_kevent_fp() at kern_kevent_fp+0x399/frame 0xfffffe0045c369c0
      kern_kevent() at kern_kevent+0x9f/frame 0xfffffe0045c36a20
      sys_kevent() at sys_kevent+0x12a/frame 0xfffffe0045c36ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045c36bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045c36bf0
      --- syscall (363, FreeBSD ELF64, sys_kevent), rip = 0x80121193a, rsp = 0x7fffffffe988, rbp = 0x7fffffffe9d0 ---
      
      Tracing command sh pid 51491 tid 100309 td 0xfffff80121c544b0
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045c226a0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045c226e0
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045c22760
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045c22790
      _sleep() at _sleep+0x27d/frame 0xfffffe0045c22810
      kern_wait6() at kern_wait6+0x5de/frame 0xfffffe0045c228e0
      sys_wait4() at sys_wait4+0x72/frame 0xfffffe0045c22ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045c22bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045c22bf0
      --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800d8bdba, rsp = 0x7fffffffe888, rbp = 0x7fffffffe930 ---
      
      Tracing command pinger pid 43301 tid 100304 td 0xfffff8004ecbd000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045c09700
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045c09740
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045c097c0
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045c097f0
      _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19e/frame 0xfffffe0045c09850
      seltdwait() at seltdwait+0xa4/frame 0xfffffe0045c09890
      kern_select() at kern_select+0x8fa/frame 0xfffffe0045c09a90
      sys_select() at sys_select+0x54/frame 0xfffffe0045c09ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045c09bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045c09bf0
      --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8019c86aa, rsp = 0x7fffffffdbf8, rbp = 0x7fffffffdc30 ---
      
      Tracing command pinger pid 73253 tid 100193 td 0xfffff8013855d960
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b62700
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b62740
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b627c0
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045b627f0
      _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19e/frame 0xfffffe0045b62850
      seltdwait() at seltdwait+0xa4/frame 0xfffffe0045b62890
      kern_select() at kern_select+0x8fa/frame 0xfffffe0045b62a90
      sys_select() at sys_select+0x54/frame 0xfffffe0045b62ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b62bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b62bf0
      --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8019c86aa, rsp = 0x7fffffffdbf8, rbp = 0x7fffffffdc30 ---
      
      Tracing command sh pid 41872 tid 100091 td 0xfffff8000681c000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe00459d46a0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe00459d46e0
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe00459d4760
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00459d4790
      _sleep() at _sleep+0x27d/frame 0xfffffe00459d4810
      kern_wait6() at kern_wait6+0x5de/frame 0xfffffe00459d48e0
      sys_wait4() at sys_wait4+0x72/frame 0xfffffe00459d4ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe00459d4bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe00459d4bf0
      --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800d8bdba, rsp = 0x7fffffffe888, rbp = 0x7fffffffe930 ---
      
      Tracing command openvpn pid 40758 tid 100139 td 0xfffff80121c57960
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045a09780
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045a097c0
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045a09840
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045a09870
      _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19e/frame 0xfffffe0045a098d0
      seltdwait() at seltdwait+0xa4/frame 0xfffffe0045a09910
      kern_poll() at kern_poll+0x464/frame 0xfffffe0045a09ab0
      sys_poll() at sys_poll+0x61/frame 0xfffffe0045a09ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045a09bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045a09bf0
      --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x80120df3a, rsp = 0x7fffffffd128, rbp = 0x7fffffffd180 ---
      
      Tracing command unbound pid 54114 tid 100346 td 0xfffff80076ed3960
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045a72670
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045a726b0
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045a72730
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045a72760
      _sleep() at _sleep+0x27d/frame 0xfffffe0045a727e0
      kern_kevent_fp() at kern_kevent_fp+0x399/frame 0xfffffe0045a729c0
      kern_kevent() at kern_kevent+0x9f/frame 0xfffffe0045a72a20
      sys_kevent() at sys_kevent+0x12a/frame 0xfffffe0045a72ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045a72bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045a72bf0
      --- syscall (363, FreeBSD ELF64, sys_kevent), rip = 0x80169a93a, rsp = 0x7fffdf7f9e18, rbp = 0x7fffdf7f9e60 ---
      
      Tracing command unbound pid 54114 tid 100118 td 0xfffff8004ec964b0
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045a5e670
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045a5e6b0
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045a5e730
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045a5e760
      _sleep() at _sleep+0x27d/frame 0xfffffe0045a5e7e0
      kern_kevent_fp() at kern_kevent_fp+0x399/frame 0xfffffe0045a5e9c0
      kern_kevent() at kern_kevent+0x9f/frame 0xfffffe0045a5ea20
      sys_kevent() at sys_kevent+0x12a/frame 0xfffffe0045a5ead0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045a5ebf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045a5ebf0
      --- syscall (363, FreeBSD ELF64, sys_kevent), rip = 0x80169a93a, rsp = 0x7fffffffea08, rbp = 0x7fffffffea50 ---
      
      Tracing command pinger pid 29593 tid 100273 td 0xfffff80121c57000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045a45700
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045a45740
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045a457c0
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045a457f0
      _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19e/frame 0xfffffe0045a45850
      seltdwait() at seltdwait+0xa4/frame 0xfffffe0045a45890
      kern_select() at kern_select+0x8fa/frame 0xfffffe0045a45a90
      sys_select() at sys_select+0x54/frame 0xfffffe0045a45ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045a45bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045a45bf0
      --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8019c86aa, rsp = 0x7fffffffdbf8, rbp = 0x7fffffffdc30 ---
      
      Tracing command dhcpleases pid 98531 tid 100202 td 0xfffff8013865c960
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b8f390
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b8f3d0
      sleepq_wait() at sleepq_wait+0x3a/frame 0xfffffe0045b8f400
      _sleep() at _sleep+0x287/frame 0xfffffe0045b8f480
      bwait() at bwait+0x8e/frame 0xfffffe0045b8f4d0
      bufwrite() at bufwrite+0x1d8/frame 0xfffffe0045b8f510
      ffs_update() at ffs_update+0x261/frame 0xfffffe0045b8f590
      ffs_write() at ffs_write+0x54b/frame 0xfffffe0045b8f630
      VOP_WRITE_APV() at VOP_WRITE_APV+0x149/frame 0xfffffe0045b8f740
      vn_write() at vn_write+0x248/frame 0xfffffe0045b8f7c0
      vn_io_fault_doio() at vn_io_fault_doio+0x22/frame 0xfffffe0045b8f800
      vn_io_fault1() at vn_io_fault1+0x1ac/frame 0xfffffe0045b8f960
      vn_io_fault() at vn_io_fault+0x18b/frame 0xfffffe0045b8f9e0
      dofilewrite() at dofilewrite+0x87/frame 0xfffffe0045b8fa30
      kern_writev() at kern_writev+0x68/frame 0xfffffe0045b8fa80
      sys_write() at sys_write+0x63/frame 0xfffffe0045b8fad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b8fbf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b8fbf0
      --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x80095768a, rsp = 0x7fffffffe348, rbp = 0x7fffffffe370 ---
      
      Tracing command dpinger pid 97150 tid 100281 td 0xfffff80006951960
      kdb_enter() at kdb_enter+0x3e/frame 0xfffffe0045b71780
      vpanic() at vpanic+0x146/frame 0xfffffe0045b717c0
      panic() at panic+0x43/frame 0xfffffe0045b71820
      sbflush_internal() at sbflush_internal+0x7b/frame 0xfffffe0045b71840
      sbdestroy() at sbdestroy+0x12/frame 0xfffffe0045b71860
      sofree() at sofree+0x18e/frame 0xfffffe0045b71890
      soclose() at soclose+0x34f/frame 0xfffffe0045b718d0
      _fdrop() at _fdrop+0x29/frame 0xfffffe0045b718f0
      closef() at closef+0x21e/frame 0xfffffe0045b71980
      fdescfree() at fdescfree+0x4f9/frame 0xfffffe0045b71a30
      exit1() at exit1+0x576/frame 0xfffffe0045b71ac0
      sys_sys_exit() at sys_sys_exit+0xe/frame 0xfffffe0045b71ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b71bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b71bf0
      --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x800b1c18a, rsp = 0x7fffdf9fa5a8, rbp = 0x7fffdf9fa5c0 ---
      
      Tracing command dhclient pid 2597 tid 100209 td 0xfffff80121e774b0
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045bb2780
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045bb27c0
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045bb2840
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045bb2870
      _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19e/frame 0xfffffe0045bb28d0
      seltdwait() at seltdwait+0xa4/frame 0xfffffe0045bb2910
      kern_poll() at kern_poll+0x464/frame 0xfffffe0045bb2ab0
      sys_poll() at sys_poll+0x61/frame 0xfffffe0045bb2ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045bb2bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045bb2bf0
      --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x800b18f3a, rsp = 0x7fffffffec08, rbp = 0x7fffffffec40 ---
      
      Tracing command dhclient pid 95568 tid 100184 td 0xfffff8012ff344b0
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b35790
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b357d0
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b35850
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045b35880
      _cv_wait_sig() at _cv_wait_sig+0x17a/frame 0xfffffe0045b358d0
      seltdwait() at seltdwait+0xae/frame 0xfffffe0045b35910
      kern_poll() at kern_poll+0x464/frame 0xfffffe0045b35ab0
      sys_poll() at sys_poll+0x61/frame 0xfffffe0045b35ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b35bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b35bf0
      --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x800b18f3a, rsp = 0x7fffffffec18, rbp = 0x7fffffffec40 ---
      
      Tracing command sh pid 53523 tid 100094 td 0xfffff800068a0000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe00459e3790
      mi_switch() at mi_switch+0xe1/frame 0xfffffe00459e37d0
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe00459e3850
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00459e3880
      _cv_wait_sig() at _cv_wait_sig+0x17a/frame 0xfffffe00459e38d0
      tty_wait() at tty_wait+0x1c/frame 0xfffffe00459e38f0
      ttydisc_read() at ttydisc_read+0xf4/frame 0xfffffe00459e3950
      ttydev_read() at ttydev_read+0x86/frame 0xfffffe00459e3980
      devfs_read_f() at devfs_read_f+0xd8/frame 0xfffffe00459e39e0
      dofileread() at dofileread+0x95/frame 0xfffffe00459e3a30
      kern_readv() at kern_readv+0x68/frame 0xfffffe00459e3a80
      sys_read() at sys_read+0x63/frame 0xfffffe00459e3ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe00459e3bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe00459e3bf0
      --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800dee6ca, rsp = 0x7fffffffe768, rbp = 0x7fffffffe860 ---
      
      Tracing command sh pid 53254 tid 100123 td 0xfffff8000681c960
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe00459ca6a0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe00459ca6e0
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe00459ca760
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00459ca790
      _sleep() at _sleep+0x27d/frame 0xfffffe00459ca810
      kern_wait6() at kern_wait6+0x5de/frame 0xfffffe00459ca8e0
      sys_wait4() at sys_wait4+0x72/frame 0xfffffe00459caad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe00459cabf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe00459cabf0
      --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800d8bdba, rsp = 0x7fffffffe818, rbp = 0x7fffffffe8c0 ---
      
      Tracing command sshlockout_pf pid 53146 tid 100192 td 0xfffff8012ff37960
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b5d8c0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b5d900
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b5d980
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045b5d9b0
      _sleep() at _sleep+0x238/frame 0xfffffe0045b5da30
      kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe0045b5da90
      sys_nanosleep() at sys_nanosleep+0x51/frame 0xfffffe0045b5dad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b5dbf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b5dbf0
      --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x800b18f7a, rsp = 0x7fffdfffdf48, rbp = 0x7fffdfffdf70 ---
      
      Tracing command sshlockout_pf pid 53146 tid 100087 td 0xfffff8000681d4b0
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe00459c0810
      mi_switch() at mi_switch+0xe1/frame 0xfffffe00459c0850
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe00459c08d0
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00459c0900
      _sleep() at _sleep+0x27d/frame 0xfffffe00459c0980
      pipe_read() at pipe_read+0x2aa/frame 0xfffffe00459c09e0
      dofileread() at dofileread+0x95/frame 0xfffffe00459c0a30
      kern_readv() at kern_readv+0x68/frame 0xfffffe00459c0a80
      sys_read() at sys_read+0x63/frame 0xfffffe00459c0ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe00459c0bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe00459c0bf0
      --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800b7b6ca, rsp = 0x7fffffffe7b8, rbp = 0x7fffffffe7d0 ---
      
      Tracing command login pid 52866 tid 100180 td 0xfffff8012ff34960
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b216a0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b216e0
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b21760
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045b21790
      _sleep() at _sleep+0x27d/frame 0xfffffe0045b21810
      kern_wait6() at kern_wait6+0x5de/frame 0xfffffe0045b218e0
      sys_wait4() at sys_wait4+0x72/frame 0xfffffe0045b21ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b21bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b21bf0
      --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800d15dba, rsp = 0x7fffffffebc8, rbp = 0x7fffffffed40 ---
      
      Tracing command getty pid 52641 tid 100076 td 0xfffff8000674f000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe00459898c0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045989900
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045989980
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe00459899b0
      _sleep() at _sleep+0x238/frame 0xfffffe0045989a30
      kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe0045989a90
      sys_nanosleep() at sys_nanosleep+0x51/frame 0xfffffe0045989ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045989bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045989bf0
      --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x800b09f7a, rsp = 0x7fffffffec38, rbp = 0x7fffffffec90 ---
      
      Tracing command minicron pid 37029 tid 100098 td 0xfffff80006903960
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe00459f78c0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe00459f7900
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe00459f7980
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe00459f79b0
      _sleep() at _sleep+0x238/frame 0xfffffe00459f7a30
      kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe00459f7a90
      sys_nanosleep() at sys_nanosleep+0x51/frame 0xfffffe00459f7ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe00459f7bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe00459f7bf0
      --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8008f2f7a, rsp = 0x7fffffffeb88, rbp = 0x7fffffffebe0 ---
      
      Tracing command minicron pid 36737 tid 100181 td 0xfffff8013801a960
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b266a0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b266e0
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b26760
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045b26790
      _sleep() at _sleep+0x27d/frame 0xfffffe0045b26810
      kern_wait6() at kern_wait6+0x5de/frame 0xfffffe0045b268e0
      sys_wait4() at sys_wait4+0x72/frame 0xfffffe0045b26ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b26bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b26bf0
      --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8008f2dba, rsp = 0x7fffffffebe8, rbp = 0x7fffffffec30 ---
      
      Tracing command minicron pid 36681 tid 100177 td 0xfffff8012fe6c960
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b128c0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b12900
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b12980
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045b129b0
      _sleep() at _sleep+0x238/frame 0xfffffe0045b12a30
      kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe0045b12a90
      sys_nanosleep() at sys_nanosleep+0x51/frame 0xfffffe0045b12ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b12bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b12bf0
      --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8008f2f7a, rsp = 0x7fffffffeb98, rbp = 0x7fffffffebf0 ---
      
      Tracing command minicron pid 36302 tid 100190 td 0xfffff801380284b0
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b536a0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b536e0
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b53760
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045b53790
      _sleep() at _sleep+0x27d/frame 0xfffffe0045b53810
      kern_wait6() at kern_wait6+0x5de/frame 0xfffffe0045b538e0
      sys_wait4() at sys_wait4+0x72/frame 0xfffffe0045b53ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b53bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b53bf0
      --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8008f2dba, rsp = 0x7fffffffebf8, rbp = 0x7fffffffec40 ---
      
      Tracing command minicron pid 36077 tid 100189 td 0xfffff80138028960
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b4e8c0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b4e900
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b4e980
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045b4e9b0
      _sleep() at _sleep+0x238/frame 0xfffffe0045b4ea30
      kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe0045b4ea90
      sys_nanosleep() at sys_nanosleep+0x51/frame 0xfffffe0045b4ead0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b4ebf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b4ebf0
      --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8008f2f7a, rsp = 0x7fffffffebb8, rbp = 0x7fffffffec10 ---
      
      Tracing command minicron pid 35618 tid 100188 td 0xfffff80138029000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b496a0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b496e0
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b49760
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045b49790
      _sleep() at _sleep+0x27d/frame 0xfffffe0045b49810
      kern_wait6() at kern_wait6+0x5de/frame 0xfffffe0045b498e0
      sys_wait4() at sys_wait4+0x72/frame 0xfffffe0045b49ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b49bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b49bf0
      --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8008f2dba, rsp = 0x7fffffffec18, rbp = 0x7fffffffec60 ---
      
      Tracing command pinger pid 30416 tid 100187 td 0xfffff801380294b0
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b44700
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b44740
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b447c0
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045b447f0
      _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19e/frame 0xfffffe0045b44850
      seltdwait() at seltdwait+0xa4/frame 0xfffffe0045b44890
      kern_select() at kern_select+0x8fa/frame 0xfffffe0045b44a90
      sys_select() at sys_select+0x54/frame 0xfffffe0045b44ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b44bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b44bf0
      --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8019c86aa, rsp = 0x7fffffffdbf8, rbp = 0x7fffffffdc30 ---
      
      Tracing command squid pid 25520 tid 100191 td 0xfffff80138028000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b58670
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b586b0
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b58730
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045b58760
      _sleep() at _sleep+0x238/frame 0xfffffe0045b587e0
      kern_kevent_fp() at kern_kevent_fp+0x399/frame 0xfffffe0045b589c0
      kern_kevent() at kern_kevent+0x9f/frame 0xfffffe0045b58a20
      sys_kevent() at sys_kevent+0x12a/frame 0xfffffe0045b58ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b58bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b58bf0
      --- syscall (363, FreeBSD ELF64, sys_kevent), rip = 0x802d5b93a, rsp = 0x7fffffffe898, rbp = 0x7fffffffe8e0 ---
      
      Tracing command squid pid 25336 tid 100172 td 0xfffff800069034b0
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045af96a0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045af96e0
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045af9760
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045af9790
      _sleep() at _sleep+0x27d/frame 0xfffffe0045af9810
      kern_wait6() at kern_wait6+0x5de/frame 0xfffffe0045af98e0
      sys_wait4() at sys_wait4+0x72/frame 0xfffffe0045af9ad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045af9bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045af9bf0
      --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x802d50dba, rsp = 0x7fffffffe998, rbp = 0x7fffffffe9d0 ---
      
      Tracing command syslogd pid 94455 tid 100173 td 0xfffff8012fe6d000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045afe710
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045afe750
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045afe7d0
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045afe800
      _cv_wait_sig() at _cv_wait_sig+0x17a/frame 0xfffffe0045afe850
      seltdwait() at seltdwait+0xae/frame 0xfffffe0045afe890
      kern_select() at kern_select+0x8fa/frame 0xfffffe0045afea90
      sys_select() at sys_select+0x54/frame 0xfffffe0045afead0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045afebf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045afebf0
      --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800b716aa, rsp = 0x7fffffffdbe8, rbp = 0x7fffffffec00 ---
      
      Tracing command powerd pid 75350 tid 100154 td 0xfffff80121dfa4b0
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045a9f700
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045a9f740
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045a9f7c0
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045a9f7f0
      _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19e/frame 0xfffffe0045a9f850
      seltdwait() at seltdwait+0xa4/frame 0xfffffe0045a9f890
      kern_select() at kern_select+0x8fa/frame 0xfffffe0045a9fa90
      sys_select() at sys_select+0x54/frame 0xfffffe0045a9fad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045a9fbf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045a9fbf0
      --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800b6a6aa, rsp = 0x7fffffffeac8, rbp = 0x7fffffffec50 ---
      
      Tracing command charon pid 59905 tid 100170 td 0xfffff80006951000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045aef840
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045aef880
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045aef900
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045aef930
      _sleep() at _sleep+0x27d/frame 0xfffffe0045aef9b0
      umtxq_sleep() at umtxq_sleep+0x125/frame 0xfffffe0045aefa10
      do_wait() at do_wait+0x387/frame 0xfffffe0045aefa90
      __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x83/frame 0xfffffe0045aefad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045aefbf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045aefbf0
      --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x800f5013c, rsp = 0x7fffde1eee58, rbp = 0x7fffde1eee80 ---
      
      Tracing command charon pid 59905 tid 100169 td 0xfffff8004ec974b0
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045aea840
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045aea880
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045aea900
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045aea930
      _sleep() at _sleep+0x27d/frame 0xfffffe0045aea9b0
      umtxq_sleep() at umtxq_sleep+0x125/frame 0xfffffe0045aeaa10
      do_wait() at do_wait+0x387/frame 0xfffffe0045aeaa90
      __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x83/frame 0xfffffe0045aeaad0
      amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045aeabf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045aeabf0
      --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x800f5013c, rsp = 0x7fffde3efe58, rbp = 0x7fffde3efe80 ---
      
      Tracing command charon pid 59905 tid 100168 td 0xfffff8004ec984b0
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045ae5840
      mi_switch() at mi_switch+0xe1/frame 0xfffffe0045ae5880
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045ae5900
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045ae5930
      _sleep() at _sleep+0x27d/frame 0xfffffe0045ae59b0
      umtxq_sleep() at umtxq_sleep+0x125/frame 0xfffffe0045ae5a10
      do_wait() at do_wait+0x387/frame 0xfffffe0045ae5a90
      __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x83/frame 0xfffffe0045ae5ad0
      a[/thread]
      
      1 Reply Last reply Reply Quote 0
      • jimpJ
        jimp Rebel Alliance Developer Netgate
        last edited by

        Looks like this:

        https://redmine.pfsense.org/issues/4689

        Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

        Need help fast? Netgate Global Support!

        Do not Chat/PM for help!

        1 Reply Last reply Reply Quote 0
        • First post
          Last post
        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.