• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

Crash dump with 2.3.4 on SG-2440 /var/crash/bounds

Scheduled Pinned Locked Moved Hardware
2 Posts 2 Posters 576 Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • P
    paul0042
    last edited by Jul 10, 2017, 3:31 PM

    We have had a couple of crashes recently but this one followed less than an hour after a restart.

    The pfSense is a Sg-2440 dump details pasted below.

    Does anyone have any ideas what may have caused this and what may stop it happening?

    All the Best,
    Paul.

    					Crash report begins.  Anonymous machine information:
    
    amd64
    10.3-RELEASE-p19
    FreeBSD 10.3-RELEASE-p19 #0 bbfdb9a1d(RELENG_2_3_4): Wed May  3 16:12:24 CDT 2017     root@factory23-amd64-builder:/builder/factory-234/tmp/obj/builder/factory-234/tmp/FreeBSD-src/sys/pfSense
    
    Crash report details:
    
    No PHP errors found.
    
    Filename: /var/crash/bounds
    1
    
    Filename: /var/crash/info.0
    Dump header from device /dev/label/swap0
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 73216B (0 MB)
      Blocksize: 512
      Dumptime: Mon Jul 10 11:47:05 2017
      Hostname: pfsense0.??????????????.???????
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p19 #0 bbfdb9a1d(RELENG_2_3_4): Wed May  3 16:12:24 CDT 2017
        root@factory23-amd64-builder:/builder/factory-234/tmp/obj/builder/factory-234/tmp/FreeBSD-src/sys/pfSense  Panic String: sbflush_internal: cc 0 || mb 0xfffff80006f0bb00 || mbcnt 795648
      Dump Parity: 608319317
      Bounds: 0
      Dump Status: good
    
    Filename: /var/crash/info.last
    Dump header from device /dev/label/swap0
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 73216B (0 MB)
      Blocksize: 512
      Dumptime: Mon Jul 10 11:47:05 2017
      Hostname: pfsense0.??????????????.???????
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p19 #0 bbfdb9a1d(RELENG_2_3_4): Wed May  3 16:12:24 CDT 2017
        root@factory23-amd64-builder:/builder/factory-234/tmp/obj/builder/factory-234/tmp/FreeBSD-src/sys/pfSense  Panic String: sbflush_internal: cc 0 || mb 0xfffff80006f0bb00 || mbcnt 795648
      Dump Parity: 608319317
      Bounds: 0
      Dump Status: good
    
    Filename: /var/crash/textdump.tar.0
    ddb.txt���������������������������������������������������������������������������������������������0600����0�������0�������140000������13130655451�  7075� �����������������������������������������������������������������������������������������������������ustar���root����������������������������wheel������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������db:0:kdb.enter.default>  run lockinfo
    db:1:lockinfo> show locks
    No such command
    db:1:locks>  show alllocks
    No such command
    db:1:alllocks>  show lockedvnods
    Locked vnodes
    db:0:kdb.enter.default>  show pcpu
    cpuid        = 1
    dynamic pcpu = 0xfffffe0099454100
    curthread    = 0xfffff80006951960: pid 97150 "dpinger"
    curpcb       = 0xfffffe0045b71cc0
    fpcurthread  = none
    idlethread   = 0xfffff800032a0960: tid 100004 "idle: cpu1"
    curpmap      = 0xfffff800032aad78
    tssp         = 0xffffffff821138f8
    commontssp   = 0xffffffff821138f8
    rsp0         = 0xfffffe0045b71cc0
    gs32p        = 0xffffffff82115350
    ldt          = 0xffffffff82115390
    tss          = 0xffffffff82115380
    db:0:kdb.enter.default>  bt
    Tracing pid 97150 tid 100281 td 0xfffff80006951960
    kdb_enter() at kdb_enter+0x3e/frame 0xfffffe0045b71780
    vpanic() at vpanic+0x146/frame 0xfffffe0045b717c0
    panic() at panic+0x43/frame 0xfffffe0045b71820
    sbflush_internal() at sbflush_internal+0x7b/frame 0xfffffe0045b71840
    sbdestroy() at sbdestroy+0x12/frame 0xfffffe0045b71860
    sofree() at sofree+0x18e/frame 0xfffffe0045b71890
    soclose() at soclose+0x34f/frame 0xfffffe0045b718d0
    _fdrop() at _fdrop+0x29/frame 0xfffffe0045b718f0
    closef() at closef+0x21e/frame 0xfffffe0045b71980
    fdescfree() at fdescfree+0x4f9/frame 0xfffffe0045b71a30
    exit1() at exit1+0x576/frame 0xfffffe0045b71ac0
    sys_sys_exit() at sys_sys_exit+0xe/frame 0xfffffe0045b71ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b71bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b71bf0
    --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x800b1c18a, rsp = 0x7fffdf9fa5a8, rbp = 0x7fffdf9fa5c0 ---
    db:0:kdb.enter.default>  ps
      pid  ppid  pgrp   uid   state   wmesg         wchan        cmd
    51704     1 51704  1002  Ss      select   0xfffff8012ffb4c40 dhcpd
    46353   293   293     0  S       accept   0xfffff800068e7066 php-fpm
    44672   293   293     0  R       CPU 255                     php-fpm
    42239 51491   293     0  S       nanslp   0xffffffff82001571 sleep
    40434 41872   293     0  S       nanslp   0xffffffff82001571 sleep
    59955     1 59948     0  S       kqread   0xfffff8013857e100 lighttpd_ls
    51491     1   293     0  S       wait     0xfffff80076f56000 sh
    43301 25520 25336   100  S       select   0xfffff8000677b6c0 pinger
    73253 25520 25336   100  S       select   0xfffff8004eca19c0 pinger
    41872     1   293     0  S       wait     0xfffff8000685e4f0 sh
    40758     1 40758     0  Ss      select   0xfffff80006778640 openvpn
    54114     1 54114    59  Ss      (threaded)                  unbound
    100346                   S       kqread   0xfffff80138423000 unbound
    100118                   S       kqread   0xfffff80076c64300 unbound
    29593 25520 25336   100  S       select   0xfffff8012fcad9c0 pinger
    98531     1 98531     0  Ds      biowr    0xfffffe001a433db0 dhcpleases
    97150     1 97150     0  REs     CPU 1                       dpinger
     2597     1  2597    65  Ss      select   0xfffff8004ee63d40 dhclient
    95568     1 95568     0  Ss      select   0xfffff800068a7ec0 dhclient
    53523 53254 53523     0  S+      ttyin    0xfffff8000607c0a8 sh
    53254 52866 53254     0  S+      wait     0xfffff800068ef9e0 sh
    53146 94455 53146     0  Ss      (threaded)                  sshlockout_pf
    100192                   S       nanslp   0xffffffff82001571 sshlockout_pf
    100087                   S       piperd   0xfffff80121d522e8 sshlockout_pf
    52866     1 52866     0  Ss+     wait     0xfffff8012ff2b9e0 login
    52641     1     1     0  S       nanslp   0xffffffff82001570 getty
    37029 36737 36737     0  S       nanslp   0xffffffff82001570 minicron
    36737     1 36737     0  Ss      wait     0xfffff8012ffd99e0 minicron
    36681 36302 36302     0  S       nanslp   0xffffffff82001571 minicron
    36302     1 36302     0  Ss      wait     0xfffff8013803d4f0 minicron
    36077 35618 35618     0  S       nanslp   0xffffffff82001571 minicron
    35618     1 35618     0  Ss      wait     0xfffff8012ffd8000 minicron
    30416 25520 25336   100  S       select   0xfffff800069477c0 pinger
    25520 25336 25336   100  S       kqread   0xfffff8012fe8c700 squid
    25336     1 25336   100  Ss      wait     0xfffff8012ff2d4f0 squid
    94455     1 94455     0  Ss      select   0xfffff800069471c0 syslogd
    75350     1 75350     0  Ss      select   0xfffff8012fcac5c0 powerd
    59905 59727 59905     0  Ss      (threaded)                  charon
    100170                   S       uwait    0xfffff8000691b280 charon
    100169                   S       uwait    0xfffff80006852300 charon
    100168                   S       uwait    0xfffff80006947b80 charon
    100167                   S       uwait    0xfffff80006945e80 charon
    100166                   S       uwait    0xfffff80006852c80 charon
    100165                   S       uwait    0xfffff8000677b480 charon
    100164                   S       uwait    0xfffff80006851500 charon
    100163                   S       uwait    0xfffff8000679ae00 charon
    100162                   S       select   0xfffff8012fcad0c0 charon
    100161                   S       select   0xfffff8012fcacf40 charon
    100160                   S       uwait    0xfffff8004ee63000 charon
    100159                   S       uwait    0xfffff80006778d00 charon
    100158                   S       uwait    0xfffff80121cfb600 charon
    100157                   S       uwait    0xfffff80121cfbc00 charon
    100156                   S       uwait    0xfffff80121cfbb00 charon
    100155                   S       uwait    0xfffff8004ee63280 charon
    100153                   S       sigwait  0xfffff80121d6b000 charon
    59727     1 59727     0  Ss      select   0xfffff800068a7940 starter
    48437     1 48437     0  Ss      (threaded)                  ntpd
    100096                   S       select   0xfffff8000691e940 ntpd
    47979     1 47979     0  Ss      nanslp   0xffffffff82001571 cron
    47204 46819 46819     0  S       kqread   0xfffff80121cccd00 nginx
    47104 46819 46819     0  S       kqread   0xfffff80121cf4600 nginx
    46819     1 46819     0  Ss      pause    0xfffff80006749a88 nginx
    28749     1 28749     0  Ss      bpf      0xfffff8004eccb800 filterlog
    27879     1 27879     0  Ss      select   0xfffff8000677b940 openvpn
    23330     1 23330    65  Ss      select   0xfffff8000691eb40 dhclient
    18134     1 18134     0  Ss      select   0xfffff8000691e040 dhclient
     8169     1  8169     0  Ss      (threaded)                  sshlockout_pf
    100108                   S       nanslp   0xffffffff82001570 sshlockout_pf
    100107                   S       uwait    0xfffff8000677b380 sshlockout_pf
     8092     1  8092     0  Ss      select   0xfffff8000679a940 sshd
      348     1   348     0  Ss      select   0xfffff80006947c40 devd
      335   333   333     0  S       kqread   0xfffff80006bb3700 check_reload_status
      333     1   333     0  Ss      kqread   0xfffff80006bb3600 check_reload_status
      293     1   293     0  Ss      kqread   0xfffff800067afd00 php-fpm
       53     0     0     0  DL      mdwait   0xfffff800066eb800 [md0]
       20     0     0     0  DL      syncer   0xffffffff82052508 [syncer]
       19     0     0     0  DL      vlruwt   0xfffff8000674b9e0 [vnlru]
       18     0     0     0  DL      (threaded)                  [bufdaemon]
    100083                   D       sdflush  0xfffff800066cbee8 [/ worker]
    100071                   D       psleep   0xffffffff82051704 [bufdaemon]
       17     0     0     0  DL      pgzero   0xffffffff8206283c [pagezero]
        9     0     0     0  DL      pollid   0xffffffff81fffe90 [idlepoll]
        8     0     0     0  DL      psleep   0xffffffff82061bc0 [vmdaemon]
        7     0     0     0  DL      (threaded)                  [pagedaemon]
    100075                   D       umarcl   0xffffffff82061540 [uma]
    100067                   D       psleep   0xffffffff82112c04 [pagedaemon]
        6     0     0     0  DL      waiting_ 0xffffffff821036c0 [sctp_iterator]
        5     0     0     0  DL      pftm     0xffffffff80d5db10 [pf purge]
       16     0     0     0  DL      (threaded)                  [usb]
    100056                   D       -        0xfffffe0000ac2e70 [usbus0]
    100055                   D       -        0xfffffe0000ac2e18 [usbus0]
    100054                   D       -        0xfffffe0000ac2dc0 [usbus0]
    100053                   D       -        0xfffffe0000ac2d68 [usbus0]
    100052                   D       -        0xfffffe0000ac2d10 [usbus0]
        4     0     0     0  DL      (threaded)                  [cam]
    100066                   D       -        0xffffffff81f360c8 [scanner]
    100019                   D       -        0xffffffff81f36280 [doneq0]
       15     0     0     0  DL      -        0xffffffff81f579c0 [rand_harvestq]
        3     0     0     0  DL      crypto_r 0xffffffff82060098 [crypto returns]
        2     0     0     0  DL      crypto_w 0xffffffff8205ff40 [crypto]
       14     0     0     0  DL      (threaded)                  [geom]
    100013                   D       -        0xffffffff820f7de8 [g_down]
    100012                   D       -        0xffffffff820f7de0 [g_up]
    100011                   D       -        0xffffffff820f7dd8 [g_event]
       13     0     0     0  DL      (threaded)                  [ng_queue]
    100010                   D       sleep    0xffffffff81ef46f8 [ng_queue1]
    100009                   D       sleep    0xffffffff81ef46f8 [ng_queue0]
       12     0     0     0  WL      (threaded)                  [intr]
    100074                   I                                   [swi1: netisr 1]
    100064                   I                                   [swi1: pfsync]
    100062                   I                                   [swi1: pf send]
    100059                   I                                   [swi0: uart uart]
    100058                   I                                   [irq269: ahci1]
    100057                   I                                   [irq268: ahci0]
    100051                   I                                   [irq22: ehci0]
    100050                   I                                   [irq267: igb3:link]
    100048                   I                                   [irq266: igb3:que 1]
    100046                   I                                   [irq265: igb3:que 0]
    100045                   I                                   [irq264: igb2:link]
    100043                   I                                   [irq263: igb2:que 1]
    100041                   I                                   [irq262: igb2:que 0]
    100040                   I                                   [irq261: igb1:link]
    100038                   I                                   [irq260: igb1:que 1]
    100036                   I                                   [irq259: igb1:que 0]
    100035                   I                                   [irq258: igb0:link]
    100033                   I                                   [irq257: igb0:que 1]
    100031                   I                                   [irq256: igb0:que 0]
    100028                   I                                   [irq16: ath0]
    100027                   I                                   [swi5: fast taskq]
    100025                   I                                   [swi6: Giant taskq]
    100023                   I                                   [swi6: task queue]
    100008                   I                                   [swi4: clock]
    100007                   I                                   [swi4: clock]
    100006                   I                                   [swi1: netisr 0]
    100005                   I                                   [swi3: vm]
       11     0     0     0  RL      (threaded)                  [idle]
    100004                   CanRun                              [idle: cpu1]
    100003                   CanRun                              [idle: cpu0]
        1     0     1     0  SLs     wait     0xfffff8000329e4f0 [init]
       10     0     0     0  DL      audit_wo 0xffffffff8210a160 [audit]
        0     0     0     0  DLs     (threaded)                  [kernel]
    100065                   D       -        0xfffff800032f1200 [CAM taskq]
    100060                   D       -        0xfffff8000608e300 [mca taskq]
    100049                   D       -        0xfffff8000348f300 [igb3 que]
    100047                   D       -        0xfffff8000348f600 [igb3 que]
    100044                   D       -        0xfffff8000601d900 [igb2 que]
    100042                   D       -        0xfffff8000601dc00 [igb2 que]
    100039                   D       -        0xfffff8000601f000 [igb1 que]
    100037                   D       -        0xfffff8000601f300 [igb1 que]
    100034                   D       -        0xfffff800032f2300 [igb0 que]
    100032                   D       -        0xfffff800032f2600 [igb0 que]
    100030                   D       -        0xfffff8000348cb00 [ath0 net80211 taskq]
    100029                   D       -        0xfffff8000348cd00 [ath0 taskq]
    100026                   D       -        0xfffff800032f0100 [thread]
    100024                   D       -        0xfffff800032f0600 [ffs_trim taskq]
    100022                   D       -        0xfffff800032f0d00 [acpi_task_2]
    100021                   D       -        0xfffff800032f0d00 [acpi_task_1]
    100020                   D       -        0xfffff800032f0d00 [acpi_task_0]
    100018                   D       -        0xfffff800032f1400 [kqueue taskq]
    100014                   D       -        0xfffff80003290900 [firmware taskq]
    100000                   D       swapin   0xffffffff820f7ee8 [swapper]
    db:0:kdb.enter.default>  alltrace
    
    Tracing command dhcpd pid 51704 tid 100082 td 0xfffff8000674d000
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe00459a7700
    mi_switch() at mi_switch+0xe1/frame 0xfffffe00459a7740
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe00459a77c0
    sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe00459a77f0
    _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19e/frame 0xfffffe00459a7850
    seltdwait() at seltdwait+0xa4/frame 0xfffffe00459a7890
    kern_select() at kern_select+0x8fa/frame 0xfffffe00459a7a90
    sys_select() at sys_select+0x54/frame 0xfffffe00459a7ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe00459a7bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe00459a7bf0
    --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800d546aa, rsp = 0x7fffffffea18, rbp = 0x7fffffffea40 ---
    
    Tracing command php-fpm pid 46353 tid 100195 td 0xfffff8013855d000
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b6c850
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b6c890
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b6c910
    sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045b6c940
    _sleep() at _sleep+0x27d/frame 0xfffffe0045b6c9c0
    kern_accept4() at kern_accept4+0x1b1/frame 0xfffffe0045b6ca70
    accept1() at accept1+0x63/frame 0xfffffe0045b6cad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b6cbf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b6cbf0
    --- syscall (30, FreeBSD ELF64, sys_accept), rip = 0x801d8101a, rsp = 0x7ffffffec6e8, rbp = 0x7ffffffec720 ---
    
    Tracing command php-fpm pid 44672 tid 100257 td 0xfffff80076b97960
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045a4ab60
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045a4aba0
    critical_exit() at critical_exit+0x7a/frame 0xfffffe0045a4abc0
    lapic_handle_timer() at lapic_handle_timer+0xae/frame 0xfffffe0045a4abf0
    Xtimerint() at Xtimerint+0x8c/frame 0xfffffe0045a4abf0
    --- interrupt, rip = 0x8009cfd23, rsp = 0x7fffffff3580, rbp = 0x7fffffff3580 ---
    
    Tracing command sleep pid 42239 tid 100311 td 0xfffff800069904b0
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045c2c8c0
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045c2c900
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045c2c980
    sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045c2c9b0
    _sleep() at _sleep+0x238/frame 0xfffffe0045c2ca30
    kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe0045c2ca90
    sys_nanosleep() at sys_nanosleep+0x51/frame 0xfffffe0045c2cad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045c2cbf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045c2cbf0
    --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8008f1f7a, rsp = 0x7fffffffec98, rbp = 0x7fffffffecd0 ---
    
    Tracing command sleep pid 40434 tid 100174 td 0xfffff8012ff46960
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b038c0
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b03900
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b03980
    sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045b039b0
    _sleep() at _sleep+0x238/frame 0xfffffe0045b03a30
    kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe0045b03a90
    sys_nanosleep() at sys_nanosleep+0x51/frame 0xfffffe0045b03ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b03bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b03bf0
    --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8008f1f7a, rsp = 0x7fffffffec88, rbp = 0x7fffffffecc0 ---
    
    Tracing command lighttpd_ls pid 59955 tid 100313 td 0xfffff800768e5960
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045c36670
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045c366b0
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045c36730
    sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045c36760
    _sleep() at _sleep+0x238/frame 0xfffffe0045c367e0
    kern_kevent_fp() at kern_kevent_fp+0x399/frame 0xfffffe0045c369c0
    kern_kevent() at kern_kevent+0x9f/frame 0xfffffe0045c36a20
    sys_kevent() at sys_kevent+0x12a/frame 0xfffffe0045c36ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045c36bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045c36bf0
    --- syscall (363, FreeBSD ELF64, sys_kevent), rip = 0x80121193a, rsp = 0x7fffffffe988, rbp = 0x7fffffffe9d0 ---
    
    Tracing command sh pid 51491 tid 100309 td 0xfffff80121c544b0
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045c226a0
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045c226e0
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045c22760
    sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045c22790
    _sleep() at _sleep+0x27d/frame 0xfffffe0045c22810
    kern_wait6() at kern_wait6+0x5de/frame 0xfffffe0045c228e0
    sys_wait4() at sys_wait4+0x72/frame 0xfffffe0045c22ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045c22bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045c22bf0
    --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800d8bdba, rsp = 0x7fffffffe888, rbp = 0x7fffffffe930 ---
    
    Tracing command pinger pid 43301 tid 100304 td 0xfffff8004ecbd000
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045c09700
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045c09740
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045c097c0
    sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045c097f0
    _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19e/frame 0xfffffe0045c09850
    seltdwait() at seltdwait+0xa4/frame 0xfffffe0045c09890
    kern_select() at kern_select+0x8fa/frame 0xfffffe0045c09a90
    sys_select() at sys_select+0x54/frame 0xfffffe0045c09ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045c09bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045c09bf0
    --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8019c86aa, rsp = 0x7fffffffdbf8, rbp = 0x7fffffffdc30 ---
    
    Tracing command pinger pid 73253 tid 100193 td 0xfffff8013855d960
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b62700
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b62740
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b627c0
    sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045b627f0
    _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19e/frame 0xfffffe0045b62850
    seltdwait() at seltdwait+0xa4/frame 0xfffffe0045b62890
    kern_select() at kern_select+0x8fa/frame 0xfffffe0045b62a90
    sys_select() at sys_select+0x54/frame 0xfffffe0045b62ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b62bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b62bf0
    --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8019c86aa, rsp = 0x7fffffffdbf8, rbp = 0x7fffffffdc30 ---
    
    Tracing command sh pid 41872 tid 100091 td 0xfffff8000681c000
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe00459d46a0
    mi_switch() at mi_switch+0xe1/frame 0xfffffe00459d46e0
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe00459d4760
    sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00459d4790
    _sleep() at _sleep+0x27d/frame 0xfffffe00459d4810
    kern_wait6() at kern_wait6+0x5de/frame 0xfffffe00459d48e0
    sys_wait4() at sys_wait4+0x72/frame 0xfffffe00459d4ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe00459d4bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe00459d4bf0
    --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800d8bdba, rsp = 0x7fffffffe888, rbp = 0x7fffffffe930 ---
    
    Tracing command openvpn pid 40758 tid 100139 td 0xfffff80121c57960
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045a09780
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045a097c0
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045a09840
    sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045a09870
    _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19e/frame 0xfffffe0045a098d0
    seltdwait() at seltdwait+0xa4/frame 0xfffffe0045a09910
    kern_poll() at kern_poll+0x464/frame 0xfffffe0045a09ab0
    sys_poll() at sys_poll+0x61/frame 0xfffffe0045a09ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045a09bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045a09bf0
    --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x80120df3a, rsp = 0x7fffffffd128, rbp = 0x7fffffffd180 ---
    
    Tracing command unbound pid 54114 tid 100346 td 0xfffff80076ed3960
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045a72670
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045a726b0
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045a72730
    sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045a72760
    _sleep() at _sleep+0x27d/frame 0xfffffe0045a727e0
    kern_kevent_fp() at kern_kevent_fp+0x399/frame 0xfffffe0045a729c0
    kern_kevent() at kern_kevent+0x9f/frame 0xfffffe0045a72a20
    sys_kevent() at sys_kevent+0x12a/frame 0xfffffe0045a72ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045a72bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045a72bf0
    --- syscall (363, FreeBSD ELF64, sys_kevent), rip = 0x80169a93a, rsp = 0x7fffdf7f9e18, rbp = 0x7fffdf7f9e60 ---
    
    Tracing command unbound pid 54114 tid 100118 td 0xfffff8004ec964b0
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045a5e670
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045a5e6b0
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045a5e730
    sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045a5e760
    _sleep() at _sleep+0x27d/frame 0xfffffe0045a5e7e0
    kern_kevent_fp() at kern_kevent_fp+0x399/frame 0xfffffe0045a5e9c0
    kern_kevent() at kern_kevent+0x9f/frame 0xfffffe0045a5ea20
    sys_kevent() at sys_kevent+0x12a/frame 0xfffffe0045a5ead0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045a5ebf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045a5ebf0
    --- syscall (363, FreeBSD ELF64, sys_kevent), rip = 0x80169a93a, rsp = 0x7fffffffea08, rbp = 0x7fffffffea50 ---
    
    Tracing command pinger pid 29593 tid 100273 td 0xfffff80121c57000
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045a45700
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045a45740
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045a457c0
    sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045a457f0
    _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19e/frame 0xfffffe0045a45850
    seltdwait() at seltdwait+0xa4/frame 0xfffffe0045a45890
    kern_select() at kern_select+0x8fa/frame 0xfffffe0045a45a90
    sys_select() at sys_select+0x54/frame 0xfffffe0045a45ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045a45bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045a45bf0
    --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8019c86aa, rsp = 0x7fffffffdbf8, rbp = 0x7fffffffdc30 ---
    
    Tracing command dhcpleases pid 98531 tid 100202 td 0xfffff8013865c960
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b8f390
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b8f3d0
    sleepq_wait() at sleepq_wait+0x3a/frame 0xfffffe0045b8f400
    _sleep() at _sleep+0x287/frame 0xfffffe0045b8f480
    bwait() at bwait+0x8e/frame 0xfffffe0045b8f4d0
    bufwrite() at bufwrite+0x1d8/frame 0xfffffe0045b8f510
    ffs_update() at ffs_update+0x261/frame 0xfffffe0045b8f590
    ffs_write() at ffs_write+0x54b/frame 0xfffffe0045b8f630
    VOP_WRITE_APV() at VOP_WRITE_APV+0x149/frame 0xfffffe0045b8f740
    vn_write() at vn_write+0x248/frame 0xfffffe0045b8f7c0
    vn_io_fault_doio() at vn_io_fault_doio+0x22/frame 0xfffffe0045b8f800
    vn_io_fault1() at vn_io_fault1+0x1ac/frame 0xfffffe0045b8f960
    vn_io_fault() at vn_io_fault+0x18b/frame 0xfffffe0045b8f9e0
    dofilewrite() at dofilewrite+0x87/frame 0xfffffe0045b8fa30
    kern_writev() at kern_writev+0x68/frame 0xfffffe0045b8fa80
    sys_write() at sys_write+0x63/frame 0xfffffe0045b8fad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b8fbf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b8fbf0
    --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x80095768a, rsp = 0x7fffffffe348, rbp = 0x7fffffffe370 ---
    
    Tracing command dpinger pid 97150 tid 100281 td 0xfffff80006951960
    kdb_enter() at kdb_enter+0x3e/frame 0xfffffe0045b71780
    vpanic() at vpanic+0x146/frame 0xfffffe0045b717c0
    panic() at panic+0x43/frame 0xfffffe0045b71820
    sbflush_internal() at sbflush_internal+0x7b/frame 0xfffffe0045b71840
    sbdestroy() at sbdestroy+0x12/frame 0xfffffe0045b71860
    sofree() at sofree+0x18e/frame 0xfffffe0045b71890
    soclose() at soclose+0x34f/frame 0xfffffe0045b718d0
    _fdrop() at _fdrop+0x29/frame 0xfffffe0045b718f0
    closef() at closef+0x21e/frame 0xfffffe0045b71980
    fdescfree() at fdescfree+0x4f9/frame 0xfffffe0045b71a30
    exit1() at exit1+0x576/frame 0xfffffe0045b71ac0
    sys_sys_exit() at sys_sys_exit+0xe/frame 0xfffffe0045b71ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b71bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b71bf0
    --- syscall (1, FreeBSD ELF64, sys_sys_exit), rip = 0x800b1c18a, rsp = 0x7fffdf9fa5a8, rbp = 0x7fffdf9fa5c0 ---
    
    Tracing command dhclient pid 2597 tid 100209 td 0xfffff80121e774b0
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045bb2780
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045bb27c0
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045bb2840
    sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045bb2870
    _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19e/frame 0xfffffe0045bb28d0
    seltdwait() at seltdwait+0xa4/frame 0xfffffe0045bb2910
    kern_poll() at kern_poll+0x464/frame 0xfffffe0045bb2ab0
    sys_poll() at sys_poll+0x61/frame 0xfffffe0045bb2ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045bb2bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045bb2bf0
    --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x800b18f3a, rsp = 0x7fffffffec08, rbp = 0x7fffffffec40 ---
    
    Tracing command dhclient pid 95568 tid 100184 td 0xfffff8012ff344b0
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b35790
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b357d0
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b35850
    sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045b35880
    _cv_wait_sig() at _cv_wait_sig+0x17a/frame 0xfffffe0045b358d0
    seltdwait() at seltdwait+0xae/frame 0xfffffe0045b35910
    kern_poll() at kern_poll+0x464/frame 0xfffffe0045b35ab0
    sys_poll() at sys_poll+0x61/frame 0xfffffe0045b35ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b35bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b35bf0
    --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x800b18f3a, rsp = 0x7fffffffec18, rbp = 0x7fffffffec40 ---
    
    Tracing command sh pid 53523 tid 100094 td 0xfffff800068a0000
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe00459e3790
    mi_switch() at mi_switch+0xe1/frame 0xfffffe00459e37d0
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe00459e3850
    sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00459e3880
    _cv_wait_sig() at _cv_wait_sig+0x17a/frame 0xfffffe00459e38d0
    tty_wait() at tty_wait+0x1c/frame 0xfffffe00459e38f0
    ttydisc_read() at ttydisc_read+0xf4/frame 0xfffffe00459e3950
    ttydev_read() at ttydev_read+0x86/frame 0xfffffe00459e3980
    devfs_read_f() at devfs_read_f+0xd8/frame 0xfffffe00459e39e0
    dofileread() at dofileread+0x95/frame 0xfffffe00459e3a30
    kern_readv() at kern_readv+0x68/frame 0xfffffe00459e3a80
    sys_read() at sys_read+0x63/frame 0xfffffe00459e3ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe00459e3bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe00459e3bf0
    --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800dee6ca, rsp = 0x7fffffffe768, rbp = 0x7fffffffe860 ---
    
    Tracing command sh pid 53254 tid 100123 td 0xfffff8000681c960
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe00459ca6a0
    mi_switch() at mi_switch+0xe1/frame 0xfffffe00459ca6e0
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe00459ca760
    sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00459ca790
    _sleep() at _sleep+0x27d/frame 0xfffffe00459ca810
    kern_wait6() at kern_wait6+0x5de/frame 0xfffffe00459ca8e0
    sys_wait4() at sys_wait4+0x72/frame 0xfffffe00459caad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe00459cabf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe00459cabf0
    --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800d8bdba, rsp = 0x7fffffffe818, rbp = 0x7fffffffe8c0 ---
    
    Tracing command sshlockout_pf pid 53146 tid 100192 td 0xfffff8012ff37960
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b5d8c0
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b5d900
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b5d980
    sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045b5d9b0
    _sleep() at _sleep+0x238/frame 0xfffffe0045b5da30
    kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe0045b5da90
    sys_nanosleep() at sys_nanosleep+0x51/frame 0xfffffe0045b5dad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b5dbf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b5dbf0
    --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x800b18f7a, rsp = 0x7fffdfffdf48, rbp = 0x7fffdfffdf70 ---
    
    Tracing command sshlockout_pf pid 53146 tid 100087 td 0xfffff8000681d4b0
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe00459c0810
    mi_switch() at mi_switch+0xe1/frame 0xfffffe00459c0850
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe00459c08d0
    sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe00459c0900
    _sleep() at _sleep+0x27d/frame 0xfffffe00459c0980
    pipe_read() at pipe_read+0x2aa/frame 0xfffffe00459c09e0
    dofileread() at dofileread+0x95/frame 0xfffffe00459c0a30
    kern_readv() at kern_readv+0x68/frame 0xfffffe00459c0a80
    sys_read() at sys_read+0x63/frame 0xfffffe00459c0ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe00459c0bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe00459c0bf0
    --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800b7b6ca, rsp = 0x7fffffffe7b8, rbp = 0x7fffffffe7d0 ---
    
    Tracing command login pid 52866 tid 100180 td 0xfffff8012ff34960
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b216a0
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b216e0
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b21760
    sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045b21790
    _sleep() at _sleep+0x27d/frame 0xfffffe0045b21810
    kern_wait6() at kern_wait6+0x5de/frame 0xfffffe0045b218e0
    sys_wait4() at sys_wait4+0x72/frame 0xfffffe0045b21ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b21bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b21bf0
    --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800d15dba, rsp = 0x7fffffffebc8, rbp = 0x7fffffffed40 ---
    
    Tracing command getty pid 52641 tid 100076 td 0xfffff8000674f000
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe00459898c0
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045989900
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045989980
    sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe00459899b0
    _sleep() at _sleep+0x238/frame 0xfffffe0045989a30
    kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe0045989a90
    sys_nanosleep() at sys_nanosleep+0x51/frame 0xfffffe0045989ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045989bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045989bf0
    --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x800b09f7a, rsp = 0x7fffffffec38, rbp = 0x7fffffffec90 ---
    
    Tracing command minicron pid 37029 tid 100098 td 0xfffff80006903960
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe00459f78c0
    mi_switch() at mi_switch+0xe1/frame 0xfffffe00459f7900
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe00459f7980
    sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe00459f79b0
    _sleep() at _sleep+0x238/frame 0xfffffe00459f7a30
    kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe00459f7a90
    sys_nanosleep() at sys_nanosleep+0x51/frame 0xfffffe00459f7ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe00459f7bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe00459f7bf0
    --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8008f2f7a, rsp = 0x7fffffffeb88, rbp = 0x7fffffffebe0 ---
    
    Tracing command minicron pid 36737 tid 100181 td 0xfffff8013801a960
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b266a0
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b266e0
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b26760
    sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045b26790
    _sleep() at _sleep+0x27d/frame 0xfffffe0045b26810
    kern_wait6() at kern_wait6+0x5de/frame 0xfffffe0045b268e0
    sys_wait4() at sys_wait4+0x72/frame 0xfffffe0045b26ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b26bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b26bf0
    --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8008f2dba, rsp = 0x7fffffffebe8, rbp = 0x7fffffffec30 ---
    
    Tracing command minicron pid 36681 tid 100177 td 0xfffff8012fe6c960
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b128c0
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b12900
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b12980
    sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045b129b0
    _sleep() at _sleep+0x238/frame 0xfffffe0045b12a30
    kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe0045b12a90
    sys_nanosleep() at sys_nanosleep+0x51/frame 0xfffffe0045b12ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b12bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b12bf0
    --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8008f2f7a, rsp = 0x7fffffffeb98, rbp = 0x7fffffffebf0 ---
    
    Tracing command minicron pid 36302 tid 100190 td 0xfffff801380284b0
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b536a0
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b536e0
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b53760
    sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045b53790
    _sleep() at _sleep+0x27d/frame 0xfffffe0045b53810
    kern_wait6() at kern_wait6+0x5de/frame 0xfffffe0045b538e0
    sys_wait4() at sys_wait4+0x72/frame 0xfffffe0045b53ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b53bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b53bf0
    --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8008f2dba, rsp = 0x7fffffffebf8, rbp = 0x7fffffffec40 ---
    
    Tracing command minicron pid 36077 tid 100189 td 0xfffff80138028960
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b4e8c0
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b4e900
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b4e980
    sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045b4e9b0
    _sleep() at _sleep+0x238/frame 0xfffffe0045b4ea30
    kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe0045b4ea90
    sys_nanosleep() at sys_nanosleep+0x51/frame 0xfffffe0045b4ead0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b4ebf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b4ebf0
    --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x8008f2f7a, rsp = 0x7fffffffebb8, rbp = 0x7fffffffec10 ---
    
    Tracing command minicron pid 35618 tid 100188 td 0xfffff80138029000
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b496a0
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b496e0
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b49760
    sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045b49790
    _sleep() at _sleep+0x27d/frame 0xfffffe0045b49810
    kern_wait6() at kern_wait6+0x5de/frame 0xfffffe0045b498e0
    sys_wait4() at sys_wait4+0x72/frame 0xfffffe0045b49ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b49bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b49bf0
    --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8008f2dba, rsp = 0x7fffffffec18, rbp = 0x7fffffffec60 ---
    
    Tracing command pinger pid 30416 tid 100187 td 0xfffff801380294b0
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b44700
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b44740
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b447c0
    sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045b447f0
    _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19e/frame 0xfffffe0045b44850
    seltdwait() at seltdwait+0xa4/frame 0xfffffe0045b44890
    kern_select() at kern_select+0x8fa/frame 0xfffffe0045b44a90
    sys_select() at sys_select+0x54/frame 0xfffffe0045b44ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b44bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b44bf0
    --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8019c86aa, rsp = 0x7fffffffdbf8, rbp = 0x7fffffffdc30 ---
    
    Tracing command squid pid 25520 tid 100191 td 0xfffff80138028000
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045b58670
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045b586b0
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045b58730
    sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045b58760
    _sleep() at _sleep+0x238/frame 0xfffffe0045b587e0
    kern_kevent_fp() at kern_kevent_fp+0x399/frame 0xfffffe0045b589c0
    kern_kevent() at kern_kevent+0x9f/frame 0xfffffe0045b58a20
    sys_kevent() at sys_kevent+0x12a/frame 0xfffffe0045b58ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045b58bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045b58bf0
    --- syscall (363, FreeBSD ELF64, sys_kevent), rip = 0x802d5b93a, rsp = 0x7fffffffe898, rbp = 0x7fffffffe8e0 ---
    
    Tracing command squid pid 25336 tid 100172 td 0xfffff800069034b0
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045af96a0
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045af96e0
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045af9760
    sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045af9790
    _sleep() at _sleep+0x27d/frame 0xfffffe0045af9810
    kern_wait6() at kern_wait6+0x5de/frame 0xfffffe0045af98e0
    sys_wait4() at sys_wait4+0x72/frame 0xfffffe0045af9ad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045af9bf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045af9bf0
    --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x802d50dba, rsp = 0x7fffffffe998, rbp = 0x7fffffffe9d0 ---
    
    Tracing command syslogd pid 94455 tid 100173 td 0xfffff8012fe6d000
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045afe710
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045afe750
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045afe7d0
    sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045afe800
    _cv_wait_sig() at _cv_wait_sig+0x17a/frame 0xfffffe0045afe850
    seltdwait() at seltdwait+0xae/frame 0xfffffe0045afe890
    kern_select() at kern_select+0x8fa/frame 0xfffffe0045afea90
    sys_select() at sys_select+0x54/frame 0xfffffe0045afead0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045afebf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045afebf0
    --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800b716aa, rsp = 0x7fffffffdbe8, rbp = 0x7fffffffec00 ---
    
    Tracing command powerd pid 75350 tid 100154 td 0xfffff80121dfa4b0
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045a9f700
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045a9f740
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045a9f7c0
    sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0045a9f7f0
    _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x19e/frame 0xfffffe0045a9f850
    seltdwait() at seltdwait+0xa4/frame 0xfffffe0045a9f890
    kern_select() at kern_select+0x8fa/frame 0xfffffe0045a9fa90
    sys_select() at sys_select+0x54/frame 0xfffffe0045a9fad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045a9fbf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045a9fbf0
    --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800b6a6aa, rsp = 0x7fffffffeac8, rbp = 0x7fffffffec50 ---
    
    Tracing command charon pid 59905 tid 100170 td 0xfffff80006951000
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045aef840
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045aef880
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045aef900
    sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045aef930
    _sleep() at _sleep+0x27d/frame 0xfffffe0045aef9b0
    umtxq_sleep() at umtxq_sleep+0x125/frame 0xfffffe0045aefa10
    do_wait() at do_wait+0x387/frame 0xfffffe0045aefa90
    __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x83/frame 0xfffffe0045aefad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045aefbf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045aefbf0
    --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x800f5013c, rsp = 0x7fffde1eee58, rbp = 0x7fffde1eee80 ---
    
    Tracing command charon pid 59905 tid 100169 td 0xfffff8004ec974b0
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045aea840
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045aea880
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045aea900
    sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045aea930
    _sleep() at _sleep+0x27d/frame 0xfffffe0045aea9b0
    umtxq_sleep() at umtxq_sleep+0x125/frame 0xfffffe0045aeaa10
    do_wait() at do_wait+0x387/frame 0xfffffe0045aeaa90
    __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x83/frame 0xfffffe0045aeaad0
    amd64_syscall() at amd64_syscall+0x40f/frame 0xfffffe0045aeabf0
    Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0045aeabf0
    --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x800f5013c, rsp = 0x7fffde3efe58, rbp = 0x7fffde3efe80 ---
    
    Tracing command charon pid 59905 tid 100168 td 0xfffff8004ec984b0
    sched_switch() at sched_switch+0x2b3/frame 0xfffffe0045ae5840
    mi_switch() at mi_switch+0xe1/frame 0xfffffe0045ae5880
    sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe0045ae5900
    sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0045ae5930
    _sleep() at _sleep+0x27d/frame 0xfffffe0045ae59b0
    umtxq_sleep() at umtxq_sleep+0x125/frame 0xfffffe0045ae5a10
    do_wait() at do_wait+0x387/frame 0xfffffe0045ae5a90
    __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x83/frame 0xfffffe0045ae5ad0
    a[/thread]
    
    1 Reply Last reply Reply Quote 0
    • J
      jimp Rebel Alliance Developer Netgate
      last edited by Jul 10, 2017, 5:51 PM

      Looks like this:

      https://redmine.pfsense.org/issues/4689

      Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

      Need help fast? Netgate Global Support!

      Do not Chat/PM for help!

      1 Reply Last reply Reply Quote 0
      2 out of 2
      • First post
        2/2
        Last post
      Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
        This community forum collects and processes your personal information.
        consent.not_received