Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    PF Sense Crash with 100MB/s WAN throughput

    Scheduled Pinned Locked Moved General pfSense Questions
    3 Posts 2 Posters 1.0k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Z Offline
      zeroluck
      last edited by

      I started Downloading GTA V at ~14MB/sec and PF Sense crashed. I don't think the system rebooted. Here's the core dump:

      Crash report begins.  Anonymous machine information:
      
      amd64
      10.1-RELEASE-p9
      FreeBSD 10.1-RELEASE-p9 #0 57b23e7(releng/10.1)-dirty: Mon Apr 13 20:30:25 CDT 2015     root@pfs22-amd64-builder:/usr/obj.amd64/usr/pfSensesrc/src/sys/pfSense_SMP.10
      
      Crash report details:
      
      Filename: /var/crash/bounds
      1
      
      Filename: /var/crash/info.0
      Dump header from device /dev/ada0s1b
        Architecture: amd64
        Architecture Version: 1
        Dump Length: 105984B (0 MB)
        Blocksize: 512
        Dumptime: Sat May  9 09:20:10 2015
        Hostname: fuckmountain.no-ip.biz
        Magic: FreeBSD Text Dump
        Version String: FreeBSD 10.1-RELEASE-p9 #0 57b23e7(releng/10.1)-dirty: Mon Apr 13 20:30:25 CDT 2015
          root@pfs22-amd64-builder:/usr/obj.amd64/usr/pfSensesrc/src/sys/pfSense_SMP.10
        Panic String: 
        Dump Parity: 3387775489
        Bounds: 0
        Dump Status: good
      
      Filename: /var/crash/info.last
      Dump header from device /dev/ada0s1b
        Architecture: amd64
        Architecture Version: 1
        Dump Length: 105984B (0 MB)
        Blocksize: 512
        Dumptime: Sat May  9 09:20:10 2015
        Hostname: fuckmountain.no-ip.biz
        Magic: FreeBSD Text Dump
        Version String: FreeBSD 10.1-RELEASE-p9 #0 57b23e7(releng/10.1)-dirty: Mon Apr 13 20:30:25 CDT 2015
          root@pfs22-amd64-builder:/usr/obj.amd64/usr/pfSensesrc/src/sys/pfSense_SMP.10
        Panic String: 
        Dump Parity: 3387775489
        Bounds: 0
        Dump Status: good
      
      Filename: /var/crash/minfree
      2048
      
      Filename: /var/crash/textdump.tar.0
      ddb.txt06000014000012523413632  7073 ustarrootwheeldb:0:kdb.enter.default>  run lockinfo
      db:1:lockinfo> show locks
      No such command
      db:1:locks>  show alllocks
      No such command
      db:1:alllocks>  show lockedvnods
      Locked vnodes
      db:0:kdb.enter.default>  show pcpu
      cpuid        = 1
      dynamic pcpu = 0xfffffe00998ac280
      curthread    = 0xfffff80003500000: pid 12 "irq27: bge0 bge1"
      curpcb       = 0xfffffe004d117cc0
      fpcurthread  = none
      idlethread   = 0xfffff800032d0920: tid 100004 "idle: cpu1"
      curpmap      = 0xffffffff8217c548
      tssp         = 0xffffffff821974f8
      commontssp   = 0xffffffff821974f8
      rsp0         = 0xfffffe004d117cc0
      gs32p        = 0xffffffff82198f50
      ldt          = 0xffffffff82198f90
      tss          = 0xffffffff82198f80
      db:0:kdb.enter.default>  bt
      Tracing pid 12 tid 100045 td 0xfffff80003500000
      bge_rxeof() at bge_rxeof+0x41e/frame 0xfffffe004d117ae0
      bge_intr() at bge_intr+0x179/frame 0xfffffe004d117b20
      intr_event_execute_handlers() at intr_event_execute_handlers+0xab/frame 0xfffffe004d117b60
      ithread_loop() at ithread_loop+0x96/frame 0xfffffe004d117bb0
      fork_exit() at fork_exit+0x9a/frame 0xfffffe004d117bf0
      fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe004d117bf0
      --- trap 0, rip = 0, rsp = 0xfffffe004d117cb0, rbp = 0 ---
      db:0:kdb.enter.default>  ps
        pid  ppid  pgrp   uid   state   wmesg         wchan        cmd
      52025   250   250     0  S       nanslp   0xffffffff81f683f2 php-fpm
      40803 11207   250     0  S       nanslp   0xffffffff81f683f3 sleep
      47830     1 47830     0  Ss      (threaded)                  snort
      100410                   S       nanslp   0xffffffff81f683f0 snort
      100208                   S       bpf      0xfffff80133cbd400 snort
      32800     1 32800     0  Ss      (threaded)                  ntpd
      100102                   S       select   0xfffff800065c4b40 ntpd
      11207     1   250     0  S       wait     0xfffff8006b5984c0 sh
      30644     1 30338 65534  S       select   0xfffff8006802e2c0 dnsmasq
      26223     1 26223    65  Ss      select   0xfffff80133c419c0 dhclient
      21763     1 21763     0  Ss      select   0xfffff80006511c40 dhclient
      36135     1 36135  1002  Ss      select   0xfffff80133e04140 dhcpd
      34166     1 34166    65  Ss      select   0xfffff800d6cf0ac0 dhclient
      24607     1 24607     0  Ss      select   0xfffff80133a0ac40 dhclient
       2950  2912  2912     0  S       piperd   0xfffff8000653b5d0 rrdtool
       2912     1  2912     0  Ss      select   0xfffff8006b1386c0 apinger
      18688 18676 18688     0  S+      ttyin    0xfffff800050c70a8 sh
      18676 18048 18676     0  S+      wait     0xfffff800064d6980 sh
      18362 77823 18362     0  Ss      (threaded)                  sshlockout_pf
      100174                   S       nanslp   0xffffffff81f683f2 sshlockout_pf
      100075                   S       piperd   0xfffff800065e32e8 sshlockout_pf
      18048     1 18048     0  Ss+     wait     0xfffff800066184c0 login
      11941 11551 11551     0  S       nanslp   0xffffffff81f683f2 minicron
      11551     1 11551     0  Ss      wait     0xfffff80006443000 minicron
      11319 10984 10984     0  S       nanslp   0xffffffff81f683f1 minicron
      10984     1 10984     0  Ss      wait     0xfffff800d6fd0980 minicron
      10812 10357 10357     0  S       nanslp   0xffffffff81f683f3 minicron
      10357     1 10357     0  Ss      wait     0xfffff800684994c0 minicron
      95938 94723   250     0  S       bpf      0xfffff80006f5c800 bandwidthd
      95053 94723   250     0  S       bpf      0xfffff80006f5c200 bandwidthd
      95003 94723   250     0  S       bpf      0xfffff80068066e00 bandwidthd
      94723     1   250     0  S       bpf      0xfffff80006f5c000 bandwidthd
      94044 92206   250     0  S       bpf      0xfffff80006f5c400 bandwidthd
      93233 92206   250     0  S       bpf      0xfffff80006f5c600 bandwidthd
      92617 92206   250     0  S       bpf      0xfffff80133435400 bandwidthd
      92206     1   250     0  S       bpf      0xfffff80006f8ec00 bandwidthd
      78373     1 78373     0  Ss      nanslp   0xffffffff81f683f2 cron
      77823     1 77823     0  Ss      select   0xfffff800d6cf09c0 syslogd
      44882     1 44882     0  Ss      select   0xfffff80006431140 miniupnpd
      34325 34262 34325     0  Ss      (threaded)                  charon
      100128                   S       uwait    0xfffff80006f63700 charon
      100127                   S       uwait    0xfffff80006f63980 charon
      100126                   S       uwait    0xfffff80006434b00 charon
      100125                   S       uwait    0xfffff80006434d00 charon
      100124                   S       uwait    0xfffff80006434e00 charon
      100123                   S       uwait    0xfffff80006434a80 charon
      100122                   S       uwait    0xfffff80006f62100 charon
      100121                   S       uwait    0xfffff80006434980 charon
      100120                   S       uwait    0xfffff800065c4180 charon
      100119                   S       uwait    0xfffff800065c4080 charon
      100118                   S       uwait    0xfffff800065c3f00 charon
      100117                   S       uwait    0xfffff800065c3e00 charon
      100116                   S       select   0xfffff8006802ecc0 charon
      100115                   S       select   0xfffff80006f7a5c0 charon
      100114                   S       accept   0xfffff80068121066 charon
      100113                   S       uwait    0xfffff800065c3a00 charon
      100101                   S       sigwait  0xfffff80006f43000 charon
      34262     1 34262     0  Ss      select   0xfffff80006f63940 starter
      23228     1 23004     0  S       kqread   0xfffff800680b4a00 lighttpd
      16483     1 16483     0  Ss      select   0xfffff80006479a40 inetd
      15294     1 15294     0  Ss      bpf      0xfffff80006440a00 filterlog
        277     1   277     0  Ss      select   0xfffff80006513440 devd
        267   265   265     0  S       kqread   0xfffff800065db700 check_reload_status
        265     1   265     0  Ss      kqread   0xfffff80006658800 check_reload_status
        250     1   250     0  Ss      kqread   0xfffff800032f4000 php-fpm
         63     0     0     0  DL      mdwait   0xfffff800065fe800 [md0]
         20     0     0     0  DL      syncer   0xffffffff81fb9388 [syncer]
         19     0     0     0  DL      vlruwt   0xfffff80006443980 [vnlru]
         18     0     0     0  DL      psleep   0xffffffff81fb8584 [bufdaemon]
         17     0     0     0  DL      pgzero   0xffffffff8210b30c [pagezero]
          9     0     0     0  DL      pollid   0xffffffff81f66d70 [idlepoll]
          8     0     0     0  DL      psleep   0xffffffff8210aa40 [vmdaemon]
          7     0     0     0  DL      psleep   0xffffffff82196804 [pagedaemon]
          6     0     0     0  DL      waiting_ 0xffffffff82187270 [sctp_iterator]
          5     0     0     0  DL      pftm     0xffffffff80d065a0 [pf purge]
         16     0     0     0  DL      (threaded)                  [usb]
      100042                   D       -        0xfffffe00009df460 [usbus1]
      100041                   D       -        0xfffffe00009df408 [usbus1]
      100040                   D       -        0xfffffe00009df3b0 [usbus1]
      100039                   D       -        0xfffffe00009df358 [usbus1]
      100038                   D       -        0xfffffe00009c3460 [usbus0]
      100037                   D       -        0xfffffe00009c3408 [usbus0]
      100036                   D       -        0xfffffe00009c33b0 [usbus0]
      100035                   D       -        0xfffffe00009c3358 [usbus0]
          4     0     0     0  DL      (threaded)                  [cam]
      100056                   D       -        0xffffffff81ea0f40 [scanner]
      100024                   D       -        0xffffffff81ea1100 [doneq0]
          3     0     0     0  DL      crypto_r 0xffffffff82108f10 [crypto returns]
          2     0     0     0  DL      crypto_w 0xffffffff82108db8 [crypto]
         15     0     0     0  DL      -        0xffffffff81ebe600 [rand_harvestq]
         14     0     0     0  DL      (threaded)                  [geom]
      100019                   D       -        0xffffffff8217b9e0 [g_down]
      100018                   D       -        0xffffffff8217b9d8 [g_up]
      100017                   D       -        0xffffffff8217b9d0 [g_event]
         13     0     0     0  DL      (threaded)                  [ng_queue]
      100016                   D       sleep    0xffffffff81e5f3e8 [ng_queue3]
      100015                   D       sleep    0xffffffff81e5f3e8 [ng_queue2]
      100014                   D       sleep    0xffffffff81e5f3e8 [ng_queue1]
      100013                   D       sleep    0xffffffff81e5f3e8 [ng_queue0]
         12     0     0     0  RL      (threaded)                  [intr]
      100066                   I                                   [swi1: netisr 3]
      100065                   I                                   [swi1: netisr 2]
      100064                   I                                   [swi1: netisr 1]
      100054                   I                                   [swi1: pfsync]
      100052                   I                                   [swi1: pf send]
      100049                   I                                   [irq7: ppc0]
      100048                   I                                   [swi0: uart uart]
      100047                   I                                   [irq1: atkbd0]
      100045                   Run     CPU 1                       [irq27: bge0 bge1]
      100044                   I                                   [irq15: ata1]
      100043                   I                                   [irq14: ata0]
      100034                   I                                   [irq19: ohci0 ohci1]
      100029                   I                                   [swi5: fast taskq]
      100027                   I                                   [swi6: Giant taskq]
      100025                   I                                   [swi6: task queue]
      100012                   I                                   [swi3: vm]
      100011                   I                                   [swi4: clock]
      100010                   I                                   [swi4: clock]
      100009                   I                                   [swi4: clock]
      100008                   I                                   [swi4: clock]
      100007                   I                                   [swi1: netisr 0]
         11     0     0     0  RL      (threaded)                  [idle]
      100006                   Run     CPU 3                       [idle: cpu3]
      100005                   Run     CPU 2                       [idle: cpu2]
      100004                   CanRun                              [idle: cpu1]
      100003                   Run     CPU 0                       [idle: cpu0]
          1     0     1     0  SLs     wait     0xfffff800032ce4c0 [init]
         10     0     0     0  DL      audit_wo 0xffffffff8218ddf0 [audit]
          0     0     0     0  DLs     (threaded)                  [kernel]
      100104                   D       -        0xfffff8000648be00 [dummynet]
      100055                   D       -        0xfffff800032f1d00 [CAM taskq]
      100050                   D       -        0xfffff800050a4900 [mca taskq]
      100046                   D       -        0xfffff800034b5300 [em0 taskq]
      100033                   D       -        0xfffff800032c4900 [acpi_task_2]
      100032                   D       -        0xfffff800032c4900 [acpi_task_1]
      100031                   D       -        0xfffff800032c4900 [acpi_task_0]
      100030                   D       -        0xfffff800032c4b00 [kqueue taskq]
      100028                   D       -        0xfffff800032f1100 [thread]
      100026                   D       -        0xfffff800032f1600 [ffs_trim taskq]
      100020                   D       -        0xfffff800032f3500 [firmware taskq]
      100000                   D       swapin   0xffffffff8217bad8 [swapper]
      40896 95003   250     0  Z                                   bandwidthd
      41208 92617   250     0  Z                                   bandwidthd
      13796 94723   250     0  Z                                   bandwidthd
      13545 92206   250     0  Z                                   bandwidthd
      43322 93233   250     0  Z                                   bandwidthd
      43229 95053   250     0  Z                                   bandwidthd
      89047 95938   250     0  Z                                   bandwidthd
      88749 94044   250     0  Z                                   bandwidthd
      db:0:kdb.enter.default>  alltrace
      
      Tracing command php-fpm pid 52025 tid 100152 td 0xfffff8000688b920
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d3448d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d344910
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d344990
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe004d3449c0
      _sleep() at _sleep+0x238/frame 0xfffffe004d344a40
      kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe004d344aa0
      sys_nanosleep() at sys_nanosleep+0x51/frame 0xfffffe004d344ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d344bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d344bf0
      --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x80191540a, rsp = 0x7fffffffab98, rbp = 0x7fffffffabf0 ---
      
      Tracing command sleep pid 40803 tid 100069 td 0xfffff800065c8920
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d1ab8d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d1ab910
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d1ab990
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe004d1ab9c0
      _sleep() at _sleep+0x238/frame 0xfffffe004d1aba40
      kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe004d1abaa0
      sys_nanosleep() at sys_nanosleep+0x51/frame 0xfffffe004d1abae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d1abbf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d1abbf0
      --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x80095940a, rsp = 0x7fffffffec88, rbp = 0x7fffffffecc0 ---
      
      Tracing command snort pid 47830 tid 100410 td 0xfffff8006b975000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d3498d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d349910
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d349990
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe004d3499c0
      _sleep() at _sleep+0x238/frame 0xfffffe004d349a40
      kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe004d349aa0
      sys_nanosleep() at sys_nanosleep+0x51/frame 0xfffffe004d349ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d349bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d349bf0
      --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x80221d40a, rsp = 0x7fffffbfdec8, rbp = 0x7fffffbfdf20 ---
      
      Tracing command snort pid 47830 tid 100208 td 0xfffff80133e4d920
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d4367d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d436810
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d436890
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe004d4368c0
      _sleep() at _sleep+0x238/frame 0xfffffe004d436940
      bpfread() at bpfread+0x1c7/frame 0xfffffe004d436990
      devfs_read_f() at devfs_read_f+0xeb/frame 0xfffffe004d4369f0
      dofileread() at dofileread+0x95/frame 0xfffffe004d436a40
      kern_readv() at kern_readv+0x68/frame 0xfffffe004d436a90
      sys_read() at sys_read+0x63/frame 0xfffffe004d436ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d436bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d436bf0
      --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x80223e31a, rsp = 0x7fffffffe718, rbp = 0x7fffffffe750 ---
      
      Tracing command ntpd pid 32800 tid 100102 td 0xfffff800066c3490
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d254720
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d254760
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d2547e0
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe004d254810
      _cv_wait_sig() at _cv_wait_sig+0x16a/frame 0xfffffe004d254860
      seltdwait() at seltdwait+0xae/frame 0xfffffe004d2548a0
      kern_select() at kern_select+0x8fa/frame 0xfffffe004d254aa0
      sys_select() at sys_select+0x54/frame 0xfffffe004d254ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d254bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d254bf0
      --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80146629a, rsp = 0x7fffffffdb78, rbp = 0x7fffffffdbb0 ---
      
      Tracing command sh pid 11207 tid 100258 td 0xfffff80006838490
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d44f6d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d44f710
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d44f790
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe004d44f7c0
      _sleep() at _sleep+0x27d/frame 0xfffffe004d44f840
      kern_wait6() at kern_wait6+0x5f4/frame 0xfffffe004d44f8f0
      sys_wait4() at sys_wait4+0x72/frame 0xfffffe004d44fae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d44fbf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d44fbf0
      --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800d66fba, rsp = 0x7fffffffe888, rbp = 0x7fffffffe930 ---
      
      Tracing command dnsmasq pid 30644 tid 100211 td 0xfffff80133e4c920
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d445720
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d445760
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d4457e0
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe004d445810
      _cv_wait_sig() at _cv_wait_sig+0x16a/frame 0xfffffe004d445860
      seltdwait() at seltdwait+0xae/frame 0xfffffe004d4458a0
      kern_select() at kern_select+0x8fa/frame 0xfffffe004d445aa0
      sys_select() at sys_select+0x54/frame 0xfffffe004d445ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d445bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d445bf0
      --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x8014dc29a, rsp = 0x7fffffffe978, rbp = 0x7fffffffec00 ---
      
      Tracing command dhclient pid 26223 tid 100149 td 0xfffff8000644f920
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d3357b0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d3357f0
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d335870
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe004d3358a0
      _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x18b/frame 0xfffffe004d335900
      seltdwait() at seltdwait+0xa4/frame 0xfffffe004d335940
      sys_poll() at sys_poll+0x3a3/frame 0xfffffe004d335ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d335bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d335bf0
      --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x800b38a1a, rsp = 0x7fffffffebd8, rbp = 0x7fffffffec10 ---
      
      Tracing command dhclient pid 21763 tid 100082 td 0xfffff800065c7920
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d1ec7c0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d1ec800
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d1ec880
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe004d1ec8b0
      _cv_wait_sig() at _cv_wait_sig+0x16a/frame 0xfffffe004d1ec900
      seltdwait() at seltdwait+0xae/frame 0xfffffe004d1ec940
      sys_poll() at sys_poll+0x3a3/frame 0xfffffe004d1ecae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d1ecbf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d1ecbf0
      --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x800b38a1a, rsp = 0x7fffffffebe8, rbp = 0x7fffffffec10 ---
      
      Tracing command dhcpd pid 36135 tid 100207 td 0xfffff80133e4e000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d431710
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d431750
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d4317d0
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe004d431800
      _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x18b/frame 0xfffffe004d431860
      seltdwait() at seltdwait+0xa4/frame 0xfffffe004d4318a0
      kern_select() at kern_select+0x8fa/frame 0xfffffe004d431aa0
      sys_select() at sys_select+0x54/frame 0xfffffe004d431ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d431bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d431bf0
      --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800d5329a, rsp = 0x7fffffffea38, rbp = 0x7fffffffea60 ---
      
      Tracing command dhclient pid 34166 tid 100265 td 0xfffff80133e4c490
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d4727b0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d4727f0
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d472870
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe004d4728a0
      _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x18b/frame 0xfffffe004d472900
      seltdwait() at seltdwait+0xa4/frame 0xfffffe004d472940
      sys_poll() at sys_poll+0x3a3/frame 0xfffffe004d472ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d472bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d472bf0
      --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x800b38a1a, rsp = 0x7fffffffebd8, rbp = 0x7fffffffec10 ---
      
      Tracing command dhclient pid 24607 tid 100070 td 0xfffff800065c8490
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d1b07c0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d1b0800
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d1b0880
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe004d1b08b0
      _cv_wait_sig() at _cv_wait_sig+0x16a/frame 0xfffffe004d1b0900
      seltdwait() at seltdwait+0xae/frame 0xfffffe004d1b0940
      sys_poll() at sys_poll+0x3a3/frame 0xfffffe004d1b0ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d1b0bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d1b0bf0
      --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x800b38a1a, rsp = 0x7fffffffebe8, rbp = 0x7fffffffec10 ---
      
      Tracing command rrdtool pid 2950 tid 100166 td 0xfffff800d6dba000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d380820
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d380860
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d3808e0
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe004d380910
      _sleep() at _sleep+0x27d/frame 0xfffffe004d380990
      pipe_read() at pipe_read+0x2aa/frame 0xfffffe004d3809f0
      dofileread() at dofileread+0x95/frame 0xfffffe004d380a40
      kern_readv() at kern_readv+0x68/frame 0xfffffe004d380a90
      sys_read() at sys_read+0x63/frame 0xfffffe004d380ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d380bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d380bf0
      --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8016f631a, rsp = 0x7fffffffeb28, rbp = 0x7fffffffeb40 ---
      
      Tracing command apinger pid 2912 tid 100261 td 0xfffff80068553490
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d45e7b0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d45e7f0
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d45e870
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe004d45e8a0
      _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x18b/frame 0xfffffe004d45e900
      seltdwait() at seltdwait+0xa4/frame 0xfffffe004d45e940
      sys_poll() at sys_poll+0x3a3/frame 0xfffffe004d45eae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d45ebf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d45ebf0
      --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x800921a1a, rsp = 0x7fffffffcb88, rbp = 0x7fffffffec20 ---
      
      Tracing command sh pid 18688 tid 100144 td 0xfffff800684b6920
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d31c7a0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d31c7e0
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d31c860
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe004d31c890
      _cv_wait_sig() at _cv_wait_sig+0x16a/frame 0xfffffe004d31c8e0
      tty_wait() at tty_wait+0x1c/frame 0xfffffe004d31c900
      ttydisc_read() at ttydisc_read+0xf4/frame 0xfffffe004d31c960
      ttydev_read() at ttydev_read+0x86/frame 0xfffffe004d31c990
      devfs_read_f() at devfs_read_f+0xeb/frame 0xfffffe004d31c9f0
      dofileread() at dofileread+0x95/frame 0xfffffe004d31ca40
      kern_readv() at kern_readv+0x68/frame 0xfffffe004d31ca90
      sys_read() at sys_read+0x63/frame 0xfffffe004d31cae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d31cbf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d31cbf0
      --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800e1331a, rsp = 0x7fffffffe768, rbp = 0x7fffffffe860 ---
      
      Tracing command sh pid 18676 tid 100067 td 0xfffff800064e0920
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d1a16d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d1a1710
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d1a1790
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe004d1a17c0
      _sleep() at _sleep+0x27d/frame 0xfffffe004d1a1840
      kern_wait6() at kern_wait6+0x5f4/frame 0xfffffe004d1a18f0
      sys_wait4() at sys_wait4+0x72/frame 0xfffffe004d1a1ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d1a1bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d1a1bf0
      --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800d66fba, rsp = 0x7fffffffe818, rbp = 0x7fffffffe8c0 ---
      
      Tracing command sshlockout_pf pid 18362 tid 100174 td 0xfffff800d6d54490
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d38a8d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d38a910
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d38a990
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe004d38a9c0
      _sleep() at _sleep+0x238/frame 0xfffffe004d38aa40
      kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe004d38aaa0
      sys_nanosleep() at sys_nanosleep+0x51/frame 0xfffffe004d38aae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d38abf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d38abf0
      --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x800b8040a, rsp = 0x7fffffbfdf48, rbp = 0x7fffffbfdf70 ---
      
      Tracing command sshlockout_pf pid 18362 tid 100075 td 0xfffff800064df000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d1c9820
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d1c9860
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d1c98e0
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe004d1c9910
      _sleep() at _sleep+0x27d/frame 0xfffffe004d1c9990
      pipe_read() at pipe_read+0x2aa/frame 0xfffffe004d1c99f0
      dofileread() at dofileread+0x95/frame 0xfffffe004d1c9a40
      kern_readv() at kern_readv+0x68/frame 0xfffffe004d1c9a90
      sys_read() at sys_read+0x63/frame 0xfffffe004d1c9ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d1c9bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d1c9bf0
      --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800ba131a, rsp = 0x7fffffffe7d8, rbp = 0x7fffffffe7f0 ---
      
      Tracing command login pid 18048 tid 100094 td 0xfffff800065e9490
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d22c6d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d22c710
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d22c790
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe004d22c7c0
      _sleep() at _sleep+0x27d/frame 0xfffffe004d22c840
      kern_wait6() at kern_wait6+0x5f4/frame 0xfffffe004d22c8f0
      sys_wait4() at sys_wait4+0x72/frame 0xfffffe004d22cae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d22cbf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d22cbf0
      --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800cf0fba, rsp = 0x7fffffffebc8, rbp = 0x7fffffffed40 ---
      
      Tracing command minicron pid 11941 tid 100080 td 0xfffff8000665b920
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d1e28d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d1e2910
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d1e2990
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe004d1e29c0
      _sleep() at _sleep+0x238/frame 0xfffffe004d1e2a40
      kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe004d1e2aa0
      sys_nanosleep() at sys_nanosleep+0x51/frame 0xfffffe004d1e2ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d1e2bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d1e2bf0
      --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x80095a40a, rsp = 0x7fffffffeb88, rbp = 0x7fffffffebe0 ---
      
      Tracing command minicron pid 11551 tid 100077 td 0xfffff80006450920
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d1d36d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d1d3710
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d1d3790
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe004d1d37c0
      _sleep() at _sleep+0x27d/frame 0xfffffe004d1d3840
      kern_wait6() at kern_wait6+0x5f4/frame 0xfffffe004d1d38f0
      sys_wait4() at sys_wait4+0x72/frame 0xfffffe004d1d3ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d1d3bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d1d3bf0
      --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8008cefba, rsp = 0x7fffffffebe8, rbp = 0x7fffffffec30 ---
      
      Tracing command minicron pid 11319 tid 100157 td 0xfffff800683cc000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d3538d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d353910
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d353990
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe004d3539c0
      _sleep() at _sleep+0x238/frame 0xfffffe004d353a40
      kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe004d353aa0
      sys_nanosleep() at sys_nanosleep+0x51/frame 0xfffffe004d353ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d353bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d353bf0
      --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x80095a40a, rsp = 0x7fffffffeb98, rbp = 0x7fffffffebf0 ---
      
      Tracing command minicron pid 10984 tid 100159 td 0xfffff800687f3920
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d35d6d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d35d710
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d35d790
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe004d35d7c0
      _sleep() at _sleep+0x27d/frame 0xfffffe004d35d840
      kern_wait6() at kern_wait6+0x5f4/frame 0xfffffe004d35d8f0
      sys_wait4() at sys_wait4+0x72/frame 0xfffffe004d35dae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d35dbf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d35dbf0
      --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8008cefba, rsp = 0x7fffffffebf8, rbp = 0x7fffffffec40 ---
      
      Tracing command minicron pid 10812 tid 100163 td 0xfffff800d6dbb000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d3718d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d371910
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d371990
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe004d3719c0
      _sleep() at _sleep+0x238/frame 0xfffffe004d371a40
      kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe004d371aa0
      sys_nanosleep() at sys_nanosleep+0x51/frame 0xfffffe004d371ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d371bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d371bf0
      --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x80095a40a, rsp = 0x7fffffffebb8, rbp = 0x7fffffffec10 ---
      
      Tracing command minicron pid 10357 tid 100146 td 0xfffff800684b6000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d3266d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d326710
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d326790
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe004d3267c0
      _sleep() at _sleep+0x27d/frame 0xfffffe004d326840
      kern_wait6() at kern_wait6+0x5f4/frame 0xfffffe004d3268f0
      sys_wait4() at sys_wait4+0x72/frame 0xfffffe004d326ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d326bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d326bf0
      --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x8008cefba, rsp = 0x7fffffffec18, rbp = 0x7fffffffec60 ---
      
      Tracing command bandwidthd pid 95938 tid 100136 td 0xfffff800683be920
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d2f47d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d2f4810
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d2f4890
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe004d2f48c0
      _sleep() at _sleep+0x238/frame 0xfffffe004d2f4940
      bpfread() at bpfread+0x1c7/frame 0xfffffe004d2f4990
      devfs_read_f() at devfs_read_f+0xeb/frame 0xfffffe004d2f49f0
      dofileread() at dofileread+0x95/frame 0xfffffe004d2f4a40
      kern_readv() at kern_readv+0x68/frame 0xfffffe004d2f4a90
      sys_read() at sys_read+0x63/frame 0xfffffe004d2f4ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d2f4bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d2f4bf0
      --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8014b131a, rsp = 0x7fffffffe8d8, rbp = 0x7fffffffea10 ---
      
      Tracing command bandwidthd pid 95053 tid 100090 td 0xfffff8000665a000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d2187d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d218810
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d218890
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe004d2188c0
      _sleep() at _sleep+0x238/frame 0xfffffe004d218940
      bpfread() at bpfread+0x1c7/frame 0xfffffe004d218990
      devfs_read_f() at devfs_read_f+0xeb/frame 0xfffffe004d2189f0
      dofileread() at dofileread+0x95/frame 0xfffffe004d218a40
      kern_readv() at kern_readv+0x68/frame 0xfffffe004d218a90
      sys_read() at sys_read+0x63/frame 0xfffffe004d218ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d218bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d218bf0
      --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8014b131a, rsp = 0x7fffffffe8d8, rbp = 0x7fffffffea10 ---
      
      Tracing command bandwidthd pid 95003 tid 100160 td 0xfffff800684b5000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d3627d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d362810
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d362890
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe004d3628c0
      _sleep() at _sleep+0x238/frame 0xfffffe004d362940
      bpfread() at bpfread+0x1c7/frame 0xfffffe004d362990
      devfs_read_f() at devfs_read_f+0xeb/frame 0xfffffe004d3629f0
      dofileread() at dofileread+0x95/frame 0xfffffe004d362a40
      kern_readv() at kern_readv+0x68/frame 0xfffffe004d362a90
      sys_read() at sys_read+0x63/frame 0xfffffe004d362ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d362bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d362bf0
      --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8014b131a, rsp = 0x7fffffffe8d8, rbp = 0x7fffffffea10 ---
      
      Tracing command bandwidthd pid 94723 tid 100088 td 0xfffff800065bb490
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d20e7d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d20e810
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d20e890
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe004d20e8c0
      _sleep() at _sleep+0x238/frame 0xfffffe004d20e940
      bpfread() at bpfread+0x1c7/frame 0xfffffe004d20e990
      devfs_read_f() at devfs_read_f+0xeb/frame 0xfffffe004d20e9f0
      dofileread() at dofileread+0x95/frame 0xfffffe004d20ea40
      kern_readv() at kern_readv+0x68/frame 0xfffffe004d20ea90
      sys_read() at sys_read+0x63/frame 0xfffffe004d20eae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d20ebf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d20ebf0
      --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8014b131a, rsp = 0x7fffffffe8d8, rbp = 0x7fffffffea10 ---
      
      Tracing command bandwidthd pid 94044 tid 100093 td 0xfffff800066ad920
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d2277d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d227810
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d227890
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe004d2278c0
      _sleep() at _sleep+0x238/frame 0xfffffe004d227940
      bpfread() at bpfread+0x1c7/frame 0xfffffe004d227990
      devfs_read_f() at devfs_read_f+0xeb/frame 0xfffffe004d2279f0
      dofileread() at dofileread+0x95/frame 0xfffffe004d227a40
      kern_readv() at kern_readv+0x68/frame 0xfffffe004d227a90
      sys_read() at sys_read+0x63/frame 0xfffffe004d227ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d227bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d227bf0
      --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8014b131a, rsp = 0x7fffffffe8d8, rbp = 0x7fffffffea10 ---
      
      Tracing command bandwidthd pid 93233 tid 100110 td 0xfffff80068156000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d27c7d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d27c810
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d27c890
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe004d27c8c0
      _sleep() at _sleep+0x238/frame 0xfffffe004d27c940
      bpfread() at bpfread+0x1c7/frame 0xfffffe004d27c990
      devfs_read_f() at devfs_read_f+0xeb/frame 0xfffffe004d27c9f0
      dofileread() at dofileread+0x95/frame 0xfffffe004d27ca40
      kern_readv() at kern_readv+0x68/frame 0xfffffe004d27ca90
      sys_read() at sys_read+0x63/frame 0xfffffe004d27cae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d27cbf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d27cbf0
      --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8014b131a, rsp = 0x7fffffffe8d8, rbp = 0x7fffffffea10 ---
      
      Tracing command bandwidthd pid 92617 tid 100147 td 0xfffff800684b5920
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d32b7d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d32b810
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d32b890
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe004d32b8c0
      _sleep() at _sleep+0x238/frame 0xfffffe004d32b940
      bpfread() at bpfread+0x1c7/frame 0xfffffe004d32b990
      devfs_read_f() at devfs_read_f+0xeb/frame 0xfffffe004d32b9f0
      dofileread() at dofileread+0x95/frame 0xfffffe004d32ba40
      kern_readv() at kern_readv+0x68/frame 0xfffffe004d32ba90
      sys_read() at sys_read+0x63/frame 0xfffffe004d32bae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d32bbf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d32bbf0
      --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8014b131a, rsp = 0x7fffffffe8d8, rbp = 0x7fffffffea10 ---
      
      Tracing command bandwidthd pid 92206 tid 100142 td 0xfffff80006450490
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d3127d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d312810
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d312890
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe004d3128c0
      _sleep() at _sleep+0x238/frame 0xfffffe004d312940
      bpfread() at bpfread+0x1c7/frame 0xfffffe004d312990
      devfs_read_f() at devfs_read_f+0xeb/frame 0xfffffe004d3129f0
      dofileread() at dofileread+0x95/frame 0xfffffe004d312a40
      kern_readv() at kern_readv+0x68/frame 0xfffffe004d312a90
      sys_read() at sys_read+0x63/frame 0xfffffe004d312ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d312bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d312bf0
      --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8014b131a, rsp = 0x7fffffffe8d8, rbp = 0x7fffffffea10 ---
      
      Tracing command cron pid 78373 tid 100151 td 0xfffff8000644f000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d33f8d0
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d33f910
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d33f990
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe004d33f9c0
      _sleep() at _sleep+0x238/frame 0xfffffe004d33fa40
      kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe004d33faa0
      sys_nanosleep() at sys_nanosleep+0x51/frame 0xfffffe004d33fae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d33fbf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d33fbf0
      --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x800d8040a, rsp = 0x7fffffffea88, rbp = 0x7fffffffecb0 ---
      
      Tracing command syslogd pid 77823 tid 100167 td 0xfffff800d6cb9920
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d385720
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d385760
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d3857e0
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe004d385810
      _cv_wait_sig() at _cv_wait_sig+0x16a/frame 0xfffffe004d385860
      seltdwait() at seltdwait+0xae/frame 0xfffffe004d3858a0
      kern_select() at kern_select+0x8fa/frame 0xfffffe004d385aa0
      sys_select() at sys_select+0x54/frame 0xfffffe004d385ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d385bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d385bf0
      --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800b9529a, rsp = 0x7fffffffdbf8, rbp = 0x7fffffffec10 ---
      
      Tracing command miniupnpd pid 44882 tid 100141 td 0xfffff800683cc920
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d30d710
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d30d750
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d30d7d0
      sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe004d30d800
      _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x18b/frame 0xfffffe004d30d860
      seltdwait() at seltdwait+0xa4/frame 0xfffffe004d30d8a0
      kern_select() at kern_select+0x8fa/frame 0xfffffe004d30daa0
      sys_select() at sys_select+0x54/frame 0xfffffe004d30dae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d30dbf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d30dbf0
      --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80120029a, rsp = 0x7fffffffe028, rbp = 0x7fffffffec50 ---
      
      Tracing command charon pid 34325 tid 100128 td 0xfffff800683a2490
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d2d6850
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d2d6890
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d2d6910
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe004d2d6940
      _sleep() at _sleep+0x27d/frame 0xfffffe004d2d69c0
      umtxq_sleep() at umtxq_sleep+0x125/frame 0xfffffe004d2d6a20
      do_wait() at do_wait+0x329/frame 0xfffffe004d2d6aa0
      __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x83/frame 0xfffffe004d2d6ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d2d6bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d2d6bf0
      --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x8011638cc, rsp = 0x7ffffddeee58, rbp = 0x7ffffddeee80 ---
      
      Tracing command charon pid 34325 tid 100127 td 0xfffff800683a2920
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d2d1850
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d2d1890
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d2d1910
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe004d2d1940
      _sleep() at _sleep+0x27d/frame 0xfffffe004d2d19c0
      umtxq_sleep() at umtxq_sleep+0x125/frame 0xfffffe004d2d1a20
      do_wait() at do_wait+0x329/frame 0xfffffe004d2d1aa0
      __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x83/frame 0xfffffe004d2d1ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d2d1bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d2d1bf0
      --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x8011638cc, rsp = 0x7ffffdfefe58, rbp = 0x7ffffdfefe80 ---
      
      Tracing command charon pid 34325 tid 100126 td 0xfffff800683a3000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d2cc850
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d2cc890
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d2cc910
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe004d2cc940
      _sleep() at _sleep+0x27d/frame 0xfffffe004d2cc9c0
      umtxq_sleep() at umtxq_sleep+0x125/frame 0xfffffe004d2cca20
      do_wait() at do_wait+0x329/frame 0xfffffe004d2ccaa0
      __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x83/frame 0xfffffe004d2ccae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d2ccbf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d2ccbf0
      --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x8011638cc, rsp = 0x7ffffe1f0e58, rbp = 0x7ffffe1f0e80 ---
      
      Tracing command charon pid 34325 tid 100125 td 0xfffff800683a3490
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d2c7850
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d2c7890
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d2c7910
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe004d2c7940
      _sleep() at _sleep+0x27d/frame 0xfffffe004d2c79c0
      umtxq_sleep() at umtxq_sleep+0x125/frame 0xfffffe004d2c7a20
      do_wait() at do_wait+0x329/frame 0xfffffe004d2c7aa0
      __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x83/frame 0xfffffe004d2c7ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d2c7bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d2c7bf0
      --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x8011638cc, rsp = 0x7ffffe3f1e58, rbp = 0x7ffffe3f1e80 ---
      
      Tracing command charon pid 34325 tid 100124 td 0xfffff800683a3920
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d2c2850
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d2c2890
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d2c2910
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe004d2c2940
      _sleep() at _sleep+0x27d/frame 0xfffffe004d2c29c0
      umtxq_sleep() at umtxq_sleep+0x125/frame 0xfffffe004d2c2a20
      do_wait() at do_wait+0x329/frame 0xfffffe004d2c2aa0
      __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x83/frame 0xfffffe004d2c2ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d2c2bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d2c2bf0
      --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x8011638cc, rsp = 0x7ffffe5f2e58, rbp = 0x7ffffe5f2e80 ---
      
      Tracing command charon pid 34325 tid 100123 td 0xfffff800683a4000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d2bd850
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d2bd890
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d2bd910
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe004d2bd940
      _sleep() at _sleep+0x27d/frame 0xfffffe004d2bd9c0
      umtxq_sleep() at umtxq_sleep+0x125/frame 0xfffffe004d2bda20
      do_wait() at do_wait+0x329/frame 0xfffffe004d2bdaa0
      __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x83/frame 0xfffffe004d2bdae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d2bdbf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d2bdbf0
      --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x8011638cc, rsp = 0x7ffffe7f3e58, rbp = 0x7ffffe7f3e80 ---
      
      Tracing command charon pid 34325 tid 100122 td 0xfffff800683a4490
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d2b8850
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d2b8890
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d2b8910
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe004d2b8940
      _sleep() at _sleep+0x27d/frame 0xfffffe004d2b89c0
      umtxq_sleep() at umtxq_sleep+0x125/frame 0xfffffe004d2b8a20
      do_wait() at do_wait+0x329/frame 0xfffffe004d2b8aa0
      __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x83/frame 0xfffffe004d2b8ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d2b8bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d2b8bf0
      --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x8011638cc, rsp = 0x7ffffe9f4e58, rbp = 0x7ffffe9f4e80 ---
      
      Tracing command charon pid 34325 tid 100121 td 0xfffff800683a4920
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d2b3850
      mi_switch() at mi_switch+0xe1/frame 0xfffffe004d2b3890
      sleepq_catch_signals() at sleepq_catch_signals+0xab/frame 0xfffffe004d2b3910
      sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe004d2b3940
      _sleep() at _sleep+0x27d/frame 0xfffffe004d2b39c0
      umtxq_sleep() at umtxq_sleep+0x125/frame 0xfffffe004d2b3a20
      do_wait() at do_wait+0x329/frame 0xfffffe004d2b3aa0
      __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x83/frame 0xfffffe004d2b3ae0
      amd64_syscall() at amd64_syscall+0x351/frame 0xfffffe004d2b3bf0
      Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe004d2b3bf0
      --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip = 0x8011638cc, rsp = 0x7ffffebf5e58, rbp = 0x7ffffebf5e80 ---
      
      Tracing command charon pid 34325 tid 100120 td 0xfffff80006f3a000
      sched_switch() at sched_switch+0x2b3/frame 0xfffffe004d2ae850
      mi_switch() at mi_switch+0xe1/fr[/thread]
      
      1 Reply Last reply Reply Quote 0
      • Z Offline
        zeroluck
        last edited by

        The forum truncated my crash report (and the rest of my post) even though it was in code tags and it worked fine in the forum post preview. Here's my system information:

        Name	fuckmountain.no-ip.biz
        Version	2.2.2-RELEASE (amd64) 
        built on Mon Apr 13 20:10:22 CDT 2015 
        FreeBSD 10.1-RELEASE-p9
        
        You are on the latest version.
        Platform	pfSense
        CPU Type	Dual Core AMD Opteron(tm) Processor 285
        4 CPUs: 2 package(s) x 2 core(s)
        Uptime	00 Hour 13 Minutes 04 Seconds
        Current date/time	
        Sat May 9 9:34:09 CDT 2015
        DNS server(s)	127.0.0.1
        208.67.222.222
        208.67.220.220
        208.67.222.220
        208.67.220.222
        Last config change	Sat May 9 6:30:10 CDT 2015
        State table size	
        0% (1731/508000) 
        Show states
        MBUF Usage	
        10% (2540/26584)
        Load average	
        0.82, 0.86, 0.56
        CPU usage	
        0%
        Memory usage	
        10% of 5084 MB
        SWAP usage	
        0% of 16384 MB
        Disk usage	
        / (ufs): 2% of 129G	
        /var/run (ufs in RAM): 4% of 3.4M	
        
        1 Reply Last reply Reply Quote 0
        • jimpJ Offline
          jimp Rebel Alliance Developer Netgate
          last edited by

          That looks like a problem with the Broadcom (bge) card or driver in that system. Is that an on-board or add-on NIC?
          What, if any, changes have you put in place in /boot/loader.conf.local ?

          Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

          Need help fast? Netgate Global Support!

          Do not Chat/PM for help!

          1 Reply Last reply Reply Quote 0
          • First post
            Last post
          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.