Kernel lockup
-
I wonder if it may be a good idea to track 11-STABLE instead of 11.0 as .0 releases of FreeBSD can be problematic.
The stable branch is a much more moving target than the releng branch, it would be hell a lot more work to keep up with the changes.
-
Have done the 2 ram tests I planned.
Also have the dump device correctly showing during the boot process now so if I have another panic I should get a dump to backtrace.
I have disabled turbo mode and EIST on the cpu and dynamic voltage on the ram.
-
chrcoluk what hardware are you running pfSense
on? Your sig says "Braswell N3150" but that isn't specific. Sorry if I missed it in another post. -
chrcoluk what hardware are you running pfSense
on?That is unacceptable usage. You must say "What hardware are you running pfSense
software on?";D ;D ::) ::)
SCNR. Won't further comment on this new round of legal bleep. I hoped the unfortunate episode ~2+ years ago was enough of a lesson. Apparently not.
-
chrcoluk
Did you add kern.ipc.nmbclusters="1000000"?
What is your WAN bandwidth? -
I dont know the exact hardware packaged as I purchased a compact unit from amazon which is simply labeled as a BRASWELL N3150 unit, which tells you I have a celeron N3150 on a braswell chipset.
I think I am going to adjust the bios again to enable EIST and turbo mode but also configure powerd to never let the clocks drop below the stock max speed that means I get turbo mode back again for extra performance whilst not letting it drop to idle voltages, I will keep the dynamic voltages disabled on the ram.
Also nmbclusters has been increased in the loader.conf from almost day one of using pfsense.
WAN bandwidth at the time of the crash was just idle, the absolute max bandwidth usage possible is about 70mbit down and 20mbit up.
-
Ok, try to bump voltage on memory, +0.1V
-
I dont have that control in the bios.
But I have disabled now dynamic voltage mode (DVFS) so it should always be at full voltage now instead of power saving lower voltages.
-
Am getting issues again, the good news is I am getting proper kernel dumps now, the last two I hit the submit button, but to let more eyes look at the problem I will dump the data here as well, the bit of research I have done seems to indicate its filesystem related, but I am most definitely not sure on this so appreciate an opinion from anyone.
In the meantime since my laptop uses the same spec ram I am going to swap the ram with my laptop memory, even tho this existing ram has passed stability tests.
Crash report begins. Anonymous machine information: amd64 11.0-RELEASE-p8 FreeBSD 11.0-RELEASE-p8 #78 3f54ce3814d(RELENG_2_4): Wed Mar 15 18:23:59 CDT 2017 root@buildbot2.netgate.com:/builder/ce/tmp/obj/builder/ce/tmp/FreeBSD-src/sys/pfSense Crash report details: No PHP errors found. Filename: /var/crash/bounds 1 Filename: /var/crash/info.0 Dump header from device: /dev/ada0p3 Architecture: amd64 Architecture Version: 1 Dump Length: 123392 Blocksize: 512 Dumptime: Wed Jun 7 00:40:31 2017 Hostname: PFSENSE.home Magic: FreeBSD Text Dump Version String: FreeBSD 11.0-RELEASE-p8 #78 3f54ce3814d(RELENG_2_4): Wed Mar 15 18:23:59 CDT 2017 root@buildbot2.netgate.com:/builder/ce/tmp/obj/builder/ce/tmp/FreeBSD-src/sys/pfSense Panic String: Dump Parity: 318326786 Bounds: 0 Dump Status: good Filename: /var/crash/info.last Dump header from device: /dev/ada0p3 Architecture: amd64 Architecture Version: 1 Dump Length: 123392 Blocksize: 512 Dumptime: Wed Jun 7 00:40:31 2017 Hostname: PFSENSE.home Magic: FreeBSD Text Dump Version String: FreeBSD 11.0-RELEASE-p8 #78 3f54ce3814d(RELENG_2_4): Wed Mar 15 18:23:59 CDT 2017 root@buildbot2.netgate.com:/builder/ce/tmp/obj/builder/ce/tmp/FreeBSD-src/sys/pfSense Panic String: Dump Parity: 318326786 Bounds: 0 Dump Status: good Filename: /var/crash/textdump.tar.0 ddb.txt06000014000013115636557 7106 ustarrootwheeldb:0:kdb.enter.default> run lockinfo db:1:lockinfo> show locks No such command db:1:locks> show alllocks No such command db:1:alllocks> show lockedvnods Locked vnodes db:0:kdb.enter.default> show pcpu cpuid = 1 dynamic pcpu = 0xfffffe0167de2000 curthread = 0xfffff8015e162a00: pid 54120 "sh" curpcb = 0xfffffe0113c6dcc0 fpcurthread = 0xfffff8015e162a00: pid 54120 "sh" idlethread = 0xfffff8000490aa00: tid 100004 "idle: cpu1" curpmap = 0xfffff801621b3138 tssp = 0xffffffff82a1dd78 commontssp = 0xffffffff82a1dd78 rsp0 = 0xfffffe0113c6dcc0 gs32p = 0xffffffff82a245d0 ldt = 0xffffffff82a24610 tss = 0xffffffff82a24600 db:0:kdb.enter.default> bt Tracing pid 54120 tid 100774 td 0xfffff8015e162a00 turnstile_broadcast() at turnstile_broadcast+0x9c/frame 0xfffffe0113c6d480 __rw_wunlock_hard() at __rw_wunlock_hard+0x8f/frame 0xfffffe0113c6d4b0 vm_map_delete() at vm_map_delete+0x3dc/frame 0xfffffe0113c6d530 vm_map_remove() at vm_map_remove+0x47/frame 0xfffffe0113c6d560 exec_new_vmspace() at exec_new_vmspace+0x22f/frame 0xfffffe0113c6d5e0 exec_elf64_imgact() at exec_elf64_imgact+0xa58/frame 0xfffffe0113c6d6f0 kern_execve() at kern_execve+0x74d/frame 0xfffffe0113c6da50 sys_execve() at sys_execve+0x4a/frame 0xfffffe0113c6dad0 amd64_syscall() at amd64_syscall+0x4ce/frame 0xfffffe0113c6dbf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0113c6dbf0 --- syscall (59, FreeBSD ELF64, sys_execve), rip = 0x800b40d8a, rsp = 0x7fffffffe188, rbp = 0x7fffffffe2d0 --- db:0:kdb.enter.default> ps pid ppid pgrp uid state wmesg wchan cmd 54226 53957 334 0 R CPU 0 sh 54120 53957 334 0 R CPU 1 sh 54035 53957 334 0 R CPU 2 cat 53957 33060 334 0 R CPU 3 sh 39824 334 334 0 S accept 0xfffff8003b01006c php-fpm 43353 32987 43353 0 S+ select 0xfffff801629be7c0 top 32987 2745 32987 0 S+ wait 0xfffff8003b194528 bash 2745 2354 2745 0 Ss+ wait 0xfffff8003b03d528 sh 2354 8180 2354 0 Ss select 0xfffff80102b6a5c0 sshd 21591 1 334 0 S nanslp 0xffffffff82865d21 php 21318 1 21318 0 Ss select 0xfffff80102d507c0 upsd 21063 1 21063 66 Ss select 0xfffff8003b13e6c0 apcupsd-ups 19999 19580 19580 66 S nanslp 0xffffffff82865d21 upsmon 19580 1 19580 0 Ss piperd 0xfffff80162a03be0 upsmon 33060 1 334 0 S piperd 0xfffff8015e2bf8e8 sh 15682 1 15682 136 Ss select 0xfffff8001d948240 dhcpd 88257 1 88257 59 Ss (threaded) unbound 100711 S kqread 0xfffff8015eee6c00 unbound 100760 S kqread 0xfffff80011585800 unbound 100761 S kqread 0xfffff80011585a00 unbound 100762 S kqread 0xfffff800101aea00 unbound 55992 1 55992 0 Ss (threaded) dpinger 100681 S uwait 0xfffff80162823e00 dpinger 100729 S sbwait 0xfffff8016285c4a4 dpinger 100730 S nanslp 0xffffffff82865d20 dpinger 100731 S nanslp 0xffffffff82865d21 dpinger 100732 S accept 0xfffff80162d3d72c dpinger 55571 1 55571 0 Ss (threaded) dpinger 100701 S uwait 0xfffff8001d4c3b80 dpinger 100725 S sbwait 0xfffff80162a2f144 dpinger 100726 S nanslp 0xffffffff82865d22 dpinger 100727 S nanslp 0xffffffff82865d20 dpinger 100728 S accept 0xfffff801620e2a8c dpinger 89259 1 89259 0 Ss select 0xfffff8001d4c1e40 openvpn 87846 1 87846 0 Ss select 0xfffff8016220e7c0 openvpn 68550 1 68550 0 Ss select 0xfffff80102d520c0 powerd 67165 67155 67165 0 S+ ttyin 0xfffff8001d3dfca8 sh 67155 65512 67155 0 S+ wait 0xfffff8016212f528 sh 66898 98897 66898 0 Ss (threaded) sshlockout_pf 100605 S piperd 0xfffff8001d9db000 sshlockout_pf 100683 S nanslp 0xffffffff82865d20 sshlockout_pf 66742 1 66742 0 Ss+ ttyin 0xfffff8001d46b0a8 getty 66646 1 66646 0 Ss+ ttyin 0xfffff8001d46b4a8 getty 66575 1 66575 0 Ss+ ttyin 0xfffff8001d46b8a8 getty 66538 1 66538 0 Ss+ ttyin 0xfffff8001d46bca8 getty 66211 1 66211 0 Ss+ ttyin 0xfffff8001d46c0a8 getty 66158 1 66158 0 Ss+ ttyin 0xfffff80005179ca8 getty 65836 1 65836 0 Ss+ ttyin 0xfffff8001d3df0a8 getty 65512 1 65512 0 Ss+ wait 0xfffff8001d976a50 login 50681 50482 50482 0 S nanslp 0xffffffff82865d20 minicron 50482 1 50482 0 Ss wait 0xfffff8001df18a50 minicron 50272 49938 49938 0 S nanslp 0xffffffff82865d23 minicron 49938 1 49938 0 Ss wait 0xfffff8001d976528 minicron 49476 49314 49314 0 S nanslp 0xffffffff82865d23 minicron 49314 1 49314 0 Ss wait 0xfffff8016212aa50 minicron 44216 1 44216 0 Ss (threaded) apcupsd 100658 S nanslp 0xffffffff82865d21 apcupsd 100670 S sigwait 0xfffff80162a01000 apcupsd 100671 S accept 0xfffff8016212206c apcupsd 3948 1 3848 0 S kqread 0xfffff8001dc25200 lighttpd_pfb 98897 1 98897 0 Ss select 0xfffff80102d509c0 syslogd 53013 1 53013 0 Ss kqread 0xfffff8001dbe1600 dnscrypt-proxy 52232 1 52232 0 Ss kqread 0xfffff8001db85000 dnscrypt-proxy 50361 1 50361 0 Ss kqread 0xfffff8001df2d900 dnscrypt-proxy 47847 1 47764 0 S select 0xfffff8003b3464c0 radvd 40799 1 40799 0 Ss (threaded) ntpd 100646 S select 0xfffff8003b1408c0 ntpd 40296 0 0 0 DL mdwait 0xfffff80102bf3000 [md3] 38103 1 38103 0 Ss nanslp 0xffffffff82865d22 cron 37771 37268 37268 0 S kqread 0xfffff8001dc40600 nginx 37735 37268 37268 0 S kqread 0xfffff8001d5b0700 nginx 37705 37268 37268 0 S kqread 0xfffff8001dc1c900 nginx 37554 37268 37268 0 S kqread 0xfffff8001dbbaa00 nginx 37494 37268 37268 0 S kqread 0xfffff8001db43a00 nginx 37268 1 37268 0 Ss pause 0xfffff80102eecaf8 nginx 20138 1 20138 0 Ss bpf 0xfffff8003bd67000 filterlog 15349 1 15349 0 Ss select 0xfffff8003b13e1c0 dhcp6c 14626 1 14626 65 Ss select 0xfffff8001dc38ec0 dhclient 9309 1 9309 0 Ss select 0xfffff8003b3460c0 dhclient 8519 1 8519 0 Ss (threaded) sshlockout_pf 100597 S uwait 0xfffff8001dc37600 sshlockout_pf 100601 S nanslp 0xffffffff82865d20 sshlockout_pf 8180 1 8180 0 Ss select 0xfffff8001d4c2640 sshd 393 0 0 0 DL mdwait 0xfffff8001daf1800 [md2] 383 0 0 0 DL mdwait 0xfffff8003b132000 [md1] 368 1 368 0 Ss select 0xfffff8003b1411c0 devd 350 348 348 0 S kqread 0xfffff8001dcabd00 check_reload_status 348 1 348 0 Ss kqread 0xfffff8001db94800 check_reload_status 334 1 334 0 Ss kqread 0xfffff8001ddd2100 php-fpm 103 0 0 0 DL mdwait 0xfffff8001d565000 [md0] 27 0 0 0 DL syncer 0xffffffff8295f880 [syncer] 26 0 0 0 DL vlruwt 0xfffff8001d406528 [vnlru] 25 0 0 0 DL psleep 0xffffffff8295e104 [bufdaemon] 24 0 0 0 DL - 0xffffffff8295edb4 [bufspacedaemon] 23 0 0 0 DL pgzero 0xffffffff829734e4 [pagezero] 22 0 0 0 DL psleep 0xffffffff8296f98c [vmdaemon] 21 0 0 0 DL (threaded) [pagedaemon] 100072 D psleep 0xffffffff82a1d105 [pagedaemon] 100081 D umarcl 0xffffffff8296f2b8 [uma] 20 0 0 0 DL - 0xffffffff8295d914 [soaiod4] 19 0 0 0 DL - 0xffffffff8295d914 [soaiod3] 18 0 0 0 DL - 0xffffffff8295d914 [soaiod2] 9 0 0 0 DL - 0xffffffff8295d914 [soaiod1] 17 0 0 0 DL cooling 0xfffff80005139f58 [acpi_cooling0] 16 0 0 0 DL tzpoll 0xffffffff82621138 [acpi_thermal] 8 0 0 0 DL - 0xffffffff8273a4f0 [rand_harvestq] 7 0 0 0 DL pftm 0xffffffff80f4b6a0 [pf purge] 6 0 0 0 DL waiting_ 0xffffffff82a0c740 [sctp_iterator] 5 0 0 0 DL (threaded) [zfskern] 100055 D arc_recl 0xffffffff82f3f4d0 [arc_reclaim_thread] 100056 D arc_user 0xffffffff82f3f4f0 [arc_user_evicts_thr] 100057 D arc_dnlc 0xffffffff82f3f480 [arc_dnlc_evicts_thr] 100058 D l2arc_fe 0xffffffff82f3f888 [l2arc_feed_thread] 100550 D spa->spa 0xfffff8001d55fbd0 [trim zroot] 100564 D tx->tx_q 0xfffff8001d45fa30 [txg_thread_enter] 100565 D tx->tx_s 0xfffff8001d45fa10 [txg_thread_enter] 15 0 0 0 DL (threaded) [usb] 100035 D - 0xfffffe0001121460 [usbus0] 100036 D - 0xfffffe00011214b8 [usbus0] 100037 D - 0xfffffe0001121510 [usbus0] 100038 D - 0xfffffe0001121568 [usbus0] 100039 D - 0xfffffe00011215c0 [usbus0] 4 0 0 0 DL (threaded) [cam] 100032 D - 0xffffffff8260d380 [doneq0] 100067 D - 0xffffffff8260d1c8 [scanner] 3 0 0 0 DL crypto_r 0xffffffff8296de70 [crypto returns] 2 0 0 0 DL crypto_w 0xffffffff8296dd18 [crypto] 14 0 0 0 DL (threaded) [geom] 100023 D - 0xffffffff829e4940 [g_event] 100024 D - 0xffffffff829e4948 [g_up] 100025 D - 0xffffffff829e4950 [g_down] 13 0 0 0 DL (threaded) [ng_queue] 100019 D sleep 0xffffffff825cb230 [ng_queue0] 100020 D sleep 0xffffffff825cb230 [ng_queue1] 100021 D sleep 0xffffffff825cb230 [ng_queue2] 100022 D sleep 0xffffffff825cb230 [ng_queue3] 12 0 0 0 WL (threaded) [intr] 100007 I [swi3: vm] 100008 I [swi4: clock (0)] 100009 I [swi4: clock (1)] 100010 I [swi4: clock (2)] 100011 I [swi4: clock (3)] 100012 I [swi1: netisr 0] 100013 I [swi5: fast taskq] 100015 I [swi6: task queue] 100016 I [swi6: Giant taskq] 100033 I [irq256: ahci0] 100034 I [irq257: xhci0] 100040 I [irq258: re0] 100041 I [irq259: pcib2] 100042 I [irq260: re1] 100043 I [irq261: igb0:que 0] 100045 I [irq262: igb0:link] 100046 I [irq263: igb1:que 0] 100048 I [irq264: igb1:link] 100061 I [swi1: pf send] 100062 I [swi1: pfsync] 100079 I [swi1: netisr 1] 100080 I [swi1: netisr 2] 100082 I [swi1: netisr 3] 11 0 0 0 RL (threaded) [idle] 100003 CanRun [idle: cpu0] 100004 CanRun [idle: cpu1] 100005 CanRun [idle: cpu2] 100006 CanRun [idle: cpu3] 1 0 1 0 SLs wait 0xfffff80004908528 [init] 10 0 0 0 DL audit_wo 0xffffffff82a138c0 [audit] 0 0 0 0 DLs (threaded) [kernel] 100000 D swapin 0xffffffff829e4978 [swapper] 100014 D - 0xfffff8000491ee00 [kqueue_ctx taskq] 100017 D - 0xfffff8000491e600 [aiod_kick taskq] 100018 D - 0xfffff8000491e400 [thread] 100026 D - 0xfffff8000491d600 [firmware taskq] 100029 D - 0xfffff80004ae4d00 [acpi_task_0] 100030 D - 0xfffff80004ae4d00 [acpi_task_1] 100031 D - 0xfffff80004ae4d00 [acpi_task_2] 100044 D - 0xfffff80005106b00 [igb0 que] 100047 D - 0xfffff80004ba4d00 [igb1 que] 100049 D - 0xfffff80005162800 [system_taskq_0] 100050 D - 0xfffff80005162800 [system_taskq_1] 100051 D - 0xfffff80005162800 [system_taskq_2] 100052 D - 0xfffff80005162800 [system_taskq_3] 100053 D - 0xfffff80004ae5000 [mca taskq] 100054 D - 0xfffff80005161b00 [dbu_evict] 100066 D - 0xfffff80004ae4900 [CAM taskq] 100083 D - 0xfffff8001d467300 [if_io_tqg_0] 100084 D - 0xfffff8001d467100 [if_io_tqg_1] 100085 D - 0xfffff8001d4a4b00 [if_io_tqg_2] 100086 D - 0xfffff8001d4a4900 [if_io_tqg_3] 100087 D - 0xfffff8001d4a4700 [if_config_tqg_0] 100322 D - 0xfffff8001d5fed00 [zio_null_issue] 100323 D - 0xfffff8001d5b2600 [zio_null_intr] 100324 D - 0xfffff8001d5b2300 [zio_read_issue_0] 100325 D - 0xfffff8001d5b2300 [zio_read_issue_1] 100326 D - 0xfffff8001d5b2300 [zio_read_issue_2] 100327 D - 0xfffff8001d5b2300 [zio_read_issue_3] 100328 D - 0xfffff8001d5b2300 [zio_read_issue_4] 100329 D - 0xfffff8001d5b2300 [zio_read_issue_5] 100330 D - 0xfffff8001d5b2300 [zio_read_issue_6] 100331 D - 0xfffff8001d5b2300 [zio_read_issue_7] 100332 D - 0xfffff8001d5ff700 [zio_read_intr_0_0] 100333 D - 0xfffff8001d5ff700 [zio_read_intr_0_1] 100334 D - 0xfffff8001d5ff700 [zio_read_intr_0_2] 100335 D - 0xfffff8001d5ff700 [zio_read_intr_0_3] 100336 D - 0xfffff8001d5ff700 [zio_read_intr_0_4] 100337 D - 0xfffff8001d5ff700 [zio_read_intr_0_5] 100338 D - 0xfffff8001d5ff700 [zio_read_intr_0_6] 100339 D - 0xfffff8001d5ff700 [zio_read_intr_0_7] 100340 D - 0xfffff8001d5ff700 [zio_read_intr_0_8] 100341 D - 0xfffff8001d5ff700 [zio_read_intr_0_9] 100342 D - 0xfffff8001d5ff700 [zio_read_intr_0_10] 100343 D - 0xfffff8001d5ff700 [zio_read_intr_0_11] 100344 D - 0xfffff8001d5b2200 [zio_read_intr_1_0] 100345 D - 0xfffff8001d5b2200 [zio_read_intr_1_1] 100346 D - 0xfffff8001d5b2200 [zio_read_intr_1_2] 100347 D - 0xfffff8001d5b2200 [zio_read_intr_1_3] 100348 D - 0xfffff8001d5b2200 [zio_read_intr_1_4] 100349 D - 0xfffff8001d5b2200 [zio_read_intr_1_5] 100350 D - 0xfffff8001d5b2200 [zio_read_intr_1_6] 100351 D - 0xfffff8001d5b2200 [zio_read_intr_1_7] 100352 D - 0xfffff8001d5b2200 [zio_read_intr_1_8] 100353 D - 0xfffff8001d5b2200 [zio_read_intr_1_9] 100354 D - 0xfffff8001d5b2200 [zio_read_intr_1_10] 100355 D - 0xfffff8001d5b2200 [zio_read_intr_1_11] 100356 D - 0xfffff8001d5fda00 [zio_read_intr_2_0] 100357 D - 0xfffff8001d5fda00 [zio_read_intr_2_1] 100358 D - 0xfffff8001d5fda00 [zio_read_intr_2_2] 100359 D - 0xfffff8001d5fda00 [zio_read_intr_2_3] 100360 D - 0xfffff8001d5fda00 [zio_read_intr_2_4] 100361 D - 0xfffff8001d5fda00 [zio_read_intr_2_5] 100362 D - 0xfffff8001d5fda00 [zio_read_intr_2_6] 100363 D - 0xfffff8001d5fda00 [zio_read_intr_2_7] 100364 D - 0xfffff8001d5fda00 [zio_read_intr_2_8] 100365 D - 0xfffff8001d5fda00 [zio_read_intr_2_9] 100366 D - 0xfffff8001d5fda00 [zio_read_intr_2_10] 100367 D - 0xfffff8001d5fda00 [zio_read_intr_2_11] 100368 D - 0xfffff8001d5ddc00 [zio_read_intr_3_0] 100369 D - 0xfffff8001d5ddc00 [zio_read_intr_3_1] 100370 D - 0xfffff8001d5ddc00 [zio_read_intr_3_2] 100371 D - 0xfffff8001d5ddc00 [zio_read_intr_3_3] 100372 D - 0xfffff8001d5ddc00 [zio_read_intr_3_4] 100373 D - 0xfffff8001d5ddc00 [zio_read_intr_3_5] 100374 D - 0xfffff8001d5ddc00 [zio_read_intr_3_6] 100375 D - 0xfffff8001d5ddc00 [zio_read_intr_3_7] 100376 D - 0xfffff8001d5ddc00 [zio_read_intr_3_8] 100377 D - 0xfffff8001d5ddc00 [zio_read_intr_3_9] 100378 D - 0xfffff8001d5ddc00 [zio_read_intr_3_10] 100379 D - 0xfffff8001d5ddc00 [zio_read_intr_3_11] 100380 D - 0xfffff8001d552500 [zio_read_intr_4_0] 100381 D - 0xfffff8001d552500 [zio_read_intr_4_1] 100382 D - 0xfffff8001d552500 [zio_read_intr_4_2] 100383 D - 0xfffff8001d552500 [zio_read_intr_4_3] 100384 D - 0xfffff8001d552500 [zio_read_intr_4_4] 100385 D - 0xfffff8001d552500 [zio_read_intr_4_5] 100386 D - 0xfffff8001d552500 [zio_read_intr_4_6] 100387 D - 0xfffff8001d552500 [zio_read_intr_4_7] 100388 D - 0xfffff8001d552500 [zio_read_intr_4_8] 100389 D - 0xfffff8001d552500 [zio_read_intr_4_9] 100390 D - 0xfffff8001d552500 [zio_read_intr_4_10] 100391 D - 0xfffff8001d552500 [zio_read_intr_4_11] 100392 D - 0xfffff8001d4a2800 [zio_read_intr_5_0] 100393 D - 0xfffff8001d4a2800 [zio_read_intr_5_1] 100394 D - 0xfffff8001d4a2800 [zio_read_intr_5_2] 100395 D - 0xfffff8001d4a2800 [zio_read_intr_5_3] 100396 D - 0xfffff8001d4a2800 [zio_read_intr_5_4] 100397 D - 0xfffff8001d4a2800 [zio_read_intr_5_5] 100398 D - 0xfffff8001d4a2800 [zio_read_intr_5_6] 100399 D - 0xfffff8001d4a2800 [zio_read_intr_5_7] 100400 D - 0xfffff8001d4a2800 [zio_read_intr_5_8] 100401 D - 0xfffff8001d4a2800 [zio_read_intr_5_9] 100402 D - 0xfffff8001d4a2800 [zio_read_intr_5_10] 100403 D - 0xfffff8001d4a2800 [zio_read_intr_5_11] 100404 D - 0xfffff8001d4a2300 [zio_read_intr_6_0] 100405 D - 0xfffff8001d4a2300 [zio_read_intr_6_1] 100406 D - 0xfffff8001d4a2300 [zio_read_intr_6_2] 100407 D - 0xfffff8001d4a2300 [zio_read_intr_6_3] 100408 D - 0xfffff8001d4a2300 [zio_read_intr_6_4] 100409 D - 0xfffff8001d4a2300 [zio_read_intr_6_5] 100410 D - 0xfffff8001d4a2300 [zio_read_intr_6_6] 100411 D - 0xfffff8001d4a2300 [zio_read_intr_6_7] 100412 D - 0xfffff8001d4a2300 [zio_read_intr_6_8] 100413 D - 0xfffff8001d4a2300 [zio_read_intr_6_9] 100414 D - 0xfffff8001d4a2300 [zio_read_intr_6_10] 100415 D - 0xfffff8001d4a2300 [zio_read_intr_6_11] 100416 D - 0xfffff8001d4a1000 [zio_read_intr_7_0] 100417 D - 0xfffff8001d4a1000 [zio_read_intr_7_1] 100418 D - 0xfffff8001d4a1000 [zio_read_intr_7_2] 100419 D - 0xfffff8001d4a1000 [zio_read_intr_7_3] 100420 D - 0xfffff8001d4a1000 [zio_read_intr_7_4] 100421 D - 0xfffff8001d4a1000 [zio_read_intr_7_5] 100422 D - 0xfffff8001d4a1000 [zio_read_intr_7_6] 100423 D - 0xfffff8001d4a1000 [zio_read_intr_7_7] 100424 D - 0xfffff8001d4a1000 [zio_read_intr_7_8] 100425 D - 0xfffff8001d4a1000 [zio_read_intr_7_9] 100426 D - 0xfffff8001d4a1000 [zio_read_intr_7_10] 100427 D - 0xfffff8001d4a1000 [zio_read_intr_7_11] 100428 D - 0xfffff8001d589300 [zio_write_issue_0] 100429 D - 0xfffff8001d589300 [zio_write_issue_1] 100430 D - 0xfffff8001d589300 [zio_write_issue_2] 100431 D - 0xfffff8001d589200 [zio_write_issue_hig] 100432 D - 0xfffff8001d589200 [zio_write_issue_hig] 100433 D - 0xfffff8001d589200 [zio_write_issue_hig] 100434 D - 0xfffff8001d589200 [zio_write_issue_hig] 100435 D - 0xfffff8001d589200 [zio_write_issue_hig] 100436 D - 0xfffff8001d589000 [zio_write_intr_0] 100437 D - 0xfffff8001d589000 [zio_write_intr_1] 100438 D - 0xfffff8001d589000 [zio_write_intr_2] 100439 D - 0xfffff8001d589000 [zio_write_intr_3] 100440 D - 0xfffff8001d589000 [zio_write_intr_4] 100441 D - 0xfffff8001d589000 [zio_write_intr_5] 100442 D - 0xfffff8001d589000 [zio_write_intr_6] 100443 D - 0xfffff8001d589000 [zio_write_intr_7] 100444 D - 0xfffff8001d588100 [zio_write_intr_high] 100445 D - 0xfffff8001d588100 [zio_write_intr_high] 100446 D - 0xfffff8001d588100 [zio_write_intr_high] 100447 D - 0xfffff8001d588100 [zio_write_intr_high] 100448 D - 0xfffff8001d588100 [zio_write_intr_high] 100449 D - 0xfffff8001d587800 [zio_free_issue_0_0] 100450 D - 0xfffff8001d587800 [zio_free_issue_0_1] 100451 D - 0xfffff8001d587800 [zio_free_issue_0_2] 100452 D - 0xfffff8001d587800 [zio_free_issue_0_3] 100453 D - 0xfffff8001d587800 [zio_free_issue_0_4] 100454 D - 0xfffff8001d587800 [zio_free_issue_0_5] 100455 D - 0xfffff8001d587800 [zio_free_issue_0_6] 100456 D - 0xfffff8001d587800 [zio_free_issue_0_7] 100457 D - 0xfffff8001d587800 [zio_free_issue_0_8] 100458 D - 0xfffff8001d587800 [zio_free_issue_0_9] 100459 D - 0xfffff8001d587800 [zio_free_issue_0_10] 100460 D - 0xfffff8001d587800 [zio_free_issue_0_11] 100461 D - 0xfffff8001d587100 [zio_free_issue_1_0] 100462 D - 0xfffff8001d587100 [zio_free_issue_1_1] 100463 D - 0xfffff8001d587100 [zio_free_issue_1_2] 100464 D - 0xfffff8001d587100 [zio_free_issue_1_3] 100465 D - 0xfffff8001d587100 [zio_free_issue_1_4] 100466 D - 0xfffff8001d587100 [zio_free_issue_1_5] 100467 D - 0xfffff8001d587100 [zio_free_issue_1_6] 100468 D - 0xfffff8001d587100 [zio_free_issue_1_7] 100469 D - 0xfffff8001d587100 [zio_free_issue_1_8] 100470 D - 0xfffff8001d587100 [zio_free_issue_1_9] 100471 D - 0xfffff8001d587100 [zio_free_issue_1_10] 100472 D - 0xfffff8001d587100 [zio_free_issue_1_11] 100473 D - 0xfffff8001d5b1d00 [zio_free_issue_2_0] 100474 D - 0xfffff8001d5b1d00 [zio_free_issue_2_1] 100475 D - 0xfffff8001d5b1d00 [zio_free_issue_2_2] 100476 D - 0xfffff8001d5b1d00 [zio_free_issue_2_3] 100477 D - 0xfffff8001d5b1d00 [zio_free_issue_2_4] 100478 D - 0xfffff8001d5b1d00 [zio_free_issue_2_5] 100479 D - 0xfffff8001d5b1d00 [zio_free_issue_2_6] 100480 D - 0xfffff8001d5b1d00 [zio_free_issue_2_7] 100481 D - 0xfffff8001d5b1d00 [zio_free_issue_2_8] 100482 D - 0xfffff8001d5b1d00 [zio_free_issue_2_9] 100483 D - 0xfffff8001d5b1d00 [zio_free_issue_2_10] 100484 D - 0xfffff8001d5b1d00 [zio_free_issue_2_11] 100485 D - 0xfffff8001d5b1000 [zio_free_issue_3_0] 100486 D - 0xfffff8001d5b1000 [zio_free_issue_3_1] 100487 D - 0xfffff8001d5b1000 [zio_free_issue_3_2] 100488 D - 0xfffff8001d5b1000 [zio_free_issue_3_3] 100489 D - 0xfffff8001d5b1000 [zio_free_issue_3_4] 100490 D - 0xfffff8001d5b1000 [zio_free_issue_3_5] 100491 D - 0xfffff8001d5b1000 [zio_free_issue_3_6] 100492 D - 0xfffff8001d5b1000 [zio_free_issue_3_7] 100493 D - 0xfffff8001d5b1000 [zio_free_issue_3_8] 100494 D - 0xfffff8001d5b1000 [zio_free_issue_3_9] 100495 D - 0xfffff8001d5b1000 [zio_free_issue_3_10] 100496 D - 0xfffff8001d5b1000 [zio_free_issue_3_11] 100497 D - 0xfffff8001d5afb00 [zio_free_issue_4_0] 100498 D - 0xfffff8001d5afb00 [zio_free_issue_4_1] 100499 D - 0xfffff8001d5afb00 [zio_free_issue_4_2] 100500 D - 0xfffff8001d5afb00 [zio_free_issue_4_3] 100501 D - 0xfffff8001d5afb00 [zio_free_issue_4_4] 100502 D - 0xfffff8001d5afb00 [zio_free_issue_4_5] 100503 D - 0xfffff8001d5afb00 [zio_free_issue_4_6] 100504 D - 0xfffff8001d5afb00 [zio_free_issue_4_7] 100505 D - 0xfffff8001d5afb00 [zio_free_issue_4_8] 100506 D - 0xfffff8001d5afb00 [zio_free_issue_4_9] 100507 D - 0xfffff8001d5afb00 [zio_free_issue_4_10] 100508 D - 0xfffff8001d5afb00 [zio_free_issue_4_11] 100509 D - 0xfffff8001d58ab00 [zio_free_issue_5_0] 100510 D - 0xfffff8001d58ab00 [zio_free_issue_5_1] 100511 D - 0xfffff8001d58ab00 [zio_free_issue_5_2] 100512 D - 0xfffff8001d58ab00 [zio_free_issue_5_3] 100513 D - 0xfffff8001d58ab00 [zio_free_issue_5_4] 100514 D - 0xfffff8001d58ab00 [zio_free_issue_5_5] 100515 D - 0xfffff8001d58ab00 [zio_free_issue_5_6] [/thread] -
continued here - hit post limit
100516 D - 0xfffff8001d58ab00 [zio_free_issue_5_7] 100517 D - 0xfffff8001d58ab00 [zio_free_issue_5_8] 100518 D - 0xfffff8001d58ab00 [zio_free_issue_5_9] 100519 D - 0xfffff8001d58ab00 [zio_free_issue_5_10] 100520 D - 0xfffff8001d58ab00 [zio_free_issue_5_11] 100521 D - 0xfffff8001d5dd200 [zio_free_issue_6_0] 100522 D - 0xfffff8001d5dd200 [zio_free_issue_6_1] 100523 D - 0xfffff8001d5dd200 [zio_free_issue_6_2] 100524 D - 0xfffff8001d5dd200 [zio_free_issue_6_3] 100525 D - 0xfffff8001d5dd200 [zio_free_issue_6_4] 100526 D - 0xfffff8001d5dd200 [zio_free_issue_6_5] 100527 D - 0xfffff8001d5dd200 [zio_free_issue_6_6] 100528 D - 0xfffff8001d5dd200 [zio_free_issue_6_7] 100529 D - 0xfffff8001d5dd200 [zio_free_issue_6_8] 100530 D - 0xfffff8001d5dd200 [zio_free_issue_6_9] 100531 D - 0xfffff8001d5dd200 [zio_free_issue_6_10] 100532 D - 0xfffff8001d5dd200 [zio_free_issue_6_11] 100533 D - 0xfffff8001d5dc400 [zio_free_issue_7_0] 100534 D - 0xfffff8001d5dc400 [zio_free_issue_7_1] 100535 D - 0xfffff8001d5dc400 [zio_free_issue_7_2] 100536 D - 0xfffff8001d5dc400 [zio_free_issue_7_3] 100537 D - 0xfffff8001d5dc400 [zio_free_issue_7_4] 100538 D - 0xfffff8001d5dc400 [zio_free_issue_7_5] 100539 D - 0xfffff8001d5dc400 [zio_free_issue_7_6] 100540 D - 0xfffff8001d5dc400 [zio_free_issue_7_7] 100541 D - 0xfffff8001d5dc400 [zio_free_issue_7_8] 100542 D - 0xfffff8001d5dc400 [zio_free_issue_7_9] 100543 D - 0xfffff8001d5dc400 [zio_free_issue_7_10] 100544 D - 0xfffff8001d5dc400 [zio_free_issue_7_11] 100545 D - 0xfffff8001d5db700 [zio_free_intr] 100546 D - 0xfffff8001d5db300 [zio_claim_issue] 100547 D - 0xfffff8001d5db600 [zio_claim_intr] 100548 D - 0xfffff8001d5dae00 [zio_ioctl_issue] 100549 D - 0xfffff8001d5dab00 [zio_ioctl_intr] 100551 D - 0xfffff8001d5dac00 [metaslab_group_task] 100552 D - 0xfffff8001d5dac00 [metaslab_group_task] 100553 D - 0xfffff8001d5da000 [zfs_vn_rele_taskq] 100566 D - 0xfffff8001d8dd200 [zil_clean] 100572 D - 0xfffff8001daa6600 [zil_clean] 100573 D - 0xfffff8001d9d8a00 [zil_clean] 100574 D - 0xfffff8001d6fd100 [zil_clean] oops, ran out of processes early! db:0:kdb.enter.default> alltrace Tracing command sh pid 54226 tid 100593 td 0xfffff8001dece000 cpustop_handler() at cpustop_handler+0x28/frame 0xffffffff82977730 ipi_nmi_handler() at ipi_nmi_handler+0x4a/frame 0xffffffff82977750 trap() at trap+0x3a/frame 0xffffffff82977960 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff82977960 --- trap 0x13, rip = 0xffffffff811a4a14, rsp = 0xfffffe0113a4f490, rbp = 0xfffffe0113a4f560 --- pmap_remove_pages() at pmap_remove_pages+0x384/frame 0xfffffe0113a4f560 exec_new_vmspace() at exec_new_vmspace+0x21d/frame 0xfffffe0113a4f5e0 exec_elf64_imgact() at exec_elf64_imgact+0xa58/frame 0xfffffe0113a4f6f0 kern_execve() at kern_execve+0x74d/frame 0xfffffe0113a4fa50 sys_execve() at sys_execve+0x4a/frame 0xfffffe0113a4fad0 amd64_syscall() at amd64_syscall+0x4ce/frame 0xfffffe0113a4fbf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0113a4fbf0 --- syscall (59, FreeBSD ELF64, sys_execve), rip = 0x800b40d8a, rsp = 0x7fffffffe188, rbp = 0x7fffffffe2d0 --- Tracing command sh pid 54120 tid 100774 td 0xfffff8015e162a00 turnstile_broadcast() at turnstile_broadcast+0x9c/frame 0xfffffe0113c6d480 __rw_wunlock_hard() at __rw_wunlock_hard+0x8f/frame 0xfffffe0113c6d4b0 vm_map_delete() at vm_map_delete+0x3dc/frame 0xfffffe0113c6d530 vm_map_remove() at vm_map_remove+0x47/frame 0xfffffe0113c6d560 exec_new_vmspace() at exec_new_vmspace+0x22f/frame 0xfffffe0113c6d5e0 exec_elf64_imgact() at exec_elf64_imgact+0xa58/frame 0xfffffe0113c6d6f0 kern_execve() at kern_execve+0x74d/frame 0xfffffe0113c6da50 sys_execve() at sys_execve+0x4a/frame 0xfffffe0113c6dad0 amd64_syscall() at amd64_syscall+0x4ce/frame 0xfffffe0113c6dbf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0113c6dbf0 --- syscall (59, FreeBSD ELF64, sys_execve), rip = 0x800b40d8a, rsp = 0x7fffffffe188, rbp = 0x7fffffffe2d0 --- Tracing command cat pid 54035 tid 100570 td 0xfffff8001d601000 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00ea3f4cf0 ipi_nmi_handler() at ipi_nmi_handler+0x4a/frame 0xfffffe00ea3f4d10 trap() at trap+0x3a/frame 0xfffffe00ea3f4f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00ea3f4f20 --- trap 0x13, rip = 0xffffffff82e91de0, rsp = 0xfffffe01137de588, rbp = 0xfffffe01137de5b0 --- zrl_add_impl() at zrl_add_impl/frame 0xfffffe01137de5b0 zfs_freebsd_getattr() at zfs_freebsd_getattr+0xa2e/frame 0xfffffe01137de820 VOP_GETATTR_APV() at VOP_GETATTR_APV+0x87/frame 0xfffffe01137de850 vm_mmap_vnode() at vm_mmap_vnode+0x164/frame 0xfffffe01137de990 vn_mmap() at vn_mmap+0xe5/frame 0xfffffe01137dea20 sys_mmap() at sys_mmap+0x40c/frame 0xfffffe01137dead0 amd64_syscall() at amd64_syscall+0x4ce/frame 0xfffffe01137debf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe01137debf0 --- syscall (477, FreeBSD ELF64, sys_mmap), rip = 0x800613ffa, rsp = 0x7fffffffde68, rbp = 0x7fffffffdfc0 --- Tracing command sh pid 53957 tid 101116 td 0xfffff80162b5a500 cpustop_handler() at cpustop_handler+0x28/frame 0xfffffe00ea3fccf0 ipi_nmi_handler() at ipi_nmi_handler+0x4a/frame 0xfffffe00ea3fcd10 trap() at trap+0x3a/frame 0xfffffe00ea3fcf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xfffffe00ea3fcf20 --- trap 0x13, rip = 0xffffffff811a3b4b, rsp = 0xfffffe0113c9a850, rbp = 0xfffffe0113c9a890 --- pmap_try_insert_pv_entry() at pmap_try_insert_pv_entry+0x13b/frame 0xfffffe0113c9a890 pmap_copy() at pmap_copy+0x65c/frame 0xfffffe0113c9a930 vmspace_fork() at vmspace_fork+0x8d9/frame 0xfffffe0113c9a9b0 fork1() at fork1+0x5ec/frame 0xfffffe0113c9aa70 sys_fork() at sys_fork+0x39/frame 0xfffffe0113c9aad0 amd64_syscall() at amd64_syscall+0x4ce/frame 0xfffffe0113c9abf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0113c9abf0 --- syscall (2, FreeBSD ELF64, sys_fork), rip = 0x800b5d77a, rsp = 0x7fffffffe508, rbp = 0x7fffffffe540 --- Tracing command php-fpm pid 39824 tid 100686 td 0xfffff8003b242000 sched_switch() at sched_switch+0x6cb/frame 0xfffffe0113bb2850 mi_switch() at mi_switch+0xd2/frame 0xfffffe0113bb2880 sleepq_catch_signals() at sleepq_catch_signals+0xb7/frame 0xfffffe0113bb2900 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0113bb2930 _sleep() at _sleep+0x294/frame 0xfffffe0113bb29c0 kern_accept4() at kern_accept4+0x1f2/frame 0xfffffe0113bb2a70 accept1() at accept1+0x60/frame 0xfffffe0113bb2ad0 amd64_syscall() at amd64_syscall+0x4ce/frame 0xfffffe0113bb2bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0113bb2bf0 --- syscall (30, FreeBSD ELF64, sys_accept), rip = 0x801d947da, rsp = 0x7ffffffec848, rbp = 0x7ffffffec880 --- Tracing command top pid 43353 tid 100695 td 0xfffff80162d08a00 sched_switch() at sched_switch+0x6cb/frame 0xfffffe0113b35700 mi_switch() at mi_switch+0xd2/frame 0xfffffe0113b35730 sleepq_catch_signals() at sleepq_catch_signals+0xb7/frame 0xfffffe0113b357b0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0113b357e0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1c4/frame 0xfffffe0113b35850 seltdwait() at seltdwait+0xc7/frame 0xfffffe0113b358a0 kern_select() at kern_select+0x979/frame 0xfffffe0113b35a90 sys_select() at sys_select+0x54/frame 0xfffffe0113b35ad0 amd64_syscall() at amd64_syscall+0x4ce/frame 0xfffffe0113b35bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0113b35bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x80121000a, rsp = 0x7fffffffe888, rbp = 0x7fffffffea60 --- Tracing command bash pid 32987 tid 100680 td 0xfffff801628d1000 sched_switch() at sched_switch+0x6cb/frame 0xfffffe0113b946a0 mi_switch() at mi_switch+0xd2/frame 0xfffffe0113b946d0 sleepq_catch_signals() at sleepq_catch_signals+0xb7/frame 0xfffffe0113b94750 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0113b94780 _sleep() at _sleep+0x294/frame 0xfffffe0113b94810 kern_wait6() at kern_wait6+0x2d3/frame 0xfffffe0113b948e0 sys_wait4() at sys_wait4+0x73/frame 0xfffffe0113b94ad0 amd64_syscall() at amd64_syscall+0x4ce/frame 0xfffffe0113b94bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0113b94bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800e5d57a, rsp = 0x7fffffffe7a8, rbp = 0x7fffffffe7f0 --- Tracing command sh pid 2745 tid 100748 td 0xfffff8015e029a00 sched_switch() at sched_switch+0x6cb/frame 0xfffffe0113c366a0 mi_switch() at mi_switch+0xd2/frame 0xfffffe0113c366d0 sleepq_catch_signals() at sleepq_catch_signals+0xb7/frame 0xfffffe0113c36750 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0113c36780 _sleep() at _sleep+0x294/frame 0xfffffe0113c36810 kern_wait6() at kern_wait6+0x2d3/frame 0xfffffe0113c368e0 sys_wait4() at sys_wait4+0x73/frame 0xfffffe0113c36ad0 amd64_syscall() at amd64_syscall+0x4ce/frame 0xfffffe0113c36bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0113c36bf0 --- syscall (7, FreeBSD ELF64, sys_wait4), rip = 0x800b5d57a, rsp = 0x7fffffffe768, rbp = 0x7fffffffe820 --- Tracing command sshd pid 2354 tid 100699 td 0xfffff8001dd13000 sched_switch() at sched_switch+0x6cb/frame 0xfffffe0113bbc700 mi_switch() at mi_switch+0xd2/frame 0xfffffe0113bbc730 sleepq_catch_signals() at sleepq_catch_signals+0xb7/frame 0xfffffe0113bbc7b0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0113bbc7e0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1c4/frame 0xfffffe0113bbc850 seltdwait() at seltdwait+0xc7/frame 0xfffffe0113bbc8a0 kern_select() at kern_select+0x979/frame 0xfffffe0113bbca90 sys_select() at sys_select+0x54/frame 0xfffffe0113bbcad0 amd64_syscall() at amd64_syscall+0x4ce/frame 0xfffffe0113bbcbf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0113bbcbf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x801db400a, rsp = 0x7fffffffe5a8, rbp = 0x7fffffffe5e0 --- Tracing command php pid 21591 tid 100704 td 0xfffff8001d4b5500 sched_switch() at sched_switch+0x6cb/frame 0xfffffe0113bd58c0 mi_switch() at mi_switch+0xd2/frame 0xfffffe0113bd58f0 sleepq_catch_signals() at sleepq_catch_signals+0xb7/frame 0xfffffe0113bd5970 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0113bd59a0 _sleep() at _sleep+0x26f/frame 0xfffffe0113bd5a30 kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe0113bd5a90 sys_nanosleep() at sys_nanosleep+0x53/frame 0xfffffe0113bd5ad0 amd64_syscall() at amd64_syscall+0x4ce/frame 0xfffffe0113bd5bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0113bd5bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x801d8571a, rsp = 0x7fffffffc7e8, rbp = 0x7fffffffc810 --- Tracing command upsd pid 21318 tid 100739 td 0xfffff80162cd4500 sched_switch() at sched_switch+0x6cb/frame 0xfffffe0113c09770 mi_switch() at mi_switch+0xd2/frame 0xfffffe0113c097a0 sleepq_catch_signals() at sleepq_catch_signals+0xb7/frame 0xfffffe0113c09820 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0113c09850 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1c4/frame 0xfffffe0113c098c0 seltdwait() at seltdwait+0xc7/frame 0xfffffe0113c09910 kern_poll() at kern_poll+0x296/frame 0xfffffe0113c09ab0 sys_poll() at sys_poll+0x61/frame 0xfffffe0113c09ad0 amd64_syscall() at amd64_syscall+0x4ce/frame 0xfffffe0113c09bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0113c09bf0 --- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x800d4fd9a, rsp = 0x7fffffffe668, rbp = 0x7fffffffe6a0 --- Tracing command apcupsd-ups pid 21063 tid 100611 td 0xfffff8003be18500 sched_switch() at sched_switch+0x6cb/frame 0xfffffe0113aa9700 mi_switch() at mi_switch+0xd2/frame 0xfffffe0113aa9730 sleepq_catch_signals() at sleepq_catch_signals+0xb7/frame 0xfffffe0113aa97b0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0113aa97e0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1c4/frame 0xfffffe0113aa9850 seltdwait() at seltdwait+0xc7/frame 0xfffffe0113aa98a0 kern_select() at kern_select+0x979/frame 0xfffffe0113aa9a90 sys_select() at sys_select+0x54/frame 0xfffffe0113aa9ad0 amd64_syscall() at amd64_syscall+0x4ce/frame 0xfffffe0113aa9bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0113aa9bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800b9b00a, rsp = 0x7fffffffe678, rbp = 0x7fffffffe6b0 --- Tracing command upsmon pid 19999 tid 100703 td 0xfffff8001d4b5a00 sched_switch() at sched_switch+0x6cb/frame 0xfffffe0113bd08c0 mi_switch() at mi_switch+0xd2/frame 0xfffffe0113bd08f0 sleepq_catch_signals() at sleepq_catch_signals+0xb7/frame 0xfffffe0113bd0970 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0113bd09a0 _sleep() at _sleep+0x26f/frame 0xfffffe0113bd0a30 kern_nanosleep() at kern_nanosleep+0x10e/frame 0xfffffe0113bd0a90 sys_nanosleep() at sys_nanosleep+0x53/frame 0xfffffe0113bd0ad0 amd64_syscall() at amd64_syscall+0x4ce/frame 0xfffffe0113bd0bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0113bd0bf0 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip = 0x800d3c71a, rsp = 0x7fffffffe5c8, rbp = 0x7fffffffe5f0 --- Tracing command upsmon pid 19580 tid 100685 td 0xfffff80162d0a500 sched_switch() at sched_switch+0x6cb/frame 0xfffffe0113bad810 mi_switch() at mi_switch+0xd2/frame 0xfffffe0113bad840 sleepq_catch_signals() at sleepq_catch_signals+0xb7/frame 0xfffffe0113bad8c0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0113bad8f0 _sleep() at _sleep+0x294/frame 0xfffffe0113bad980 pipe_read() at pipe_read+0x36d/frame 0xfffffe0113bad9e0 dofileread() at dofileread+0x98/frame 0xfffffe0113bada30 kern_readv() at kern_readv+0x68/frame 0xfffffe0113bada80 sys_read() at sys_read+0x84/frame 0xfffffe0113badad0 amd64_syscall() at amd64_syscall+0x4ce/frame 0xfffffe0113badbf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0113badbf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800da502a, rsp = 0x7fffffffe628, rbp = 0x7fffffffe660 --- Tracing command sh pid 33060 tid 100578 td 0xfffff8001d4e5a00 sched_switch() at sched_switch+0x6cb/frame 0xfffffe0113a04810 mi_switch() at mi_switch+0xd2/frame 0xfffffe0113a04840 sleepq_catch_signals() at sleepq_catch_signals+0xb7/frame 0xfffffe0113a048c0 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0113a048f0 _sleep() at _sleep+0x294/frame 0xfffffe0113a04980 pipe_read() at pipe_read+0x36d/frame 0xfffffe0113a049e0 dofileread() at dofileread+0x98/frame 0xfffffe0113a04a30 kern_readv() at kern_readv+0x68/frame 0xfffffe0113a04a80 sys_read() at sys_read+0x84/frame 0xfffffe0113a04ad0 amd64_syscall() at amd64_syscall+0x4ce/frame 0xfffffe0113a04bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0113a04bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x800bc602a, rsp = 0x7fffffffe6a8, rbp = 0x7fffffffe880 --- Tracing command dhcpd pid 15682 tid 100713 td 0xfffff80162d08000 sched_switch() at sched_switch+0x6cb/frame 0xfffffe0113c04700 mi_switch() at mi_switch+0xd2/frame 0xfffffe0113c04730 sleepq_catch_signals() at sleepq_catch_signals+0xb7/frame 0xfffffe0113c047b0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x10/frame 0xfffffe0113c047e0 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x1c4/frame 0xfffffe0113c04850 seltdwait() at seltdwait+0xc7/frame 0xfffffe0113c048a0 kern_select() at kern_select+0x979/frame 0xfffffe0113c04a90 sys_select() at sys_select+0x54/frame 0xfffffe0113c04ad0 amd64_syscall() at amd64_syscall+0x4ce/frame 0xfffffe0113c04bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0113c04bf0 --- syscall (93, FreeBSD ELF64, sys_select), rip = 0x800d6400a, rsp = 0x7fffffffea38, rbp = 0x7fffffffea60 --- Tracing command unbound pid 88257 tid 100711 td 0xfffff80162cc2a00 sched_switch() at sched_switch+0x6cb/frame 0xfffffe0113bf8660 mi_switch() at mi_switch+0xd2/frame 0xfffffe0113bf8690 sleepq_catch_signals() at sleepq_catch_signals+0xb7/frame 0xfffffe0113bf8710 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0113bf8740 _sleep() at _sleep+0x294/frame 0xfffffe0113bf87d0 kqueue_kevent() at kqueue_kevent+0x336/frame 0xfffffe0113bf89b0 kern_kevent() at kern_kevent+0xb5/frame 0xfffffe0113bf8a20 sys_kevent() at sys_kevent+0x11c/frame 0xfffffe0113bf8ad0 amd64_syscall() at amd64_syscall+0x4ce/frame 0xfffffe0113bf8bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0113bf8bf0 --- syscall (363, FreeBSD ELF64, sys_kevent), rip = 0x80177275a, rsp = 0x7fffffffe9f8, rbp = 0x7fffffffea40 --- Tracing command unbound pid 88257 tid 100760 td 0xfffff80162da8a00 sched_switch() at sched_switch+0x6cb/frame 0xfffffe0113c40660 mi_switch() at mi_switch+0xd2/frame 0xfffffe0113c40690 sleepq_catch_signals() at sleepq_catch_signals+0xb7/frame 0xfffffe0113c40710 sleepq_wait_sig() at sleepq_wait_sig+0xf/frame 0xfffffe0113c40740 _sleep() at _sleep+0x294/frame 0xfffffe0113c407d0 kqueue_kevent() at kqueue_kevent+0x336/frame 0xfffffe0113c409b0 kern_kevent() at kern_kevent+0xb5/frame 0xfffffe0113c40a20 sys_kevent() at sys_kevent+0x11c/frame 0xfffffe0113c40ad0 amd64_syscall() at amd64_syscall+0x4ce/frame 0xfffffe0113c40bf0 Xfast_syscall() at Xfast_syscall+0config.txt0600001341413115636557 7552 ustarrootwheeloptions CONFIG_AUTOGENERATED ident pfSense machine amd64 cpu HAMMER makeoptions WITH_CTF=1 makeoptions DEBUG=-g options CONSPEED=115200 options MROUTING options HIFN_RNDTEST options HIFN_DEBUG options MSGTQL=2048 options MSGSSZ=32 options MSGSEG=512 options MSGMNI=40 options MSGMNB=8192 options ALTQ_CODEL options ALTQ_NOPCC options ALTQ_FAIRQ options ALTQ_PRIQ options ALTQ_HFSC options ALTQ_RIO options ALTQ_RED options ALTQ_CBQ options ALTQ options TCP_SIGNATURE options IPSEC_NAT_T options NETGRAPH_PRED1 options NETGRAPH_DEFLATE options NETGRAPH_CAR options NETGRAPH_PIPE options NETGRAPH_TCPMSS options NETGRAPH_TEE options NETGRAPH_HOLE options NETGRAPH_FRAME_RELAY options NETGRAPH_ASYNC options NETGRAPH_ECHO options NETGRAPH_CISCO options NETGRAPH_BRIDGE options NETGRAPH_ONE2MANY options NETGRAPH_LMI options NETGRAPH_KSOCKET options NETGRAPH_VJC options NETGRAPH_UI options NETGRAPH_MPPC_ENCRYPTION options NETGRAPH_TTY options NETGRAPH_SOCKET options NETGRAPH_RFC1490 options NETGRAPH_PPTPGRE options NETGRAPH_PPPOE options NETGRAPH_PPP options NETGRAPH_EIFACE options NETGRAPH_IFACE options NETGRAPH_ETHER options NETGRAPH_BPF options NETGRAPH_L2TP options NETGRAPH_VLAN options NETGRAPH options IPSTEALTH options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT options PPS_SYNC options NULLFS options UNIONFS options TMPFS options GEOM_BDE options GEOM_ELI options GEOM_UZIP options GEOM_MIRROR options DDB options XENHVM options ATH_ENABLE_11N options AH_AR5416_INTERRUPT_MITIGATION options AH_SUPPORT_AR5416 options IEEE80211_SUPPORT_MESH options IEEE80211_AMPDU_AGE options IEEE80211_DEBUG options SC_PIXEL_MODE options VESA options AHD_REG_PRETTY_PRINT options AHC_REG_PRETTY_PRINT options PCI_IOV options PCI_HP options ACPI_DMAR options DEVICE_NUMA options SMP options KDB options RCTL options RACCT_DEFAULT_TO_DISABLED options RACCT options INCLUDE_CONFIG_FILE options DDB_CTF options KDTRACE_HOOKS options KDTRACE_FRAME options CAPABILITIES options CAPABILITY_MODE options AUDIT options HWPMC_HOOKS options KBD_INSTALL_CDEV options PRINTF_BUFR_SIZE=128 options _KPOSIX_PRIORITY_SCHEDULING options SYSVSEM options SYSVMSG options SYSVSHM options STACK options KTRACE options SCSI_DELAY=5000 options COMPAT_FREEBSD10 options COMPAT_FREEBSD9 options COMPAT_FREEBSD32 options GEOM_LABEL options GEOM_RAID options GEOM_PART_GPT options PSEUDOFS options PROCFS options CD9660 options MSDOSFS options NFS_ROOT options NFSLOCKD options NFSD options NFSCL options MD_ROOT options QUOTA options UFS_GJOURNAL options UFS_DIRHASH options UFS_ACL options SOFTUPDATES options FFS options SCTP options TCP_OFFLOAD options IPSEC options INET6 options INET options PREEMPTION options SCHED_ULE options NEW_PCIB options GEOM_PART_MBR options GEOM_PART_EBR_COMPAT options GEOM_PART_EBR options GEOM_PART_BSD device isa device mem device io device uart_ns8250 device cpufreq device acpi device pci device fdc device ahci device ata device mvs device siis device ahc device ahd device esp device hptiop device isp device mpt device mps device mpr device sym device trm device adv device adw device aic device bt device isci device scbus device ch device da device sa device cd device pass device ses device amr device arcmsr device ciss device dpt device hptmv device hptnr device hptrr device hpt27xx device iir device ips device mly device twa device tws device aac device aacp device aacraid device ida device mfi device mlx device mrsas device pmspcv device twe device nvme device nvd device atkbdc device atkbd device psm device kbdmux device vga device splash device sc device vt device vt_vga device vt_efifb device agp device cbb device pccard device cardbus device uart device ppc device ppbus device lpt device ppi device puc device bxe device de device em device igb device ix device ixv device ixl device ixlv device le device ti device txp device vx device miibus device ae device age device alc device ale device bce device bfe device bge device cas device dc device et device fxp device gem device hme device jme device lge device msk device nfe device nge device pcn device re device rl device sf device sge device sis device sk device ste device stge device tl device tx device vge device vr device wb device xl device wlan device wlan_wep device wlan_ccmp device wlan_tkip device wlan_amrr device an device ath device ath_pci device ath_hal device ath_rate_sample device ipw device iwi device iwn device malo device mwl device ral device wi device wpi device loop device random device padlock_rng device rdrand_rng device ether device vlan device tun device md device gif device firmware device bpf device uhci device ohci device ehci device xhci device usb device ukbd device umass device sound device snd_cmi device snd_csa device snd_emu10kx device snd_es137x device snd_hda device snd_ich device snd_via8233 device mmc device mmcsd device sdhci device virtio device virtio_pci device vtnet device virtio_blk device virtio_scsi device virtio_balloon device hyperv device xenpci device vmx device netmap device crypto device wlan_rssadapt device wlan_xauth device wlan_acl device iwifw device ipwfw device wpifw device iwnfw device uath device ralfw device ural device urtw device rum device mwlfw device zyd device upgt device udav device axe device axge device aue device cue device kue device mos device rsu device rsufw device run device runfw device rue device rtwn device rtwnfw device siba_bwn device bwn device bwi device ufoma device ucom device uslcom device uplcom device umct device uvisor device uark device uftdi device uvscom device umodem device u3g device cdce device uhid device firewire device sbp device tap device gre device if_bridge device carp device lagg device vte device enc device pf device pflog device pfsync device rndtest device hifn device ubsec device safe device padlock device speaker device mxge device cxgb device cxgbe device oce msgbuf.txt06000017726213115636557 7664 ustarrootwheelCopyright (c) 1992-2016 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-RELEASE-p8 #78 3f54ce3814d(RELENG_2_4): Wed Mar 15 18:23:59 CDT 2017 root@buildbot2.netgate.com:/builder/ce/tmp/obj/builder/ce/tmp/FreeBSD-src/sys/pfSense amd64 FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM 3.8.0) VT(efifb): resolution 1920x1080 CPU: Intel(R) Celeron(R) CPU N3150 @ 1.60GHz (1600.05-MHz K8-class CPU) Origin="GenuineIntel" Id=0x406c3 Family=0x6 Model=0x4c Stepping=3 Features=0xbfebfbff <fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,htt,tm,pbe>Features2=0x43d8e3bf <sse3,pclmulqdq,dtes64,mon,ds_cpl,vmx,est,tm2,ssse3,cx16,xtpr,pdcm,sse4.1,sse4.2,movbe,popcnt,tscdlt,aesni,rdrand>AMD Features=0x28100800 <syscall,nx,rdtscp,lm>AMD Features2=0x101 <lahf,prefetch>Structured Extended Features=0x2282 <tscadj,smep,erms,nfpusg>VT-x: (disabled in BIOS) PAT,HLT,MTF,PAUSE,EPT,UG,VPID TSC: P-state invariant, performance statistics real memory = 4294967296 (4096 MB) avail memory = 3877691392 (3698 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: <alaska a="" m="" i="">WARNING: L1 data cache covers less APIC IDs than a core 0 < 1 FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) random: unblocking device. ACPI BIOS Warning (bug): 32/64X length mismatch in FADT/Gpe0Block: 128/32 (20160527/tbfadt-650) ioapic0 <version 2.0="">irqs 0-114 on motherboard random: entropy device external interface wlan: mac acl policy registered kbd0 at kbdmux0 netmap: loaded module module_register_init: MOD_LOAD (vesa, 0xffffffff8122b980, 0) error 19 random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" cryptosoft0: <software crypto="">on motherboard padlock0: No ACE support. acpi0: <alaska a="" m="" i="">on motherboard acpi0: Power Button (fixed) unknown: I/O range not supported cpu0: <acpi cpu="">on acpi0 cpu1: <acpi cpu="">on acpi0 cpu2: <acpi cpu="">on acpi0 cpu3: <acpi cpu="">on acpi0 atrtc0: <at realtime="" clock="">port 0x70-0x77 on acpi0 atrtc0: Warning: Couldn't map I/O. Event timer "RTC" frequency 32768 Hz quality 0 attimer0: <at timer="">port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: <acpi host-pci="" bridge="">port 0xcf8-0xcff on acpi0 pci0: <acpi pci="" bus="">on pcib0 vgapci0: <vga-compatible display="">port 0xf000-0xf03f mem 0x80000000-0x80ffffff,0x90000000-0x9fffffff irq 16 at device 2.0 on pci0 vgapci0: Boot video device ahci0: <ahci sata="" controller="">port 0xf060-0xf07f mem 0x81511000-0x815117ff irq 19 at device 19.0 on pci0 ahci0: AHCI v1.31 with 2 6Gbps ports, Port Multiplier not supported ahcich0: <ahci channel="">at channel 0 on ahci0 ahcich1: <ahci channel="">at channel 1 on ahci0 xhci0: <xhci (generic)="" usb="" 3.0="" controller="">mem 0x81500000-0x8150ffff irq 20 at device 20.0 on pci0 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 pci0: <encrypt decrypt="">at device 26.0 (no driver attached) pcib1: <acpi pci-pci="" bridge="">irq 16 at device 28.0 on pci0 pci1: <acpi pci="" bus="">on pcib1 re0: <realtek 8111="" 8168="" b="" c="" cp="" d="" dp="" e="" f="" g="" pcie="" gigabit="" ethernet="">port 0xe000-0xe0ff mem 0xfffe04000-0xfffe04fff,0xfffe00000-0xfffe03fff irq 16 at device 0.0 on pci1 re0: Using 1 MSI-X message re0: ASPM disabled re0: Chip rev. 0x2c800000 re0: MAC rev. 0x00100000 miibus0: <mii bus="">on re0 rgephy0: <rtl8169s 8211="" 8110s="" 1000base-t="" media="" interface="">PHY 1 on miibus0 rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow re0: Using defaults for TSO: 65518/35/2048 re0: Ethernet address: 00:e0:4c:68:55:ae re0: netmap queues/slots: TX 1/256, RX 1/256 pcib2: <acpi pci-pci="" bridge="">irq 17 at device 28.1 on pci0 pcib2: [GIANT-LOCKED] pci2: <acpi pci="" bus="">on pcib2 re1: <realtek 8111="" 8168="" b="" c="" cp="" d="" dp="" e="" f="" g="" pcie="" gigabit="" ethernet="">port 0xd000-0xd0ff mem 0xfffd04000-0xfffd04fff,0xfffd00000-0xfffd03fff irq 17 at device 0.0 on pci2 re1: Using 1 MSI-X message re1: ASPM disabled re1: Chip rev. 0x2c800000 re1: MAC rev. 0x00100000 miibus1: <mii bus="">on re1 rgephy1: <rtl8169s 8211="" 8110s="" 1000base-t="" media="" interface="">PHY 1 on miibus1 rgephy1: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow re1: Using defaults for TSO: 65518/35/2048 re1: Ethernet address: 00:e0:4c:68:55:af re1: netmap queues/slots: TX 1/256, RX 1/256 pcib3: <acpi pci-pci="" bridge="">irq 18 at device 28.2 on pci0 pci3: <acpi pci="" bus="">on pcib3 igb0: <intel(r) 1000="" pro="" network="" connection,="" version="" -="" 2.5.3-k="">port 0xc020-0xc03f mem 0x81380000-0x813fffff,0x81404000-0x81407fff irq 18 at device 0.0 on pci3 igb0: Using MSIX interrupts with 2 vectors igb0: Ethernet address: 00:30:18:c3:d0:e5 igb0: netmap queues/slots: TX 1/4096, RX 1/4096 igb1: <intel(r) 1000="" pro="" network="" connection,="" version="" -="" 2.5.3-k="">port 0xc000-0xc01f mem 0x81280000-0x812fffff,0x81400000-0x81403fff irq 19 at device 0.1 on pci3 igb1: Using MSIX interrupts with 2 vectors igb1: Ethernet address: 00:30:18:c3:d0:e6 igb1: netmap queues/slots: TX 1/4096, RX 1/4096 isab0: <pci-isa bridge="">at device 31.0 on pci0 isa0: <isa bus="">on isab0 acpi_button0: <sleep button="">on acpi0 acpi_tz0: <thermal zone="">on acpi0 ppc0: cannot reserve I/O port range est0: <enhanced speedstep="" frequency="" control="">on cpu0 est1: <enhanced speedstep="" frequency="" control="">on cpu1 est2: <enhanced speedstep="" frequency="" control="">on cpu2 est3: <enhanced speedstep="" frequency="" control="">on cpu3 usbus0: 5.0Gbps Super Speed USB v3.0 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 1.000 msec nvme cam probe device init ugen0.1: <0x8086> at usbus0 uhub0: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 uhub0: 13 ports with 13 removable, self powered ugen0.2: <american power="" conversion="">at usbus0 ugen0.3: <vendor 0x05e3="">at usbus0 uhub1: <vendor 2="" 9="" 0x05e3="" usb2.0="" hub,="" class="" 0,="" rev="" 2.00="" 85.36,="" addr="">on usbus0 uhub1: 4 ports with 4 removable, self powered ugen0.4: <vendor 0x1997="">at usbus0 ukbd0: <vendor 0="" 3="" 0x1997="" mini="" keyboard,="" class="" 0,="" rev="" 1.10="" 20.12,="" addr="">on usbus0 kbd1 at ukbd0 ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: <kingston sms200s360g="" 608abbf0="">ATA8-ACS SATA 3.x device ada0: Serial Number 50026B72630613B0 ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes) ada0: Command Queueing enabled ada0: 57241MB (117231408 512 byte sectors) SMP: AP CPU #2 Launched! SMP: AP CPU #1 Launched! SMP: AP CPU #3 Launched! Timecounter "TSC" frequency 1600052960 Hz quality 1000 Trying to mount root from zfs:zroot/ROOT/default []... <118>Configuring crash dumps... <118>Using /dev/ada0p3 for dump device. <118>Filesystems are clean, continuing... <118>Mounting filesystems... <118>Restoring contents of RAM disk store... done. <118> <118> ___ <118> ___/ f \ <118>/ p \___/ Sense <118>\___/ \ <118> \___/ <118> <118>Welcome to pfSense 2.4.0-BETA... <118> <118>No core dumps found. <118>chmod: /var/empty: Operation not permitted <118>...ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/dnscrypt-proxy /usr/local/lib/graphviz /usr/local/lib/ipsec /usr/local/lib/mysql /usr/local/lib/perl5/5.24/mach/CORE <118>32-bit compatibility ldconfig path: <118>done. <118>External config loader 1.0 is now starting... <118>Launching the init system....... done. <118>Initializing.................. done. <118>Starting device manager (devd)...kldload: can't load ums: No such file or directory <118>done. <118>Loading configuration......done. <118>tar: Removing leading '/' from member names <118>Updating configuration...done. <118>Cleaning backup cache.................................done. <118>Setting up extended sysctls...done. padlock0: No ACE support. aesni0: <aes-cbc,aes-xts,aes-gcm,aes-icm>on motherboard coretemp0: <cpu on-die="" thermal="" sensors="">on cpu0 coretemp1: <cpu on-die="" thermal="" sensors="">on cpu1 coretemp2: <cpu on-die="" thermal="" sensors="">on cpu2 coretemp3: <cpu on-die="" thermal="" sensors="">on cpu3 <118>Setting timezone...done. <118>Configuring loopback interface...done. <118>Starting syslog...done. <118>Starting Secure Shell Services...done. <118>Setting up interfaces microcode...done. <118>Configuring loopback interface...done. <118>Creating wireless clone interfaces...done. <118>Configuring LAGG interfaces...done. <118>Configuring VLAN interfaces...done. <118>Configuring QinQ interfaces...done. <118>Configuring WAN interface... <5>igb0: link state changed to UP <118>done. <118>Configuring LAN interface...done. <118>Configuring CARP settings...done. <118>Syncing OpenVPN settings... <6>tun1: changing name to 'ovpnc1' <6>tun2: changing name to 'ovpnc2' <118>done. <6>pflog0: promiscuous mode enabled <118>Configuring firewall.....0 addresses deleted. <118>0 addresses deleted. <118>.done. <118>Starting PFLOG...done. <118>Setting up gateway monitors...done. <5>ovpnc2: link state changed to UP <5>ovpnc1: link state changed to UP <5>igb1: link state changed to UP <118>Starting DNS Resolver...done. <118>Synchronizing user settings...done. <118>Starting webConfigurator...done. <118>Configuring CRON...done. <118>Starting NTP time client...done. <118>Starting DHCP service...done. <118>Starting DHCPv6 service...done. <118>Configuring firewall.....0 addresses deleted. <118>0 addresses deleted. <118>0 addresses deleted. <118>0 addresses deleted. <118>.done. <5>igb0: link state changed to DOWN <5>igb1: link state changed to DOWN <5>igb0: link state changed to UP <5>igb1: link state changed to UP <118>Generating RRD graphs...done. <118>Starting syslog...done. <118>Starting CRON... done. <118> Starting package Cron...done. <118> Starting package iftop...done. <118> Starting package iperf...done. <118> Starting package mtr-nox11...done. <118> Starting package nmap...done. <118> Starting package pfBlockerNG...done. <118> Starting package darkstat...done. <118> Starting package ntopng...done. <118> Starting package Shellcmd...done. <118> Starting package System Patches...done. <118> Starting package Backup...done. <118> Starting package apcupsd...done. <118> Starting package nut...done. <118> Starting /usr/local/etc/rc.d/dnsbl.sh...done. <118>pfSense 2.4.0-BETA amd64 Wed Mar 15 18:17:17 CDT 2017 <118>Bootup complete <5>ovpnc1: link state changed to DOWN <5>ovpnc2: link state changed to DOWN <5>ovpnc2: link state changed to UP <5>ovpnc1: link state changed to UP <5>re0: link state changed to DOWN <5>re1: link state changed to DOWN <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200a, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:401f:15::6, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4009:4c::a, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4009:4c::a, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4009:4c::a, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a01:578:3::36c2:327f, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:809::200a, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:816::200a, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200a, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:401f:31::6, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a01:578:3::3433:8b11, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:815::200a, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80a::200a, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:400c:c0a::5f, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:401f:15::6, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a01:578:3::3410:e6ed, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:809::200a, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:809::200a, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:811::200a, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:401f:15::6, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a01:578:3::34d2:92a8, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80a::200a, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0 <7>cannot forward src fe80:4::c6e9:84ff:fe59:770, dst 2a00:1450:4007:80c::200e, nxt 6, rcvif igb1, outif igb0</cpu></cpu></cpu></cpu></aes-cbc,aes-xts,aes-gcm,aes-icm></kingston></vendor></vendor></vendor></vendor></american></enhanced></enhanced></enhanced></enhanced></thermal></sleep></isa></pci-isa></intel(r)></intel(r)></acpi></acpi></rtl8169s></mii></realtek></acpi></acpi></rtl8169s></mii></realtek></acpi></acpi></encrypt></xhci></ahci></ahci></ahci></vga-compatible></acpi></acpi></at></at></acpi></acpi></acpi></acpi></alaska></software></version></alaska></tscadj,smep,erms,nfpusg></lahf,prefetch></syscall,nx,rdtscp,lm></sse3,pclmulqdq,dtes64,mon,ds_cpl,vmx,est,tm2,ssse3,cx16,xtpr,pdcm,sse4.1,sse4.2,movbe,popcnt,tscdlt,aesni,rdrand></fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,htt,tm,pbe> -
and last bit
kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 02 fault virtual address = 0x30 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80cb910c stack pointer = 0x28:0xfffffe0113c6d450 frame pointer = 0x28:0xfffffe0113c6d480 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 54120 (sh) version.txt06000025413115636557 7630 ustarrootwheelFreeBSD 11.0-RELEASE-p8 #78 3f54ce3814d(RELENG_2_4): Wed Mar 15 18:23:59 CDT 2017 root@buildbot2.netgate.com:/builder/ce/tmp/obj/builder/ce/tmp/FreeBSD-src/sys/pfSense -
another forum with same backtrace actually has led me to this unresolved bug report on FreeBSD, has been confirmed to affect 11 as well as 12.
So I think this might be what I have
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=213903
Have compiled the pfsense kernel now with the patch, will report back here if all good or not, only just booted into it so need to give it some time.
Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.