Pfsense 2.3.1_1 crashing multiple times per day after upgrade from 2.3
-
Hi there,
after upgrading a 2.3 installation to 2.3.1_1, it started to crash randomly multiple times per day. Sometimes just once, sometimes no crash, but it is definitely not behaving properly. The firewall is running as a vm installed on a DELL server Poweredge R420 running VMware ESXi 5.5
Here is the crash log, stating that "Unfortunately a Programming Bug has been detected". Sadly I cannot make much sense out of it and any help would be appreciated.
Thanks a lot!Crash report begins. Anonymous machine information:
amd64
10.3-RELEASE-p3
FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016 root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSenseCrash report details:
Filename: /var/crash/bounds
54Filename: /var/crash/info.0
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Fri May 27 12:20:23 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 2750293089
Bounds: 0
Dump Status: goodFilename: /var/crash/info.1
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Fri May 27 12:38:02 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 1324557409
Bounds: 1
Dump Status: goodFilename: /var/crash/info.10
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sat May 28 00:46:50 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 2652711777
Bounds: 10
Dump Status: goodFilename: /var/crash/info.11
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sat May 28 04:29:48 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 1490168673
Bounds: 11
Dump Status: goodFilename: /var/crash/info.12
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sat May 28 07:23:49 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 2717526881
Bounds: 12
Dump Status: goodFilename: /var/crash/info.13
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sat May 28 11:29:34 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 984886113
Bounds: 13
Dump Status: goodFilename: /var/crash/info.14
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sat May 28 13:21:49 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 2303601249
Bounds: 14
Dump Status: goodFilename: /var/crash/info.15
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sat May 28 14:43:59 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 1264724577
Bounds: 15
Dump Status: goodFilename: /var/crash/info.16
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sat May 28 16:51:25 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 2843683425
Bounds: 16
Dump Status: goodFilename: /var/crash/info.17
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sat May 28 20:48:37 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 3241618017
Bounds: 17
Dump Status: goodFilename: /var/crash/info.18
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sat May 28 22:23:51 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 2479696481
Bounds: 18
Dump Status: goodFilename: /var/crash/info.19
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sun May 29 00:34:53 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 3387697761
Bounds: 19
Dump Status: goodFilename: /var/crash/info.2
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Fri May 27 14:49:12 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 2358322273
Bounds: 2
Dump Status: goodFilename: /var/crash/info.20
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sun May 29 00:46:34 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 250554977
Bounds: 20
Dump Status: goodFilename: /var/crash/info.21
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sun May 29 01:11:04 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 1291135585
Bounds: 21
Dump Status: goodFilename: /var/crash/info.22
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sun May 29 01:30:07 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 1006185057
Bounds: 22
Dump Status: goodFilename: /var/crash/info.23
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sun May 29 01:53:37 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 1174350433
Bounds: 23
Dump Status: goodFilename: /var/crash/info.24
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sun May 29 04:24:37 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 564275809
Bounds: 24
Dump Status: goodFilename: /var/crash/info.25
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sun May 29 05:27:03 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 1135684193
Bounds: 25
Dump Status: goodFilename: /var/crash/info.26
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sun May 29 05:48:36 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 1354115681
Bounds: 26
Dump Status: goodFilename: /var/crash/info.27
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sun May 29 07:27:02 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 1649224033
Bounds: 27
Dump Status: goodFilename: /var/crash/info.28
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sun May 29 08:57:44 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 744630625
Bounds: 28
Dump Status: goodFilename: /var/crash/info.29
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sun May 29 09:49:51 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 3681429857
Bounds: 29
Dump Status: goodFilename: /var/crash/info.3
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Fri May 27 15:12:30 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 2056725601
Bounds: 3
Dump Status: goodFilename: /var/crash/info.30
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sun May 29 11:09:49 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 419506529
Bounds: 30
Dump Status: goodFilename: /var/crash/info.31
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sun May 29 12:08:32 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 1678715233
Bounds: 31
Dump Status: goodFilename: /var/crash/info.32
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sun May 29 12:54:55 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 3138988385
Bounds: 32
Dump Status: goodFilename: /var/crash/info.33
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sun May 29 13:18:30 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 3256822113
Bounds: 33
Dump Status: goodFilename: /var/crash/info.34
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sun May 29 13:45:20 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 2485463393
Bounds: 34
Dump Status: goodFilename: /var/crash/info.35
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sun May 29 15:17:18 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 440084833
Bounds: 35
Dump Status: goodFilename: /var/crash/info.36
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sun May 29 15:56:28 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 3368298849
Bounds: 36
Dump Status: goodFilename: /var/crash/info.37
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sun May 29 19:30:39 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 4227213665
Bounds: 37
Dump Status: goodFilename: /var/crash/info.38
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sun May 29 21:34:26 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 2257791329
Bounds: 38
Dump Status: goodFilename: /var/crash/info.39
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Sun May 29 23:30:35 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 3215862113
Bounds: 39
Dump Status: goodFilename: /var/crash/info.4
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Fri May 27 15:23:07 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 4288226401
Bounds: 4
Dump Status: goodFilename: /var/crash/info.40
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Mon May 30 00:08:03 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 2276927841
Bounds: 40
Dump Status: goodFilename: /var/crash/info.41
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Mon May 30 02:29:49 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 3109562465
Bounds: 41
Dump Status: goodFilename: /var/crash/info.42
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Mon May 30 03:24:07 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 4083492961
Bounds: 42
Dump Status: goodFilename: /var/crash/info.43
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Mon May 30 05:27:27 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 3674351713
Bounds: 43
Dump Status: goodFilename: /var/crash/info.44
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Mon May 30 06:31:06 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 3457230945
Bounds: 44
Dump Status: goodFilename: /var/crash/info.45
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Mon May 30 06:57:48 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 2283219041
Bounds: 45
Dump Status: goodFilename: /var/crash/info.46
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Mon May 30 07:40:01 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 4112590945
Bounds: 46
Dump Status: goodFilename: /var/crash/info.47
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Mon May 30 08:58:30 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 1379149921
Bounds: 47
Dump Status: goodFilename: /var/crash/info.48
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Mon May 30 11:34:49 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 4258801761
Bounds: 48
Dump Status: goodFilename: /var/crash/info.49
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Mon May 30 11:50:48 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 1021061217
Bounds: 49
Dump Status: goodFilename: /var/crash/info.5
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Fri May 27 16:33:13 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 1839866977
Bounds: 5
Dump Status: goodFilename: /var/crash/info.50
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Mon May 30 12:24:10 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 249833569
Bounds: 50
Dump Status: goodFilename: /var/crash/info.51
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Mon May 30 13:40:04 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 1358309473
Bounds: 51
Dump Status: goodFilename: /var/crash/info.52
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Mon May 30 15:47:42 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 3130206305
Bounds: 52
Dump Status: goodFilename: /var/crash/info.53
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Mon May 30 16:17:12 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 2895784033
Bounds: 53
Dump Status: goodFilename: /var/crash/info.6
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Fri May 27 17:04:53 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 3518047329
Bounds: 6
Dump Status: goodFilename: /var/crash/info.7
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Fri May 27 21:05:16 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 2825464673
Bounds: 7
Dump Status: goodFilename: /var/crash/info.8
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Fri May 27 21:32:59 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 795880289
Bounds: 8
Dump Status: goodFilename: /var/crash/info.9
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Fri May 27 22:51:16 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 2147625825
Bounds: 9
Dump Status: goodFilename: /var/crash/info.last
Dump header from device /dev/da0s1b
Architecture: amd64
Architecture Version: 1
Dump Length: 156160B (0 MB)
Blocksize: 512
Dumptime: Mon May 30 16:17:12 2016
Hostname: pfsenseait.ait-sydney.net.au
Magic: FreeBSD Text Dump
Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
Panic String:
Dump Parity: 2895784033
Bounds: 53
Dump Status: goodFilename: /var/crash/textdump.tar.0
ddb.txt06000014000012721727347 7106 ustarrootwheeldb:0:kdb.enter.default> run lockinfo
db:1:lockinfo> show locks
No such command
db:1:locks> show alllocks
No such command
db:1:alllocks> show lockedvnods
Locked vnodes
db:0:kdb.enter.default> show pcpu
cpuid = 0
dynamic pcpu = 0x531500
curthread = 0xfffff800034d8960: pid 12 "irq16: le2 le5"
curpcb = 0xfffffe0000266b80
fpcurthread = none
idlethread = 0xfffff80003221000: tid 100003 "idle: cpu0"
curpmap = 0xffffffff820f85b0
tssp = 0xffffffff82113490
commontssp = 0xffffffff82113490
rsp0 = 0xfffffe0000266b80
gs32p = 0xffffffff82114ee8
ldt = 0xffffffff82114f28
tss = 0xffffffff82114f18
db:0:kdb.enter.default> bt
Tracing pid 12 tid 100034 td 0xfffff800034d8960
pppoe_findsession() at pppoe_findsession+0x80/frame 0xfffffe00002666e0
ng_pppoe_rcvdata_ether() at ng_pppoe_rcvdata_ether+0x294/frame 0xfffffe0000266790
ng_apply_item() at ng_apply_item+0x22c/frame 0xfffffe0000266820
ng_snd_item() at ng_snd_item+0x1a3/frame 0xfffffe0000266860
ether_demux() at ether_demux+0x1ef/frame 0xfffffe0000266890
ether_nh_input() at ether_nh_input+0x347/frame 0xfffffe00002668f0
netisr_dispatch_src() at netisr_dispatch_src+0x62/frame 0xfffffe0000266960
am79900_intr() at am79900_intr+0x2b3/frame 0xfffffe00002669e0
intr_event_execute_handlers() at intr_event_execute_handlers+0xab/frame 0xfffffe0000266a20
ithread_loop() at ithread_loop+0x96/frame 0xfffffe0000266a70
fork_exit() at fork_exit+0x9a/frame 0xfffffe0000266ab0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000266ab0
–- trap 0, rip = 0, rsp = 0, rbp = 0 ---
db:0:kdb.enter.default> ps
pid ppid pgrp uid state wmesg wchan cmd
14521 267 267 0 S accept 0xfffff80003b8e066 php-fpm
95097 267 267 0 S nanslp 0xffffffff82001170 php-fpm
19024 1797 22 0 S nanslp 0xffffffff82001171 sleep
74432 1 74432 0 Ss (threaded) filterdns
100175 S uwait 0xfffff800142c2f00 signal-thread
100174 S uwait 0xfffff80014546000 184.73.193.59
100173 S uwait 0xfffff8007a1c5780 64.4.37.223
100172 S uwait 0xfffff8007a1c5680 208.184.35.32
100171 S uwait 0xfffff800142cd080 208.184.35.0
100170 S uwait 0xfffff800142c7880 208.184.35.64
100169 S uwait 0xfffff800142c6f00 208.184.35.96
100168 S uwait 0xfffff80014b5ac80 108.160.160.0
100167 S uwait 0xfffff8007a1c6400 199.47.216.0
100166 S uwait 0xfffff8007a1c6300 205.189.0.0
100165 S kqread 0xfffff8007a055000 muddled.net
100164 S uwait 0xfffff8007a1c6100 vtr.net
100163 S uwait 0xfffff8007a1c6000 hkcable.com.hk
100162 S kqread 0xfffff8001485ac00 anteldata.net.uy
100161 S uwait 0xfffff80014544f00 Quadranet.com
100160 S uwait 0xfffff800142c7100 comcast.net
100159 S kqread 0xfffff80014857700 worldofwarcraft.com
100158 S kqread 0xfffff8007afb4200 filterdns
100157 S uwait 0xfffff800142c7080 demonii.com
100156 S uwait 0xfffff800142c7000 openbittorrent.com
100155 S uwait 0xfffff800142c7680 filterdns
100154 S uwait 0xfffff800142d5000 bitgravity.com
100153 S uwait 0xfffff800142c7700 openx.org
100152 S uwait 0xfffff800142c7780 fbcdn.net
100151 S uwait 0xfffff800142c7500 dropbox.com
100150 S uwait 0xfffff800142c6a80 facebook.com
100149 S uwait 0xfffff800142c7580 feeds.bbci.co.uk
100148 S kqread 0xfffff8001485a400 bankings.net
100147 S kqread 0xfffff80003a49700 publicbt.com
100146 S uwait 0xfffff800142d1080 demonii.com
100145 S uwait 0xfffff800142dff00 openbittorrent.com
68784 68636 68784 0 S+ ttyin 0xfffff8000353bca8 sh
68636 67992 68636 0 S+ wait 0xfffff8001417d4f0 sh
68483 51444 68483 0 Ss (threaded) sshlockout_pf
100144 S nanslp 0xffffffff82001170 sshlockout_pf
100143 S piperd 0xfffff80014e78ba0 sshlockout_pf
67992 1 67992 0 Ss+ wait 0xfffff80003a809e0 login
55283 54946 54946 0 S nanslp 0xffffffff82001170 minicron
54946 1 54946 0 Ss wait 0xfffff8001417c000 minicron
54658 54336 54336 0 S nanslp 0xffffffff82001171 minicron
54336 1 54336 0 Ss wait 0xfffff80014d5f4f0 minicron
54181 53689 53689 0 S nanslp 0xffffffff82001171 minicron
53689 1 53689 0 Ss wait 0xfffff800141dd4f0 minicron
51444 1 51444 0 Ss select 0xfffff80003a1f7c0 syslogd
1797 1 22 0 S+ wait 0xfffff800141de4f0 sh
96635 1 96635 0 Ss select 0xfffff80014c42d40 bsnmpd
93008 1 93008 0 Ss select 0xfffff800145441c0 dhcrelay
89384 1 89329 65534 S select 0xfffff800145440c0 dnsmasq
87688 1 87688 0 Ss (threaded) ntpd
100121 S select 0xfffff80014544c40 ntpd
86583 1 86583 0 Ss nanslp 0xffffffff82001170 cron
85993 85703 85703 0 S kqread 0xfffff8001459c600 nginx
85980 85703 85703 0 S kqread 0xfffff8001459d000 nginx
85834 85703 85703 0 S kqread 0xfffff8001459cd00 nginx
85703 1 85703 0 Ss pause 0xfffff800145ada88 nginx
41537 1 41537 0 Ss (threaded) dpinger
100114 S accept 0xfffff800142b588e dpinger
100113 S nanslp 0xffffffff82001171 dpinger
100112 S nanslp 0xffffffff82001170 dpinger
100111 S sbwait 0xfffff800142b5c24 dpinger
100070 S uwait 0xfffff80003a20a00 dpinger
41383 1 41383 0 Ss (threaded) dpinger
100110 S accept 0xfffff800142b631e dpinger
100109 S nanslp 0xffffffff82001170 dpinger
100108 S nanslp 0xffffffff82001171 dpinger
100107 S sbwait 0xfffff800142b66b4 dpinger
100089 S uwait 0xfffff800140feb80 dpinger
40744 1 40744 0 Ss (threaded) dpinger
100106 S accept 0xfffff800140f3066 dpinger
100105 S nanslp 0xffffffff82001171 dpinger
100104 S nanslp 0xffffffff82001170 dpinger
100103 S sbwait 0xfffff800140f2c24 dpinger
100098 S uwait 0xfffff80003559e80 dpinger
38087 1 38087 0 Ss select 0xfffff80014107940 xinetd
36987 1 36987 0 Ss bpf 0xfffff800141e4600 filterlog
35522 1 35522 0 Ss select 0xfffff80014107d40 openvpn
33106 1 33106 0 Ss select 0xfffff800140fddc0 openvpn
30687 1 30687 0 Ss select 0xfffff800140fe240 openvpn
27621 1 27621 0 Ss select 0xfffff8001413f9c0 openvpn
25896 1 25896 0 Ss select 0xfffff8001413fb40 openvpn
13537 1 13537 0 Ss (threaded) mpd5
100081 S select 0xfffff800039edf40 mpd5
8814 1 8814 0 Ss (threaded) sshlockout_pf
100079 S nanslp 0xffffffff82001171 sshlockout_pf
100077 S uwait 0xfffff800039efb80 sshlockout_pf
8540 1 8540 0 Ss select 0xfffff80003a1e2c0 sshd
7032 1 7032 0 Ss (threaded) mpd5
100078 S select 0xfffff80003a1f5c0 mpd5
302 1 302 0 Ss select 0xfffff800039ef140 devd
291 289 289 0 S kqread 0xfffff80003a0dc00 check_reload_status
289 1 289 0 Ss kqread 0xfffff80003a0d600 check_reload_status
267 1 267 0 Ss kqread 0xfffff80003a49800 php-fpm
52 0 0 0 DL mdwait 0xfffff800039d3000 [md0]
21 0 0 0 DL vlruwt 0xfffff800039f7000 [vnlru]
20 0 0 0 DL syncer 0xffffffff82052108 [syncer]
19 0 0 0 DL psleep 0xffffffff82051304 [bufdaemon]
18 0 0 0 DL pollid 0xffffffff81fffa90 [idlepoll]
17 0 0 0 DL pgzero 0xffffffff8206243c [pagezero]
16 0 0 0 DL psleep 0xffffffff820617c0 [vmdaemon]
9 0 0 0 DL (threaded) [pagedaemon]
100054 D umarcl 0xffffffff82061140 [uma]
100046 D psleep 0xffffffff82112804 [pagedaemon]
8 0 0 0 DL waiting_ 0xffffffff821032c0 [sctp_iterator]
7 0 0 0 DL pftm 0xffffffff80d5d630 [pf purge]
6 0 0 0 DL - 0xfffff80003546648 [fdc0]
5 0 0 0 DL idle 0xfffffe0000946000 [mpt_recovery0]
4 0 0 0 DL (threaded) [cam]
100045 D - 0xffffffff81f35cc8 [scanner]
100019 D - 0xffffffff81f35e80 [doneq0]
15 0 0 0 DL - 0xffffffff81f575c0 [rand_harvestq]
3 0 0 0 DL crypto_r 0xffffffff8205fc98 [crypto returns]
2 0 0 0 DL crypto_w 0xffffffff8205fb40 [crypto]
14 0 0 0 DL (threaded) [geom]
100013 D - 0xffffffff820f79f8 [g_down]
100012 D - 0xffffffff820f79f0 [g_up]
100011 D - 0xffffffff820f79e8 [g_event]
13 0 0 0 DL (threaded) [ng_queue]
100010 D sleep 0xffffffff81ef4318 [ng_queue1]
100009 D sleep 0xffffffff81ef4318 [ng_queue0]
12 0 0 0 RL (threaded) [intr]
100053 I [swi1: netisr 1]
100043 I [swi1: pfsync]
100041 I [swi1: pf send]
100038 I [swi0: uart uart]
100037 I [irq7: ppc0]
100036 I [irq12: psm0]
100035 I [irq1: atkbd0]
100034 Run CPU 0 [irq16: le2 le5]
100033 I [irq19: le1 le4]
100032 RunQ [irq18: le0]
100030 I [irq17: le3 mpt0]
100029 I [irq15: ata1]
100028 I [irq14: ata0]
100027 I [swi5: fast taskq]
100025 I [swi6: Giant taskq]
100023 I [swi6: task queue]
100008 RunQ [swi1: netisr 0]
100007 I [swi3: vm]
100006 I [swi4: clock]
100005 I [swi4: clock]
11 0 0 0 RL (threaded) [idle]
100004 Run CPU 1 [idle: cpu1]
100003 CanRun [idle: cpu0]
1 0 1 0 SLs wait 0xfffff8000321e4f0 [init]
10 0 0 0 DL audit_wo 0xffffffff82109d60 [audit]
0 0 0 0 DLs (threaded) [kernel]
100102 D - 0xfffff80014227e00 [dummynet]
100044 D - 0xfffff800032d6e00 [CAM taskq]
100026 D - 0xfffff800032d5d00 -
I've started to have this same issue and I think the "programming bug detected" started happening at the same time as yours.
Did you manage to work out what the issue was? It happens at really random times… sometimes it can goes weeks without issue then experience 2 or 3 of these reboots in 15 minutes!
I'm guessing it could be a hardware fault though finding the root cause would be helpful.
Thanks
Tim