4. Pfsense 2.3.1_1 crashing multiple times per day after upgrade from 2.3

Pfsense 2.3.1_1 crashing multiple times per day after upgrade from 2.3

  • R

    Hi there,
    after upgrading a 2.3 installation to 2.3.1_1, it started to crash randomly multiple times per day. Sometimes just once, sometimes no crash, but it is definitely not behaving properly. The firewall is running as a vm installed on a DELL server Poweredge R420 running VMware ESXi 5.5
    Here is the crash log, stating that "Unfortunately a Programming Bug has been detected". Sadly I cannot make much sense out of it and any help would be appreciated.
    Thanks a lot!

    Crash report begins.  Anonymous machine information:

    amd64
    10.3-RELEASE-p3
    FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016    root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense

    Crash report details:

    Filename: /var/crash/bounds
    54

    Filename: /var/crash/info.0
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Fri May 27 12:20:23 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 2750293089
      Bounds: 0
      Dump Status: good

    Filename: /var/crash/info.1
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Fri May 27 12:38:02 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 1324557409
      Bounds: 1
      Dump Status: good

    Filename: /var/crash/info.10
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sat May 28 00:46:50 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 2652711777
      Bounds: 10
      Dump Status: good

    Filename: /var/crash/info.11
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sat May 28 04:29:48 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 1490168673
      Bounds: 11
      Dump Status: good

    Filename: /var/crash/info.12
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sat May 28 07:23:49 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 2717526881
      Bounds: 12
      Dump Status: good

    Filename: /var/crash/info.13
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sat May 28 11:29:34 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 984886113
      Bounds: 13
      Dump Status: good

    Filename: /var/crash/info.14
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sat May 28 13:21:49 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 2303601249
      Bounds: 14
      Dump Status: good

    Filename: /var/crash/info.15
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sat May 28 14:43:59 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 1264724577
      Bounds: 15
      Dump Status: good

    Filename: /var/crash/info.16
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sat May 28 16:51:25 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 2843683425
      Bounds: 16
      Dump Status: good

    Filename: /var/crash/info.17
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sat May 28 20:48:37 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 3241618017
      Bounds: 17
      Dump Status: good

    Filename: /var/crash/info.18
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sat May 28 22:23:51 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 2479696481
      Bounds: 18
      Dump Status: good

    Filename: /var/crash/info.19
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sun May 29 00:34:53 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 3387697761
      Bounds: 19
      Dump Status: good

    Filename: /var/crash/info.2
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Fri May 27 14:49:12 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 2358322273
      Bounds: 2
      Dump Status: good

    Filename: /var/crash/info.20
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sun May 29 00:46:34 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 250554977
      Bounds: 20
      Dump Status: good

    Filename: /var/crash/info.21
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sun May 29 01:11:04 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 1291135585
      Bounds: 21
      Dump Status: good

    Filename: /var/crash/info.22
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sun May 29 01:30:07 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 1006185057
      Bounds: 22
      Dump Status: good

    Filename: /var/crash/info.23
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sun May 29 01:53:37 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 1174350433
      Bounds: 23
      Dump Status: good

    Filename: /var/crash/info.24
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sun May 29 04:24:37 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 564275809
      Bounds: 24
      Dump Status: good

    Filename: /var/crash/info.25
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sun May 29 05:27:03 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 1135684193
      Bounds: 25
      Dump Status: good

    Filename: /var/crash/info.26
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sun May 29 05:48:36 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 1354115681
      Bounds: 26
      Dump Status: good

    Filename: /var/crash/info.27
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sun May 29 07:27:02 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 1649224033
      Bounds: 27
      Dump Status: good

    Filename: /var/crash/info.28
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sun May 29 08:57:44 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 744630625
      Bounds: 28
      Dump Status: good

    Filename: /var/crash/info.29
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sun May 29 09:49:51 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 3681429857
      Bounds: 29
      Dump Status: good

    Filename: /var/crash/info.3
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Fri May 27 15:12:30 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 2056725601
      Bounds: 3
      Dump Status: good

    Filename: /var/crash/info.30
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sun May 29 11:09:49 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 419506529
      Bounds: 30
      Dump Status: good

    Filename: /var/crash/info.31
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sun May 29 12:08:32 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 1678715233
      Bounds: 31
      Dump Status: good

    Filename: /var/crash/info.32
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sun May 29 12:54:55 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 3138988385
      Bounds: 32
      Dump Status: good

    Filename: /var/crash/info.33
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sun May 29 13:18:30 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 3256822113
      Bounds: 33
      Dump Status: good

    Filename: /var/crash/info.34
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sun May 29 13:45:20 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 2485463393
      Bounds: 34
      Dump Status: good

    Filename: /var/crash/info.35
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sun May 29 15:17:18 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 440084833
      Bounds: 35
      Dump Status: good

    Filename: /var/crash/info.36
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sun May 29 15:56:28 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 3368298849
      Bounds: 36
      Dump Status: good

    Filename: /var/crash/info.37
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sun May 29 19:30:39 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 4227213665
      Bounds: 37
      Dump Status: good

    Filename: /var/crash/info.38
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sun May 29 21:34:26 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 2257791329
      Bounds: 38
      Dump Status: good

    Filename: /var/crash/info.39
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Sun May 29 23:30:35 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 3215862113
      Bounds: 39
      Dump Status: good

    Filename: /var/crash/info.4
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Fri May 27 15:23:07 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 4288226401
      Bounds: 4
      Dump Status: good

    Filename: /var/crash/info.40
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Mon May 30 00:08:03 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 2276927841
      Bounds: 40
      Dump Status: good

    Filename: /var/crash/info.41
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Mon May 30 02:29:49 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 3109562465
      Bounds: 41
      Dump Status: good

    Filename: /var/crash/info.42
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Mon May 30 03:24:07 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 4083492961
      Bounds: 42
      Dump Status: good

    Filename: /var/crash/info.43
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Mon May 30 05:27:27 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 3674351713
      Bounds: 43
      Dump Status: good

    Filename: /var/crash/info.44
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Mon May 30 06:31:06 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 3457230945
      Bounds: 44
      Dump Status: good

    Filename: /var/crash/info.45
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Mon May 30 06:57:48 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 2283219041
      Bounds: 45
      Dump Status: good

    Filename: /var/crash/info.46
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Mon May 30 07:40:01 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 4112590945
      Bounds: 46
      Dump Status: good

    Filename: /var/crash/info.47
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Mon May 30 08:58:30 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 1379149921
      Bounds: 47
      Dump Status: good

    Filename: /var/crash/info.48
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Mon May 30 11:34:49 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 4258801761
      Bounds: 48
      Dump Status: good

    Filename: /var/crash/info.49
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Mon May 30 11:50:48 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 1021061217
      Bounds: 49
      Dump Status: good

    Filename: /var/crash/info.5
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Fri May 27 16:33:13 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 1839866977
      Bounds: 5
      Dump Status: good

    Filename: /var/crash/info.50
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Mon May 30 12:24:10 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 249833569
      Bounds: 50
      Dump Status: good

    Filename: /var/crash/info.51
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Mon May 30 13:40:04 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 1358309473
      Bounds: 51
      Dump Status: good

    Filename: /var/crash/info.52
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Mon May 30 15:47:42 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 3130206305
      Bounds: 52
      Dump Status: good

    Filename: /var/crash/info.53
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Mon May 30 16:17:12 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 2895784033
      Bounds: 53
      Dump Status: good

    Filename: /var/crash/info.6
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Fri May 27 17:04:53 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 3518047329
      Bounds: 6
      Dump Status: good

    Filename: /var/crash/info.7
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Fri May 27 21:05:16 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 2825464673
      Bounds: 7
      Dump Status: good

    Filename: /var/crash/info.8
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Fri May 27 21:32:59 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 795880289
      Bounds: 8
      Dump Status: good

    Filename: /var/crash/info.9
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Fri May 27 22:51:16 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 2147625825
      Bounds: 9
      Dump Status: good

    Filename: /var/crash/info.last
    Dump header from device /dev/da0s1b
      Architecture: amd64
      Architecture Version: 1
      Dump Length: 156160B (0 MB)
      Blocksize: 512
      Dumptime: Mon May 30 16:17:12 2016
      Hostname: pfsenseait.ait-sydney.net.au
      Magic: FreeBSD Text Dump
      Version String: FreeBSD 10.3-RELEASE-p3 #2 1988fec(RELENG_2_3_1): Wed May 25 14:14:46 CDT 2016
        root@ce23-amd64-builder:/builder/pfsense-231/tmp/obj/builder/pfsense-231/tmp/FreeBSD-src/sys/pfSense
      Panic String:
      Dump Parity: 2895784033
      Bounds: 53
      Dump Status: good

    Filename: /var/crash/textdump.tar.0
    ddb.txt06000014000012721727347  7106 ustarrootwheeldb:0:kdb.enter.default>  run lockinfo
    db:1:lockinfo> show locks
    No such command
    db:1:locks>  show alllocks
    No such command
    db:1:alllocks>  show lockedvnods
    Locked vnodes
    db:0:kdb.enter.default>  show pcpu
    cpuid        = 0
    dynamic pcpu = 0x531500
    curthread    = 0xfffff800034d8960: pid 12 "irq16: le2 le5"
    curpcb      = 0xfffffe0000266b80
    fpcurthread  = none
    idlethread  = 0xfffff80003221000: tid 100003 "idle: cpu0"
    curpmap      = 0xffffffff820f85b0
    tssp        = 0xffffffff82113490
    commontssp  = 0xffffffff82113490
    rsp0        = 0xfffffe0000266b80
    gs32p        = 0xffffffff82114ee8
    ldt          = 0xffffffff82114f28
    tss          = 0xffffffff82114f18
    db:0:kdb.enter.default>  bt
    Tracing pid 12 tid 100034 td 0xfffff800034d8960
    pppoe_findsession() at pppoe_findsession+0x80/frame 0xfffffe00002666e0
    ng_pppoe_rcvdata_ether() at ng_pppoe_rcvdata_ether+0x294/frame 0xfffffe0000266790
    ng_apply_item() at ng_apply_item+0x22c/frame 0xfffffe0000266820
    ng_snd_item() at ng_snd_item+0x1a3/frame 0xfffffe0000266860
    ether_demux() at ether_demux+0x1ef/frame 0xfffffe0000266890
    ether_nh_input() at ether_nh_input+0x347/frame 0xfffffe00002668f0
    netisr_dispatch_src() at netisr_dispatch_src+0x62/frame 0xfffffe0000266960
    am79900_intr() at am79900_intr+0x2b3/frame 0xfffffe00002669e0
    intr_event_execute_handlers() at intr_event_execute_handlers+0xab/frame 0xfffffe0000266a20
    ithread_loop() at ithread_loop+0x96/frame 0xfffffe0000266a70
    fork_exit() at fork_exit+0x9a/frame 0xfffffe0000266ab0
    fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0000266ab0
    –- trap 0, rip = 0, rsp = 0, rbp = 0 ---
    db:0:kdb.enter.default>  ps
      pid  ppid  pgrp  uid  state  wmesg        wchan        cmd
    14521  267  267    0  S      accept  0xfffff80003b8e066 php-fpm
    95097  267  267    0  S      nanslp  0xffffffff82001170 php-fpm
    19024  1797    22    0  S      nanslp  0xffffffff82001171 sleep
    74432    1 74432    0  Ss      (threaded)                  filterdns
    100175                  S      uwait    0xfffff800142c2f00 signal-thread
    100174                  S      uwait    0xfffff80014546000 184.73.193.59
    100173                  S      uwait    0xfffff8007a1c5780 64.4.37.223
    100172                  S      uwait    0xfffff8007a1c5680 208.184.35.32
    100171                  S      uwait    0xfffff800142cd080 208.184.35.0
    100170                  S      uwait    0xfffff800142c7880 208.184.35.64
    100169                  S      uwait    0xfffff800142c6f00 208.184.35.96
    100168                  S      uwait    0xfffff80014b5ac80 108.160.160.0
    100167                  S      uwait    0xfffff8007a1c6400 199.47.216.0
    100166                  S      uwait    0xfffff8007a1c6300 205.189.0.0
    100165                  S      kqread  0xfffff8007a055000 muddled.net
    100164                  S      uwait    0xfffff8007a1c6100 vtr.net
    100163                  S      uwait    0xfffff8007a1c6000 hkcable.com.hk
    100162                  S      kqread  0xfffff8001485ac00 anteldata.net.uy
    100161                  S      uwait    0xfffff80014544f00 Quadranet.com
    100160                  S      uwait    0xfffff800142c7100 comcast.net
    100159                  S      kqread  0xfffff80014857700 worldofwarcraft.com
    100158                  S      kqread  0xfffff8007afb4200 filterdns
    100157                  S      uwait    0xfffff800142c7080 demonii.com
    100156                  S      uwait    0xfffff800142c7000 openbittorrent.com
    100155                  S      uwait    0xfffff800142c7680 filterdns
    100154                  S      uwait    0xfffff800142d5000 bitgravity.com
    100153                  S      uwait    0xfffff800142c7700 openx.org
    100152                  S      uwait    0xfffff800142c7780 fbcdn.net
    100151                  S      uwait    0xfffff800142c7500 dropbox.com
    100150                  S      uwait    0xfffff800142c6a80 facebook.com
    100149                  S      uwait    0xfffff800142c7580 feeds.bbci.co.uk
    100148                  S      kqread  0xfffff8001485a400 bankings.net
    100147                  S      kqread  0xfffff80003a49700 publicbt.com
    100146                  S      uwait    0xfffff800142d1080 demonii.com
    100145                  S      uwait    0xfffff800142dff00 openbittorrent.com
    68784 68636 68784    0  S+      ttyin    0xfffff8000353bca8 sh
    68636 67992 68636    0  S+      wait    0xfffff8001417d4f0 sh
    68483 51444 68483    0  Ss      (threaded)                  sshlockout_pf
    100144                  S      nanslp  0xffffffff82001170 sshlockout_pf
    100143                  S      piperd  0xfffff80014e78ba0 sshlockout_pf
    67992    1 67992    0  Ss+    wait    0xfffff80003a809e0 login
    55283 54946 54946    0  S      nanslp  0xffffffff82001170 minicron
    54946    1 54946    0  Ss      wait    0xfffff8001417c000 minicron
    54658 54336 54336    0  S      nanslp  0xffffffff82001171 minicron
    54336    1 54336    0  Ss      wait    0xfffff80014d5f4f0 minicron
    54181 53689 53689    0  S      nanslp  0xffffffff82001171 minicron
    53689    1 53689    0  Ss      wait    0xfffff800141dd4f0 minicron
    51444    1 51444    0  Ss      select  0xfffff80003a1f7c0 syslogd
    1797    1    22    0  S+      wait    0xfffff800141de4f0 sh
    96635    1 96635    0  Ss      select  0xfffff80014c42d40 bsnmpd
    93008    1 93008    0  Ss      select  0xfffff800145441c0 dhcrelay
    89384    1 89329 65534  S      select  0xfffff800145440c0 dnsmasq
    87688    1 87688    0  Ss      (threaded)                  ntpd
    100121                  S      select  0xfffff80014544c40 ntpd
    86583    1 86583    0  Ss      nanslp  0xffffffff82001170 cron
    85993 85703 85703    0  S      kqread  0xfffff8001459c600 nginx
    85980 85703 85703    0  S      kqread  0xfffff8001459d000 nginx
    85834 85703 85703    0  S      kqread  0xfffff8001459cd00 nginx
    85703    1 85703    0  Ss      pause    0xfffff800145ada88 nginx
    41537    1 41537    0  Ss      (threaded)                  dpinger
    100114                  S      accept  0xfffff800142b588e dpinger
    100113                  S      nanslp  0xffffffff82001171 dpinger
    100112                  S      nanslp  0xffffffff82001170 dpinger
    100111                  S      sbwait  0xfffff800142b5c24 dpinger
    100070                  S      uwait    0xfffff80003a20a00 dpinger
    41383    1 41383    0  Ss      (threaded)                  dpinger
    100110                  S      accept  0xfffff800142b631e dpinger
    100109                  S      nanslp  0xffffffff82001170 dpinger
    100108                  S      nanslp  0xffffffff82001171 dpinger
    100107                  S      sbwait  0xfffff800142b66b4 dpinger
    100089                  S      uwait    0xfffff800140feb80 dpinger
    40744    1 40744    0  Ss      (threaded)                  dpinger
    100106                  S      accept  0xfffff800140f3066 dpinger
    100105                  S      nanslp  0xffffffff82001171 dpinger
    100104                  S      nanslp  0xffffffff82001170 dpinger
    100103                  S      sbwait  0xfffff800140f2c24 dpinger
    100098                  S      uwait    0xfffff80003559e80 dpinger
    38087    1 38087    0  Ss      select  0xfffff80014107940 xinetd
    36987    1 36987    0  Ss      bpf      0xfffff800141e4600 filterlog
    35522    1 35522    0  Ss      select  0xfffff80014107d40 openvpn
    33106    1 33106    0  Ss      select  0xfffff800140fddc0 openvpn
    30687    1 30687    0  Ss      select  0xfffff800140fe240 openvpn
    27621    1 27621    0  Ss      select  0xfffff8001413f9c0 openvpn
    25896    1 25896    0  Ss      select  0xfffff8001413fb40 openvpn
    13537    1 13537    0  Ss      (threaded)                  mpd5
    100081                  S      select  0xfffff800039edf40 mpd5
    8814    1  8814    0  Ss      (threaded)                  sshlockout_pf
    100079                  S      nanslp  0xffffffff82001171 sshlockout_pf
    100077                  S      uwait    0xfffff800039efb80 sshlockout_pf
    8540    1  8540    0  Ss      select  0xfffff80003a1e2c0 sshd
    7032    1  7032    0  Ss      (threaded)                  mpd5
    100078                  S      select  0xfffff80003a1f5c0 mpd5
      302    1  302    0  Ss      select  0xfffff800039ef140 devd
      291  289  289    0  S      kqread  0xfffff80003a0dc00 check_reload_status
      289    1  289    0  Ss      kqread  0xfffff80003a0d600 check_reload_status
      267    1  267    0  Ss      kqread  0xfffff80003a49800 php-fpm
      52    0    0    0  DL      mdwait  0xfffff800039d3000 [md0]
      21    0    0    0  DL      vlruwt  0xfffff800039f7000 [vnlru]
      20    0    0    0  DL      syncer  0xffffffff82052108 [syncer]
      19    0    0    0  DL      psleep  0xffffffff82051304 [bufdaemon]
      18    0    0    0  DL      pollid  0xffffffff81fffa90 [idlepoll]
      17    0    0    0  DL      pgzero  0xffffffff8206243c [pagezero]
      16    0    0    0  DL      psleep  0xffffffff820617c0 [vmdaemon]
        9    0    0    0  DL      (threaded)                  [pagedaemon]
    100054                  D      umarcl  0xffffffff82061140 [uma]
    100046                  D      psleep  0xffffffff82112804 [pagedaemon]
        8    0    0    0  DL      waiting_ 0xffffffff821032c0 [sctp_iterator]
        7    0    0    0  DL      pftm    0xffffffff80d5d630 [pf purge]
        6    0    0    0  DL      -        0xfffff80003546648 [fdc0]
        5    0    0    0  DL      idle    0xfffffe0000946000 [mpt_recovery0]
        4    0    0    0  DL      (threaded)                  [cam]
    100045                  D      -        0xffffffff81f35cc8 [scanner]
    100019                  D      -        0xffffffff81f35e80 [doneq0]
      15    0    0    0  DL      -        0xffffffff81f575c0 [rand_harvestq]
        3    0    0    0  DL      crypto_r 0xffffffff8205fc98 [crypto returns]
        2    0    0    0  DL      crypto_w 0xffffffff8205fb40 [crypto]
      14    0    0    0  DL      (threaded)                  [geom]
    100013                  D      -        0xffffffff820f79f8 [g_down]
    100012                  D      -        0xffffffff820f79f0 [g_up]
    100011                  D      -        0xffffffff820f79e8 [g_event]
      13    0    0    0  DL      (threaded)                  [ng_queue]
    100010                  D      sleep    0xffffffff81ef4318 [ng_queue1]
    100009                  D      sleep    0xffffffff81ef4318 [ng_queue0]
      12    0    0    0  RL      (threaded)                  [intr]
    100053                  I                                  [swi1: netisr 1]
    100043                  I                                  [swi1: pfsync]
    100041                  I                                  [swi1: pf send]
    100038                  I                                  [swi0: uart uart]
    100037                  I                                  [irq7: ppc0]
    100036                  I                                  [irq12: psm0]
    100035                  I                                  [irq1: atkbd0]
    100034                  Run    CPU 0                      [irq16: le2 le5]
    100033                  I                                  [irq19: le1 le4]
    100032                  RunQ                                [irq18: le0]
    100030                  I                                  [irq17: le3 mpt0]
    100029                  I                                  [irq15: ata1]
    100028                  I                                  [irq14: ata0]
    100027                  I                                  [swi5: fast taskq]
    100025                  I                                  [swi6: Giant taskq]
    100023                  I                                  [swi6: task queue]
    100008                  RunQ                                [swi1: netisr 0]
    100007                  I                                  [swi3: vm]
    100006                  I                                  [swi4: clock]
    100005                  I                                  [swi4: clock]
      11    0    0    0  RL      (threaded)                  [idle]
    100004                  Run    CPU 1                      [idle: cpu1]
    100003                  CanRun                              [idle: cpu0]
        1    0    1    0  SLs    wait    0xfffff8000321e4f0 [init]
      10    0    0    0  DL      audit_wo 0xffffffff82109d60 [audit]
        0    0    0    0  DLs    (threaded)                  [kernel]
    100102                  D      -        0xfffff80014227e00 [dummynet]
    100044                  D      -        0xfffff800032d6e00 [CAM taskq]
    100026                  D      -        0xfffff800032d5d00

    0
  • S

    I've started to have this same issue and I think the "programming bug detected" started happening at the same time as yours.

    Did you manage to work out what the issue was? It happens at really random times… sometimes it can goes weeks without issue then experience 2 or 3 of these reboots in 15 minutes!

    I'm guessing it could be a hardware fault though finding the root cause would be helpful.

    Thanks
    Tim

    0
Log in to reply
 

Products

Services

Support

News

Resources

Company

Our Mission

We provide leading-edge network security at a fair price - regardless of organizational size or network sophistication. We believe that an open-source security model offers disruptive pricing along with the agility required to quickly address emerging threats.

Subscribe to our Newsletter

Product information, software announcements, and special offers. See our newsletter archive for past announcements.

© Copyright 2019 Rubicon Communications, LLC | Privacy Policy