Firewall crashing twice a day

rfmperkins

I was hope to get some help I have been running this firewall configuration for a few weeks a couple of days ago it seems the firewall crashes one to two times a day anyone have an ideas?
Seems to be random times, I am using igc3 for the WAN (1G) and ix0 for the LAN (10G) interfaces.

Hardware in use
Debug_Output.txt

stephenw10

Backtraces:

db:1:pfs> bt
Tracing pid 7 tid 100122 td 0xfffff800033be000
kdb_enter() at kdb_enter+0x33/frame 0xfffffe0103492c90
panic() at panic+0x43/frame 0xfffffe0103492cf0
trap_fatal() at trap_fatal+0x40b/frame 0xfffffe0103492d50
calltrap() at calltrap+0x8/frame 0xfffffe0103492d50
--- trap 0x9, rip = 0xffffffff80fcce9b, rsp = 0xfffffe0103492e20, rbp = 0xfffffe0103492e40 ---
pf_state_expires() at pf_state_expires+0xb/frame 0xfffffe0103492e40
pf_purge_expired_states() at pf_purge_expired_states+0xd8/frame 0xfffffe0103492e90
pf_purge_thread() at pf_purge_thread+0x15b/frame 0xfffffe0103492ef0
fork_exit() at fork_exit+0x7b/frame 0xfffffe0103492f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0103492f30
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---

db:1:pfs> bt
Tracing pid 0 tid 112088 td 0xfffff80684050740
kdb_enter() at kdb_enter+0x33/frame 0xfffffe0103a87b10
panic() at panic+0x43/frame 0xfffffe0103a87b70
trap_fatal() at trap_fatal+0x40b/frame 0xfffffe0103a87bd0
calltrap() at calltrap+0x8/frame 0xfffffe0103a87bd0
--- trap 0x9, rip = 0xffffffff80fffeed, rsp = 0xfffffe0103a87ca0, rbp = 0xfffffe0103a87d30 ---
pf_handle_getstates() at pf_handle_getstates+0x1ed/frame 0xfffffe0103a87d30
nl_taskqueue_handler() at nl_taskqueue_handler+0x44e/frame 0xfffffe0103a87e40
taskqueue_run_locked() at taskqueue_run_locked+0x182/frame 0xfffffe0103a87ec0
taskqueue_thread_loop() at taskqueue_thread_loop+0xc2/frame 0xfffffe0103a87ef0
fork_exit() at fork_exit+0x7b/frame 0xfffffe0103a87f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0103a87f30
--- trap 0x16, rip = 0x82bda96ea, rsp = 0x857a8faf8, rbp = 0x857a8fb10 ---

db:1:pfs> bt
Tracing pid 19654 tid 100250 td 0xfffff80008c1a000
kdb_enter() at kdb_enter+0x33/frame 0xfffffe0103659980
panic() at panic+0x43/frame 0xfffffe01036599e0
trap_fatal() at trap_fatal+0x40b/frame 0xfffffe0103659a40
calltrap() at calltrap+0x8/frame 0xfffffe0103659a40
--- trap 0x9, rip = 0xffffffff811906a4, rsp = 0xfffffe0103659b10, rbp = 0xfffffe0103659b60 ---
vm_object_shadow() at vm_object_shadow+0x84/frame 0xfffffe0103659b60
vm_map_lookup() at vm_map_lookup+0x932/frame 0xfffffe0103659c40
vm_fault() at vm_fault+0xf8/frame 0xfffffe0103659d70
vm_fault_trap() at vm_fault_trap+0x4d/frame 0xfffffe0103659dc0
trap_pfault() at trap_pfault+0x1be/frame 0xfffffe0103659e10
trap() at trap+0x475/frame 0xfffffe0103659f30
calltrap() at calltrap+0x8/frame 0xfffffe0103659f30
--- trap 0xc, rip = 0x37fd2430f88f, rsp = 0x37fd207b9938, rbp = 0x37fd207b9a90 ---

db:1:pfs> bt
Tracing pid 99631 tid 100634 td 0xfffff80809a83000
kdb_enter() at kdb_enter+0x33/frame 0xfffffe01039f68f0
panic() at panic+0x43/frame 0xfffffe01039f6950
trap_fatal() at trap_fatal+0x40b/frame 0xfffffe01039f69b0
trap_pfault() at trap_pfault+0x46/frame 0xfffffe01039f6a00
calltrap() at calltrap+0x8/frame 0xfffffe01039f6a00
--- trap 0xc, rip = 0xffffffff812e5012, rsp = 0xfffffe01039f6ad0, rbp = 0xfffffe01039f6b00 ---
pmap_pv_promote_pde() at pmap_pv_promote_pde+0x172/frame 0xfffffe01039f6b00
pmap_promote_pde() at pmap_promote_pde+0x33a/frame 0xfffffe01039f6b80
pmap_enter() at pmap_enter+0xdfb/frame 0xfffffe01039f6c50
vm_fault() at vm_fault+0x1383/frame 0xfffffe01039f6d70
vm_fault_trap() at vm_fault_trap+0x4d/frame 0xfffffe01039f6dc0
trap_pfault() at trap_pfault+0x1be/frame 0xfffffe01039f6e10
trap() at trap+0x475/frame 0xfffffe01039f6f30
calltrap() at calltrap+0x8/frame 0xfffffe01039f6f30
--- trap 0xc, rip = 0x3d3e7c6e7e35, rsp = 0x3d3e773bda18, rbp = 0x3d3e773bda60 ---

Seeing different crashes like that usually points at a hardware issue.

What's the history of that device? Has it even been running stable?