Interpret crash report

[ocin28]

Hi! I'm a newbie on using pfsense. I would just like to know what this crash report means so that I can fix it and avoid it.

Crash report begins.  Anonymous machine information:

i386
8.3-RELEASE-p16
FreeBSD 8.3-RELEASE-p16 #0: Mon Aug 25 08:25:41 EDT 2014    root@pf2_1_1_i386.pfsense.org:/usr/obj.i386/usr/pfSensesrc/src/sys/pfSense_SMP.8

Crash report details:

Filename: /var/crash/bounds
1

Filename: /var/crash/info.0
Dump header from device /dev/ad4s1b
  Architecture: i386
  Architecture Version: 1
  Dump Length: 68608B (0 MB)
  Blocksize: 512
  Dumptime: Sun Nov  8 22:36:59 2015
  Hostname: barrianet-fw.barrianetcafe.net
  Magic: FreeBSD Text Dump
  Version String: FreeBSD 8.3-RELEASE-p16 #0: Mon Aug 25 08:25:41 EDT 2014
    root@pf2_1_1_i386.pfsense.org:/usr/obj.i386/usr/pfSensesrc/src/sys/pfSense_SMP.8
  Panic String:
  Dump Parity: 1883255903
  Bounds: 0
  Dump Status: good

Filename: /var/crash/minfree
2048

Filename: /var/crash/textdump.tar.0
ddb.txt06000014000012617657013  7102 ustarrootwheeldb:0:kdb.enter.default>  run lockinfo
db:1:lockinfo> show locks
No such command
db:1:locks>  show alllocks
No such command
db:1:alllocks>  show lockedvnods
Locked vnodes
db:0:kdb.enter.default>  show pcpu
cpuid        = 0
dynamic pcpu = 0x306700
curthread    = 0xc6302b80: pid 19620 "netstat"
curpcb      = 0xecd29d80
fpcurthread  = none
idlethread  = 0xc49948a0: tid 100003 "idle: cpu0"
APIC ID      = 0
currentldt  = 0x50
db:0:kdb.enter.default>  bt
Tracing pid 19620 tid 100129 td 0xc6302b80
pmap_remove_pages(c4998d60,0,0,c6302b80,0,…) at pmap_remove_pages+0x10d
vmspace_exit(c6302b80,c52f92b0,ffffffff,0,c69f3d20,...) at vmspace_exit+0x94
exit1(c6302b80,0,ecd29d1c,c0e8ef2b,c6302b80,...) at exit1+0x3a2
sys_exit(c6302b80,ecd29cec,46,c0ac9aac,c6302b80,...) at sys_exit+0x1d
syscall(ecd29d28) at syscall+0x1fb
Xint0x80_syscall() at Xint0x80_syscall+0x21
--- syscall (1, FreeBSD ELF32, sys_exit), eip = 0x2814c3cf, esp = 0xbfbfed6c, ebp = 0xbfbfed78 ---
db:0:kdb.enter.default>  ps
  pid  ppid  pgrp  uid  state  wmesg    wchan    cmd
19781 19431 21465    0  S      piperd  0xc4da4930 awk
19622 19431 21465    0  R                          grep
19620 19431 21465    0  RE      CPU 0              netstat
19431 30778 21465    0  S      wait    0xc52a42b0 sh
12697 32428  261    0  S      nanslp  0xc15571c4 sleep
30778 21465 21465    0  S      (threaded)          php
100103                  S      piperd  0xc4da47a8 php
11024 20749 20749    0  S      accept  0xc51ed84a php
12902    1 12902  1002  Ss      select  0xc4bfe524 dhcpd
32731  3874  3399    62  S      sbwait  0xc68ac0b8 initial thread
32603  3874  3399    62  S      sbwait  0xc688c3f0 initial thread
32337  3874  3399    62  S      sbwait  0xc63a00b8 initial thread
32206  3874  3399    62  S      sbwait  0xc63a1a60 initial thread
32010  3874  3399    62  S      sbwait  0xc633358c initial thread
4072  3874  3399    62  S      piperd  0xc4da3188 unlinkd
3874  3399  3399    62  S      kqread  0xc5318d00 initial thread
3399    1  3399    0  Ss      wait    0xc51fa000 squid
87573    1 87573    0  Ss      (threaded)          ntpd
100120                  S      select  0xc6306ba4 ntpd
32428    1  261    0  S      wait    0xc52f9810 sh
15395    1 15395    65  Ss      select  0xc5348864 dhclient
8921    1  8921    0  Ss      select  0xc5347164 dhclient
26348    1 26142 65534  S      select  0xc6306964 dnsmasq
15398 14576 15398    0  S+      select  0xc5217024 top
14576 12997 14576    0  S+      pause    0xc52a3868 tcsh
12997 10872 12997    0  Ss+    wait    0xc52f8000 sh
10872  9792 10872    0  Ss      select  0xc53481e4 sshd
31741 31537 31537    0  S      piperd  0xc52f6000 rrdtool
31537    1 31537    0  Ss      select  0xc52161a4 apinger
61530 60582 61530    0  S+      ttyin    0xc4b04070 sh
60582 59667 60582    0  S+      wait    0xc51fb2b0 sh
60379 39637 60379    0  Ss      (threaded)          sshlockout_pf
100115                  S      nanslp  0xc15571c4 sshlockout_pf
100071                  S      piperd  0xc52f6310 initial thread
59667    1 59667    0  Ss+    wait    0xc52f8560 login
48634 48155 48155    0  S      nanslp  0xc15571c4 minicron
48155    1 48155    0  Ss      wait    0xc52a4000 minicron
48076 47786 47786    0  S      nanslp  0xc15571c4 minicron
47786    1 47786    0  Ss      wait    0xc51f9560 minicron
47661 47290 47290    0  S      nanslp  0xc15571c4 minicron
47290    1 47290    0  Ss      wait    0xc51f92b0 minicron
41772    1 41772    0  Ss      nanslp  0xc15571c4 cron
39637    1 39637    0  Ss      select  0xc5216ca4 syslogd
21465 20575 21465    0  Ss      wait    0xc4dadac0 initial thread
20749 20575 20749    0  Ss      wait    0xc4dae560 initial thread
20575    1 20496    0  S      kqread  0xc5289a00 lighttpd
14008    1 14008    0  Ss      select  0xc5215d64 inetd
13250    1    27    0  S+      piperd  0xc4da3930 logger
12966    1    27    0  S+      bpf      0xc5140d00 tcpdump
9952    1  9952    0  Ss      (threaded)          sshlockout_pf
100082                  S      nanslp  0xc15571c4 sshlockout_pf
100081                  S      uwait    0xc4feb2c0 initial thread
9792    1  9792    0  Ss      select  0xc4d307a4 sshd
  273    1  273    0  Ss      select  0xc4d30524 devd
  263  261  261    0  S      kqread  0xc4d80800 check_reload_status
  261    1  261    0  Ss      kqread  0xc4d80c80 check_reload_status
  69    0    0    0  SL      mdwait  0xc4d8c000 [md0]
  39    0    0    0  SL      (threaded)          zfskern
100069                  D      l2arc_fe 0xc4f25b04 [l2arc_feed_thread]
100068                  D      arc_recl 0xc4f1697c [arc_reclaim_thread]
  26    0    0    0  SL      sdflush  0xc1585b60 [softdepflush]
  25    0    0    0  SL      vlruwt  0xc4d4f2b0 [vnlru]
  24    0    0    0  SL      syncer  0xc156af38 [syncer]
  23    0    0    0  SL      psleep  0xc156ac68 [bufdaemon]
  22    0    0    0  SL      pollid  0xc15566fc [idlepoll]
  21    0    0    0  SL      pgzero  0xc1586814 [pagezero]
  20    0    0    0  SL      psleep  0xc158643c [vmdaemon]
  19    0    0    0  SL      psleep  0xc1586404 [pagedaemon]
  18    0    0    0  SL      ccb_scan 0xc151f5d4 [xpt_thrd]
    9    0    0    0  SL      pftm    0xc050c700 [pfpurge]
    8    0    0    0  SL      waiting_ 0xc1572338 [sctp_iterator]
    7    0    0    0  SL      -        0xc4a4fe3c [fdc0]
  17    0    0    0  SL      cooling  0xc4b2c4d4 [acpi_cooling0]
  16    0    0    0  SL      tzpoll  0xc152462c [acpi_thermal]
  15    0    0    0  SL      (threaded)          usb
100040                  D      -        0xc4b19d34 [usbus2]
100039                  D      -        0xc4b19d04 [usbus2]
100038                  D      -        0xc4b19cd4 [usbus2]
100037                  D      -        0xc4b19ca4 [usbus2]
100036                  D      -        0xc4b11b5c [usbus1]
100035                  D      -        0xc4b11b2c [usbus1]
100034                  D      -        0xc4b11afc [usbus1]
100033                  D      -        0xc4b11acc [usbus1]
100032                  D      -        0xc4b07b5c [usbus0]
100031                  D      -        0xc4b07b2c [usbus0]
100030                  D      -        0xc4b07afc [usbus0]
100029                  D      -        0xc4b07acc [usbus0]
  14    0    0    0  SL      -        0xc1557024 [yarrow]
    6    0    0    0  SL      crypto_r 0xc158510c [crypto returns]
    5    0    0    0  SL      crypto_w 0xc15850e8 [crypto]
    4    0    0    0  SL      -        0xc15547e4 [g_down]
    3    0    0    0  SL      -        0xc15547e0 [g_up]
    2    0    0    0  SL      -        0xc15547d8 [g_event]
  13    0    0    0  SL      sleep    0xc14f2fa0 [ng_queue0]
  12    0    0    0  WL      (threaded)          intr
100050                  I                          [irq1: atkbd0]
100049                  I                          [irq7: ppc0]
100048                  I                          [swi0: uart]
100044                  I                          [irq17: fxp0]
100043                  I                          [irq21: rl0]
100042                  I                          [irq15: ata1]
100041                  I                          [irq14: ata0]
100028                  I                          [irq19: ohci0 ohci1+]
100027                  I                          [irq22: atapci1]
100026                  I                          [irq23: atapci0]
100025                  I                          [irq9: acpi0]
100024                  I                          [swi6: Giant taskq]
100022                  I                          [swi5: +]
100019                  I                          [swi2: cambio]
100015                  I                          [swi6: task queue]
100006                  I                          [swi3: vm]
100005                  I                          [swi4: clock]
100004                  I                          [swi1: netisr 0]
  11    0    0    0  RL                          [idle: cpu0]
    1    0    1    0  SLs    wait    0xc4992ac0 [init]
  10    0    0    0  SL      audit_wo 0xc1585500 [audit]
    0    0    0    0  SLs    (threaded)          kernel
100130                  D      -        0xc6450740 [dummynet]
100067                  D      -        0xc4d77480 [system_taskq]
100023                  D      -        0xc4aa3440

[ocin28]

Please can someone enlighten me on how to deal with it. Thanks.

[David_W]

Upgrade to pfSense 2.2.5, which was released in the last few days. If you have 64 bit capable hardware, switch to the amd64 architecture.

You're running 2.1.1, which is based around much older underpinnings than 2.2.x and isn't even the last version in the End of Life 2.1.x branch. Realistically, there is little interest in crash reports from an old release. (There are some narrow usage cases that require sticking with 2.1.5, such as the problems with limiters in 2.2.x, but unless you know you need to stick with 2.1.x, leave it behind.)

If I'm reading the crash report correctly, the crash happened when netstat was trying to exit in a call that was removing pages from the paging table. It's possible you have unreliable memory on this box that led to paging table corruption, but it might just be one of those things.

[divsys]

If I'm reading the crash report correctly, the crash happened when netstat was trying to exit in a call that was removing pages from the paging table. It's possible you have unreliable memory on this box that led to paging table corruption, but it might just be one of those things.

Even if it is a hardware issue (hard to even guess from 1 crash report and no description of the physical setup) I'd definitely support the suggestion to get up to the latest version.

Unless you're running a whole raft of packages (no mention so unknown?) it should be as simple as saving the config from your current box then run the auto-update process.
If you post your hardware and packages we can give you a heasd up on potential issues.

So far 2.2.5 has been very stable.

[ocin28]

Thanks for the suggestions guys. I've been dealing with this crash report for weeks now. I've already tried the latest version but had bad luck on implementing squid + squid guard. It seems that if I'm using the latest version which is 2.2.5, only https websites are accessible if transparent proxy is enabled.

I'm also suspecting this is a hardware issue cause I'm using an old P4 machine with 512x2 ddr1 ram and 80gb of hard drive dedicated to pfsense.

I'll try once agaain the latest version if this crash will disappear. Also, is there anyway to know if my processor supports 64bit version without opening the case. Like CPU-Z on windows. Thank you.