Diagnostics: Crash reporter



  • Hi People,

    I am having crash reports often, as there is ongoing crash.
    I did submit it once (first time).

    So my question is how can I get results and resolution.

    Thanks you all in advance.



  • They only get looked at individually if the user also contacts ESF for support or posts here with detail of what they were doing and can point to their crash report (e.g. IP address it would have come from)…

    Are they "real" crashes - i.e. FreeBSD kernel panic with traceback sort of thing?
    Or is it a PHP "crash" - just some error being reported in "user code"?

    Describe your system, what you are doing when the crash happens (if anything), and something of what appears in the crash report you are submitting.



  • Hi,

    Well I will give you all details:

    2.1.5-RELEASE (amd64)
    built on Mon Aug 25 07:44:45 EDT 2014
    FreeBSD 8.3-RELEASE-p16

    We are using it for some "Free WiFi" project, so we have custom landing page, which are working with radius and captive portal etc.
    As I am not developer I can't give details right now, but it started after our developers added some video ad. I mean "like before logging in user must watch video then skip it…" So surely PHP crash.

    I do detect it by monitoring interfaces, so they are getting zero traffic.

    Also please see attachment for crash.

    crash_report_pfsense.txt



  • It is a real FreeBSD crash:

    Fatal trap 12: page fault while in kernel mode
    

    In theory adding a video to play should not be able to cause a kernel panic. Something is trying to use a buffer in the kernel that was not locked into memory (the disk driver?) There were lots of disk file system problems reported also, so something has happened to the file system/disk.

    Is there a reason you are still on 2.1.5?

    There won't be any more support/fixes for that from FreeBSD or pfSense - it is based on FreeBSD 8.3 and now pfSense 2.2.n is based on FreeBSD10.1



  • Well I will surely update it.
    Also I have to mention that I use CentOS KVM, so pfSense is VM.
    I had to force shutdown VM several times, so do you mean there might me some disk issue?

    Also one question: If I upgrade pfSense to 2.2.x, FreeBSD upgrades too, doesn't it?

    Thanks



  • <118>mount: 
    <118>/dev/ad0s1a R/W mount of / denied. Filesystem is not clean - run fsck.
    <118>: 
    <118>Operation not permitted
    <118>** /dev/ad0s1a
    <118>** Last Mounted on /
    <118>** Root file system
    <118>** Phase 1 - Check Blocks and Sizes
    <118>INCORRECT BLOCK COUNT I=194392 (36 should be 0)
    <118>CORRECT? yes
    <118>
    <118>INCORRECT BLOCK COUNT I=196243 (2048 should be 0)
    <118>CORRECT? yes
    <118>
    <118>INCORRECT BLOCK COUNT I=196338 (8 should be 0)
    <118>CORRECT? yes
    
    

    All that fsck output should not be happening. But I guess if you had to force shutdown the VM then there would be some disk file system things reported. I am just surprised how much of it there is.

    If I upgrade pfSense to 2.2.x, FreeBSD upgrades too, doesn't it?

    Yes, the pfSense upgrade installs the new FreeBSD, all needed utilities (PHP…) and the pfSense code all in 1 go.

    Being in a VM you can snapshot it and upgrade with low risk.



  • Surely I will take full backup before upgrading :).
    But firstly I would like to fix this "crash issue".

    Do you recommend to run fsck (well… in "safe" mode) ?

    Thanks.



  • <118>81079 files, 1277833 used, 751162 free (12450 frags, 92339 blocks, 0.6% fragmentation)
    <118>
    <118>***** FILE SYSTEM MARKED CLEAN *****
    <118>
    <118>***** FILE SYSTEM WAS MODIFIED *****
    
    

    That indicates that fsck was run by the boot script and thinks it has fixed everything. I would just do a clean boot and confirm that you do not get all that fsck output before the "Welcome to pfSense" any more.

    As far as fixing goes, on pfSense 2.1.* FreeBSD 8.3 all you can really do is remove whatever changes were made when it started crashing, and confirm that it does not crash any more. Then you will know the userland changes that are inducing this crash.

    To really fix anything you would then have to upgrade to a current pfSense/FreeBSD release, confirm that an ordinary setup does not crash, then make the changes that were causing it to crash, see what happens and continue from there.



  • Well, you made good point.

    Thank you very much for support.