2 Crashes this week after years of stability.

mifronte

After years of stability my pfSense bare hardware using the Supermicro C2758 (A1SRi-2758F) crashes twice this week. The first time I was running V2.4.4 and the second time V2.4.5p1. The only thing I can make out in the debug file is at the end where it indicates a possible hardware failure.

This C2758 Supermicro board is prior to the LPC hardware bug fix and is wondering if this is the case or something else. Looks like in both crashes, pfSense rebooted.

Hopefully someone who can better decipher the crash report and give me more insights. Will be looking to get new hardware in the meantime.

pfSense Crash 20200619.tar.0

JKnott

@mifronte

There could be a lot of reasons for a crash. An intermittent failure on the mom board, bad memory or even a build up of dust. One thing you can do is run MemTest for several hours. I have it installed on a USB stick, but many Linux distros include it in the install media.

jimp

It's almost certainly hardware. since it's an NMI message

<2>NMI ISA 60, EISA 0
<2>I/O channel check, likely hardware failure.
panic: NMI indicates hardware failure
cpuid = 4
KDB: enter: panic

mifronte

It is now constantly crashing. Will swap the unit out to figure out how to do testing on the headless unit. It has IPMI so hopefully I can connect a LAN cable to the IPMI port directly to my laptop.

Too bad the crash report doesn't say which part of the hardware. Will try to blow the dirt out first.

jimp

You might check in the BIOS and see if it has any kind of event log, I can't remember what that particular model might have had.

The OS doesn't get a lot to go on from those, it's the BIOS passing the data on, so it can only print what it has to work with.

mifronte

Can't even get it to boot to get into the BIOS. I have a feeling the AVR54 hardware bug of the Supermicro C2758 board finally surfaced. The IPMI can just power it on and off, but beyond that, I don't even get the Supermicro Logo screen to enter the BIOS.

Gertjan

Definitely a hardware issue.
If needed, we can send flowers ;)

mifronte

Requested RMA from SuperMicro and they sent a replacement motherboard. Swapped out the motherboards and pfSense booted up with no issues. Whew! That was a close call.

Guess it took longer for the AVR54 hardware bug to surface in my usage environment.

mifronte

I need to change my signature to show the latest release that I am running. A quick search shows that I now may need 5 thumbs up to change my signature.

So would 5 people be so kind as to give me 5 thumbs up for getting my pfSense back up and running?

Thank you in advance.