Storage detached, semi-catastrophic failure
-
After a reboot, the drives were up and running again like nothing had happened, a scrub showed no inconsistencies.
-
Yes, hardware, firmware or low level OS related at least. The system is not old, we built it in the beginning of August so not more than 4½ months. Our theories so far goes:
-
Controller failure: Not very likely, they are on the chipset controller (Intel C256) and we do not expect any stability issues with it
-
Drive failure:As a reboot fixed it, both drives were healthy and zfs did not find any inconsistencies after a scrub, we don't suspect drive hardware failure. Maybe some oddity with the firmware, like a counter overflowing or similar, but again, these are enterprise PM893:s, we don't expect it
-
OS: Given the info from the two posts I linked, and due to the two drives going down at the same time, this is perhaps the most plausible reason. FreeBSD not liking something with this specific combination of hardware
We'll check the BIOS and also plan to add a third drive to the mirror from another brand to see if this makes any difference.
Thanks!
-
-
See my first reply, the system is up and running after a reboot.
No HDDs, these are SSDs, so PUIS would not apply here. Other power saving functionality might though, I'll see what I can find.
Thanks!
-
@AlexanderL Hows the temperature on the SSDs? They might be self shutting down for protection. Did you use a thermal pad to dissipate the SSD heat? I was reading a lot of amazon reviews on M.2 SATA SSD USB enclosures and they have a lot of complaints of how they get really really hot, plus most of the enclosures are made of aluminium and come with a thermal pad. I understand this is not a USB enclosure however my 2100 has no fan, if your running multiple SSDs heat might cause some issues. They run hot.
-
They're ok, just ran smartctl -x on both drives and everything looks top. Max temp is 40degC, averaging around 30 for both. These are 2.5" drives, not M.2 so more area to spread the heat!
BUT, the firewall went down the same way just a few hours ago, so something is definitely going on, no more info in the current logs though.
I'm a bit new to FreeBSD but reasonably home in Linux. The documentation points to /etc/syslog.conf to handle logging levels and files, but it is more or less empty so I assume PfSense uses a different approach to logging? Do you know how I would go about getting more verbose logging enabled?
-
I know you can set up remote logging and point it to something else under status / system logs / settings
-
Also smart status
View Logs.
-
Thanks, yes, been there, the S.M.A.R.T page offers basically the same as smartctl -x produces, and sadly, the system log settings does not offer any granular control over things like kernel log levels and such. I'll have to dig deeper!
-
@AlexanderL You might need a terminal session for some of them, or use edit if you know the file path in gui.
-
Yes, I've been snooping around via SSH a bit already, I know where the log files are, but the syslog does not contain anything more than what is already logged to the console. I need to find the correct method to increase the log level without risking breaking something in PfSense.
-
@AlexanderL Good luck.
