S.M.A.R.T. Status crashes pfSense

thomashong

Just fresh installed pfSense 2.3. When I add widget S.M.A.R.T. Status, it immediately crashes pfSense. After rebooting from crash, if try to login web GUI, it will immediately crash system again. To fix this, I have to choose option 15. restore recent configuration. This was happening on the 2.3 release candidate and I assumed it was because it beta software.

I am running pfSense in a Hyper-V 2012 R2 VM and recall smart status works on a physical Dell PC.

jimp

What are the details of the "crash"?

Never assume a crash is because it's beta, always report a crash with as much detail as possible. If you're the only one that has hit it, we won't know about it otherwise.

thomashong

What details are you looking for? I am new and still learning my way around pfSense. I checked the System->General logs and it shows nothing.

I can confirm it's not adding the widget. If I go to "Diagnostics / S.M.A.R.T. Status / Information & Tests" page and as soon as I click the View buttons it crashes the system immediately.

jimp

Any info about the crash you have, errors shown on the screen, etc.

Looks like maybe one other person hit it also. Not anything we can do about that, it's a HyperV issue. SMART data in HyperV doesn't seem to make much sense anyhow. Why would you add that widget? What was it doing for you before?

thomashong

I was adding widgets one at a time on my new 2.3 install. I wanted to see what each widget do and save the ones I find useful on my dashboard. The SMART status may or maybe not be useful but pfsense crashing is not acceptable.

Again, pfsense instantly crashes when I click to view SMART status. I am able to capture a screenshot of the VM console. See attachment.

pfsense-crash.jpg_thumb

jimp

pfSense isn't crashing. HyperV is removing your disk when you do that, which makes pfSense flip out because its HDD disappeared. Not much we can do about that except maybe detect HyperV and stop people from touching SMART on HyperV.

jimp

https://redmine.pfsense.org/issues/6147

fohdeesha

HyperV really just makes the disk disappear when it sees SMART queries? And people use this software in real life?

jammcla

Thanks for adding it as a bug to fix. I had the same problem, doing the same thing, checking out widgets to see what they do/look like.

JorgeOliveira

@fohdeesha:

HyperV really just makes the disk disappear when it sees SMART queries? And people use this software in real life?

+1
:)

If I am not mistaken, a possible "solution" to this would be adding a routine to detect virtualized environments and disabling SMART for them here:
https://github.com/pfsense/pfsense/blob/RELENG_2_3_0/src/etc/inc/util.inc#L2436
(It doesn't make sense for HyperV - and probably others as well)

I wonder what is the output of:

/bin/kenv smbios.system.product 2>/dev/null

on HyperV and if it is something unique for that environment…

Also found this little function that aims detecting virtualized environments:
https://github.com/pfsense/pfsense/blob/RELENG_2_3_0/src/etc/inc/pfsense-utils.inc#L1611
That could be used to workaround the problem, provided HyperV detection is possible/implemented.

Regards,
Jorge M. Oliveira

PS: The output of the following could also be useful:

dmidecode | egrep -i 'manufacturer|product' 2>/dev/null

Lots of info here: https://unix.stackexchange.com/questions/89714/easy-way-to-determine-virtualization-technology

PS 2: You can edit /etc/inc/util.inc carefully and change the function get_smart_drive_list() to: (there is an high chance this will do the trick)

/* Locate disks that can be queried for S.M.A.R.T. data. */
function get_smart_drive_list() {

	// XXX: S.M.A.R.T. Status crashes pfSense
	// https://redmine.pfsense.org/issues/6147
	// https://forum.pfsense.org/index.php?topic=109849.0
	exec("/usr/local/sbin/dmidecode | /usr/bin/egrep -i 'manufacturer|product' 2>/dev/null", $output, $rc);
	if ($rc == 0 && isset($output[0])) {
		$output = implode("\n", $output);
		if(stripos($output, 'Microsoft Corporation') !== false && stripos($output, 'Virtual Machine') !== false)
			return array();
	}

	$disk_list = explode(" ", get_single_sysctl("kern.disks"));
	foreach ($disk_list as $id => $disk) {
		// We only want certain kinds of disks for S.M.A.R.T.
		// 1 is a match, 0 is no match, False is any problem processing the regex
		if (preg_match("/^(ad|da|ada).*[0-9]{1,2}$/", $disk) !== 1) {
			unset($disk_list[$id]);
		}
	}
	sort($disk_list);
	return $disk_list;
}

Good Luck!

thomashong

If I am not mistaken, a possible "solution" to this would be adding a routine to detect virtualized environments and disabling SMART for them

It could just be Hyper-V. This user is using VirtualBox and it does not crash. See his Comprehensive Guide to pfSense 2.3 Part 5.1: General Configuration and WebUI Tour video on youtube. At 1:39.15, he can view his SMART status information under Diagnostics. Useful information like capacity, size, and SMART error message shows up. And at 1:45:40, he adds the SMART Status widget to his dashboard.

fohdeesha

For what it's worth in both xen and kvm VM's I've had pfsense in I've added smart widgets and also viewed the smart status page, and they certainly didn't spit any error let alone drop the entire disk ;D

This is most certainly a hyper-v bug, and a pretty serious one