Need driver for hardware watchdog

Cino

igor, great find!! Took me a while to get this to work, I had added ipmi.ko to my loader.conf.local a while back. Removed it and all is working now on my X7SPA-D510-HF.. The ipmi-sensors is great add-on!!


ipmi-sensors
4: System Temp (Temperature): 54.00 C (-7.00/77.00): [OK]
71: CPU Temp (Temperature): 61.00 C (-8.00/90.00): [OK]
138: CPU FAN (Fan): NA (NA/NA): [Unknown]
205: SYS FAN (Fan): 4840.00 RPM (585.00/29815.00): [OK]
272: CPU Vcore (Voltage): 1.16 V (0.66/1.41): [OK]
339: Vichcore (Voltage): 1.04 V (0.82/1.18): [OK]
406: +3.3VCC (Voltage): 3.26 V (2.88/3.65): [OK]
473: VDIMM (Voltage): 1.83 V (1.48/1.99): [OK]
540: +5 V (Voltage): 4.96 V (4.32/5.60): [OK]
607: +12 V (Voltage): 12.29 V (10.50/13.06): [OK]
674: +3.3VSB (Voltage): 3.26 V (2.88/3.65): [OK]
741: VBAT (Voltage): 3.15 V (2.62/3.39): [OK]
808: Chassis Intru (Platform Chassis Intrusion): [OK]
875: PS Status (Power Supply): [Presence detected][Unrecognized State][Unrecognized State][Unrecognized State][Unrecognized State][Unrecognized State][Unrecognized State][Unrecognized State]

btw, I have both watchdogd and bmc-watchdog running on my box. They both work, but I need to play with the options for bmc-watchdog. Watchdogd does a hard reset, but bmc-watchdog is doing a power cycle. Which I think is better for the hard drive but it still needs to clean the filesystem when it boots up.

Thanks again everyone!

MrKoen

Thanks igor for sharing your findings! Looking good. I'm going to give it a try in the next few days. Who can create a nice frontpage widget to parse and show the ipmi-sensors information? ;)

_igor_

cino: Thanks for the hint with ipmi="YES" in loader.conf. I forgot it to mention in my post. I added that setting there, it should be complete now.

Best practise is to set this in /boot/loader.conf.local. So it survives upgrades of pfsense.

Cino

@_igor_:

cino: Thanks for the hint with ipmi="YES" in loader.conf. I forgot it to mention in my post. I added that setting there, it should be complete now.

Best practise is to set this in /boot/loader.conf.local. So it survives upgrades of pfsense.

I'm sorry, I had to remove that line to get bmc_watchdog to work. When ipmi.ko is loaded, the watchdog doesn't work for me. ipmi-sensors would also return an error. I unloaded the driver and both programs started to work with no errors. So i removed it from my loader.conf.local, rebooted the box and everything works. Sorry for the confusing

Cino

@Koen:

Thanks igor for sharing your findings! Looking good. I'm going to give it a try in the next few days. Who can create a nice frontpage widget to parse and show the ipmi-sensors information? ;)

I copied page diag_system_activity.php and changed some of the code so it calls ipmi-sensors instead of top… It works :-) Upload it to /usr/local/www and you can access it via http://pfsense/diag_ipmi-sensors.php.

Scott created the page, and if i'm reading his copyright notice correct. We can make changes to it as long as we leave the notice there.

diag_ipmi-sensors.php.txt

_igor_

hey cino, that sounds strange, i have to load ipmi to get that info. Reading at the freeipmi-docs i saw that freeipmi has the ability to load its own ipmi-drivers. Maybe there is something wrong with your system-ipmi. Don't know that. Its included in the pfSense, i didn't load it from other installation.
My entry in /boot/loader.conf.local is ipmi="YES"

Much thanks for that changed/new ipmi-sensors.php! I'll try that out.

Cino

i had added ipmi="YES" to my /boot/loader.conf.local when this thread first started to test the ipmi watchdog. I never did take it out since it wasn't hurting anything. I have updated my BIOS and ipmi firmware to the latest version less then 2 months ago. Maybe that's why it doesn't work when I load the ipmi.ko driver. My ipmi is ver 2.0, i can't remember but i think it was either 1.1 or 1.5 when I my board first came out.

MrKoen

I just applied the steps to my pfSense box and it works superb! We have two great ways of using the IPMI functionality now.

igor: I hope you don't mind, but I created a tutorial on my website for this method based on your tutorial. You already created a nice tutorial in this topic, but for my own reference since I did some things slightly different and as an addition to the first method, I created a page covering this as well. Of course all credits for this method go out to you and are mentioned multiple times in the tutorial.

The tutorial can be found here:

http://www.zomers.eu/knowledge/pfSense/Pages/Configure-pfSense-2.0-RC1-to-use-Watchdog-functionality-Method2.aspx

stephenw10

Another great write up! ;D

Steve

_igor_

yeah! Great work! :-)

Cino

I have a suggestion as this has save my a$$ here and there doing remote updates and getting a panic for no reason.

Instead of putting 10_watchdog.sh in /usr/local/etc/rc.d so it autostarts AFTER pfsense finishes its startup processes. Copy the script to say the /root folder. In my case, I add it to /root/custom/scripts since I have to run scripts after every update so it updates my files for my build. Then manually add it to your config.xml under earlyshellcmd section. You can add the 'shellcmd' package and add it via the gui interface.

This way the watchdogd will start before core processes and interfaces are started.

Last night I was messing around with snort and was rebooting the box a lot… On one reboot, I get a panic after it started OpenNTPD for some reason...16 seconds later, the box rebooted... If the watchdog started when the normal autostart scripts started, it would had never rebooted.

_igor_

ah, i had that too. So i changed watchdog settings to:

bmc-watchdog -d -i 20 -e 16
```.

That helped. Since this change no more unsolicited boots. Seems to be that on heavy config reads/writes the watchdog gets its updates too late. Hope that helps you. good luck!

_igor_

Exists there any way to get that file as an entry in the menu (http://forum.pfsense.org/index.php/topic,34056.msg188926.html#msg188926)?
It would be nice to have that info "at a mouse-click". Thanks for any help to do this!

Cino

Open file /usr/local/www/fbegin.inc and add this line of code under the "//Diagnostics" section (around line 180)

$diagnostics_menu[] = array("IPMP Sensors", "/diag_ipmi-sensors.php" );

Now any firmware/gitsync will erase this change, so you will have to add it back.

jimp

Add it into your config.xml

	 <installedpackages>[...]

<menu>
			<name>IPMP Sensors</name>
			<tooltiptext>IPMP Sensors</tooltiptext>
			Diagnostics
			<url>/diag_ipmi-sensors.php</url>
		</menu>

		[...]</installedpackages>

No code hacks, and it'll never go away.

_igor_

Thanks a lot!

Cino

Thanks Jimp!!! I'm always learn something new from you :-)

_igor_

one more question: after adding the item to the config.xml, do i have to reboot or is there a manual way to get the menus reloaded to get the new entry shown?
Thanks again for your help!

jimp

If you just restore the config it may force a reboot. If you edit it in live on the box in the gui, then rm /tmp/config.cache

If you edit it from the console with viconfig, it will do that automatically and should show up next time you refresh the page in the browser.

_igor_

Another change i did, which is cosmetic:

in the "diag_ipmi-sensors.php" i changed the call of the sensors from

$text = `/usr/local/sbin/ipmi-sensors`

to

$text = `/usr/local/sbin/ipmi-sensors | sed 's/\[Unrecognized\ State\].*/\ /'`;

So the final entry of the ipmi-sensors output doesn't have the multiple "[Unrecognized State]" occurencies which destroy the page-layout. Its a crude hack, but works - with the Supermicro board.

And again thanks jimp for your quick answer to reload the menu! Great! That makes test changes quickly.