Need driver for hardware watchdog
-
@clarknova, if you don't have FreeBSD AMD64 installed on a PC.. Go to PC-BSD website and download their DVD ISO. You can run it LIVE without installing it to a harddrive. If the files are there, copy them to a USB stick then transfer them to your pfsense box. Its long shot, but it worked for me on a different issue. I needed cdcontrol to get my 3G USB Modem working.. Worked like a charm… I used there 8.0 release if I remember correctly.
-
I got myself a SuperMicro X7SPA-HF-D525 board for my pfSense 2 RC1 AMD64 server. It tends to hang every now and then so I would like to use the watchdog feature so it will reboot. I found this topic and did as told:
1. In the BIOS I went to Advanced -> IPMI configuration and set BMC Watch Dog Timer Action to "Reset System" and BMC WatchDog TimeOut to 5 minutes
2. I booted pfSense and updated /boot/loader.conf.local and added ipmi_load="YES"
3. I downloaded ichwd.ko and ichwd.ko.symbols using fetch in the console to /boot/kernel on my pfSense machineI do see the ipmi0 entires in dmesg. It still reboots every 5 minutes though. Did you guys chmod the ichwd.ko and ichwd.ko.symbols files with +x in /boot/kernel? I'm getting the message "Unsupported file layout. Exec format error." when doing a kldload ichwd in the console.
I'm using pfSense build 2.0-RC1-IPv6 (amd64)built on Thu Apr 28 03:47:19 EDT 2011.
Can anyone provide me with tips what to try?
-
You probably need to invoke a program to "tickle" the watchdog "often enough".
See the FreeBSD man page for ichwd (http://www.freebsd.org/cgi/man.cgi?query=ichwd&apropos=0&sektion=0&manpath=FreeBSD+8.2-RELEASE&format=html ) and the links in the See Also section.
-
There are at least two watchdogs on this board.
As I said in a previous message the watchdog invoked by the bios is likely the one on the Winbond SuperIO chip and NOT the ich watchdog. Thus the ICH driver will not 'pat' it.
The drivers I posted are for 32bit only so you will need some for 64bit, unless you've already done that. ;)Cino's report of success involved loading the ich drivers and starting the watchdog daemon. This will then set the watchdog and keep 'patting' it. You should leave the ipmi watchdog off in the bios.
Good luck! :)
Steve
-
Thanks for your pointers guys. I'm fairly new to the worlds of Unix and Linux, so let me rephrase your comments to see if I understand them correctly.
1. This particular board probably has two Watchdogs of which one of them is controlled by the BIOS and one can be controlled by the right driver/software
2. I should not use the BIOS setting since the ichwd driver/software can not tickle that dog to keep it awake and thus it will reboot
3. I should install a new server with FreeBSD 8.1 AMD64 and retrieve the ichwd.ko and ichwd.ko.symbols files from the /boot/kernel folder and copy them to my pfSense installation. After that I should try the kldload ichwd.ko again and I can kill the FreeBSD 8.1 server again.Makes me wonder.. what sets the timeout on the non-BIOS watchdog? Does it work like when the ichwd software sends the non-BIOS watchdog out of the door to guard the garden and whistle at it at some interval to keep it awake? And when it stops whistling at it, a chip on the mainboard activated by ichwd once it started detects the watchdog has fallen asleep because of the lack of whistles and reboots the system?
-
1. Both are hardware watchdogs and once switched on (by setting a bit on the chip) they will reset the board after a set interval. Both will have adjustable intervals, determined by the contents of a register on the chip. Both require a piece of software to periodically reset a bit in the chip resetting the timer.
2. The BIOS settings set the reisters in the Winbond SuperIO chip to activate it's watchdog and set it's timer interval. You would then need to use the ipmi diver to 'tickle' it.
The ICH watchdog is not set by the bios and is off by default. Loading the ICHWD drivers allows watchdogd (the watchdog daemon) to set the appropriate resisters to switch it on. I've not played with it but it would probably also allow setting the timer interval. (edit: with the -t switch, default is 16s).3. You will need 64bit versions, that may be the easiest way to get them. There is a 64bit version of pc-bsd which I believe will run as a live cd. You can copy the files from that.
Steve
-
Still no go here. I couldn't find out where and how to get the live bootable FreeBSD CD, so I went a head and installed a FreeBSD 8.2 AMD64 installation within VirtualBox. After finally having done that, I used the FTP command to upload the ichwd.ko and ichwd.ko.symbols files to my FTP server. Next from the console on my pfSense installation I used the same FTP command to retrieve them from my FTP server (using binary mode). I placed the files within /boot/kernel. Now when I run kldload ichwd from within /boot/kernel, I still get the same error:
KLD ichwd.ko: depends on kernel - not available or version mismatch
linker_load_file: Unsupported file type
kldload: can't load ichwd: Exec format errorI also tried chmod +x ichwd.ko but that makes no difference.
I have attached the ichwd files I retrieved from FreeBSD 8.2 AMD64 to this post. Anyone knows what else I can try?
-
Hmmm, I thought that 2.0 was based on 8.1.
What does it show if you type:uname -aObviously this has to match.
Steve
-
I personally haven't tried the AMD64 because of drivers. I may try it one of these days but I'll have to make sure I have drivers first as I dont want to lose my pico-lcd display.
PC-BSD 8.0 DVD: ftp://ftp.pcbsd.org/pub/archived/8.0/amd64/PCBSD8.0-x64-DVD.iso
PC-BSD 8.2 DVD: ftp://mirrors.isc.org/pub/pcbsd/8.2/amd64/PCBSD8.2-x64-DVD.isoI have used 8.0 in the base to copy of some file, but not drivers.
-
Your links seem slightly broken there Cino.
Shouldn't we be using 8.1 anyway? My 2.0 box isn't on I can't check.Steve
Edit: And my link was also broken! Doesn't like ftp.
-
Steve, uname -a shows the following here:
FreeBSD <fqdn of="" my="" pfsense="" box="">8.1-RELEASE-p3 FreeBSD 8.1-RELEASE-p3 #1: Fri Apr 29 21:17:50 EDT 2011 root@FreeBSD_8.0_pfSense_2.0-AMD64.snaps.pfsense.org:/usr/obj.pfSense/usr/pfSensesrc/src/sys/pfSense_SMP.8 amd64
What does this actually tell me? Is it based on FreeBSD 8.1 or 8.0 since I see both being mentioned.
I tried the PC-BSD ISO, but that didn't work within my VirtualBox image. I'll give it another try with FreeBSD 8.1. Thanks for the link!
Cino, I was running pfSense 2.0 RC1 i386 within VMWare ESXi on my SuperMicro X7SPA-HF-D525 board. This was very unstable with crashes and hangs at least 5 times a day. So I tried an install of pfSense 2.0 RC1 AMD64 directly on the hardware. This is less unstable but still creates hangs at least twice a day. This is with it being updated up to the latest beta version. I tried returning to the stable pfSense 2.0 RC1 i386 release, but I couldn't get it gitsynced with the smos git to include the IPv6 support. I remember you stating in a different forum topic that your installation on similar hardware was stable. I would recommend keeping it like that. The only reason I'm trying to get this watchdog to work is because it is so unstable.</fqdn>
-
It's based on 8.1.
The only thing I can say I've done personally is installed FreeBSD 8.1 x86 and copied files from that to pfSense 2.0Beta5.
Steve
-
It worked! Thanks guys for helping me out! I indeed needed the files from the FreeBSD 8.1 release and not the FreeBSD 8.2 release. To show my appreciation for you guys helping me out and deliver something back for it to the community, I took the time to put all steps required in one tutorial for both the i386 as well as the AMD64 versions of pfSense 2.0 RC1 together with the downloads for both of them. Find the tutorial and downloads here:
-
To show my appreciation for you guys helping me out and deliver something back for it to the community, I took the time to put all steps required in one tutorial for both the i386 as well as the AMD64 versions of pfSense 2.0 RC1 together with the downloads for both of them.
Great way to show your appreciation. Thanks.
-
Great write up. :)
Steve
-
Thanks. I'm glad to be able to help others with this great product. And you guys know what the funny thing about all of this is? Since I have the watchdog up and running, my pfSense installation hasn't crashed or freezed anymore for a whole day already! It used to crash or freeze at least 5 times a day before I got this up and running. Can't explain it, but I'm very happy with it ;)
-
Really nice write up!! I made have to try 64bit one of days now :-) Just need one more driver and i'll be good to go
Good to hear that your box is stable now
-
I want to express once more that following the steps to enable the watchdog really has made my pfSense system rock solid and stable now. It has been running for a week already without any problems, a single crash or single hang. I did a reinstall on a new disk last weekend and the crashes and hangs started right again. Once applying the watchdog driver again, the system is solid and stable again. Strange, but true. So if you're using the SuperMicro X7SPA-HF-D525 or equivalent board and experience an unstable pfSense installation, apply these easy steps.
-
Weird? :-
I sounds like the perhaps the ICH is not being correctly setup by either the bios or the standard ICH driver.
Loading the ICHWD driver sets the registers to known values preventing some unstable condition.
Speculation. ;)Steve
-
Nice write-up.
Does anyone know if this same procedure will work with a Netgate hamakua?
