If_msk.ko and if_so.ko files to match 2.1 release for Watchguard X750e

Steve Evans

I've trawled the forum, looking for a "final" version of these drivers with the correct LED behaviour. Now that 2.1 is released do we have final versions of these drivers to install? Also, is it still necessary to specify hw.msk.msi_disable=1 in boot/loader.conf? I've done so as I was previously seeing issues under very high load, and it's probably not doing any harm, but it would be good to get a solid final "approved" configuration for these drivers.

Thanks,

Steve

stephenw10

The modules compiled for 2.1 can be found here: http://forum.pfsense.org/index.php/topic,20095.msg346347.html#msg346347
I haven't had a chance to test them myself but the mods are the same.

Steve

charlie0440

I can confirm they work fine.

Steve Evans

OK, so to confirm, I download if_msk.ko-freebsd-8.3.png and if_sk.ko-freebsd-8.3.png, rename them to just have the "ko" suffix, drop them under /boot/modules, add the following lines to /boot/loader.conf, and reboot?


if_sk_load="yes"
if_msk_load="yes"

Do I still need the hw.msk.msi_disable=1 in boot/loader.conf if using these?

Thanks,

Steve

stephenw10

Yes. I would keep the msi disable sysctl. There hasn't been any specific fix gone into msk(4) that might fix it and it doesn't hurt to have it there.
Interestingly I just updated my test X750e and the original 8.1 modules work fine. You don't get any bug fixes that went between 8.1 and 8.3 though.

Steve

Steve Evans

Thanks Steve.

Given that you've done a lot to support this platform, do you think it would be a good idea to gather together the essentials as a single Watchguard X[5|7|12]50e package that's easily installable?

These network drivers would be a good start, plus the stuff I've just posted in the thread HERE. This includes your WGXepc utility, automatic fan control, and temperature display on the dashboard.

Steve

stephenw10

It would be nice to have it easily installable in one go but it's a bit trickier than that. The lcdproc-dev package should (hopefully when it's stable) be most stuff since it already includes the LED control. Not the fan speed control though. The LCD and LED control are universal for all fireboxes, not just the X-Core-e, but the driver modules aren't though it wouldn't hurt to install them on another box. Probably not good practice.

Steve

Steve Evans

I'm using lcdproc-dev myself. There are a number of issues (another one HERE) that I think a new user, or an experienced one who's just installed an update, might find useful to have a simple installer for. I have all the necessary files in one place now for easy updates; I'm just thinking of ways to make the wonderful experience of pfSense have fewer barriers to adoption for the less technical user.

Steve

stephenw10

I agree, lowering the barrier to entry can only be a good thing. A single installable package has been suggested a number of times but it's just not that easy. Not for me at least. The pfSense package system has a steep learning curve that I've hit but failed to make it up a few times. ::) Having everything in one package then puts significant responsibility on someone to maintain it. That would be across all the models as well. Separate 'packages' means if one thing breaks the rest is still good.

Steve

Steve Evans

In which case, perhaps an update in the Wiki to include the latest for all these issues now that 2.1 is stable?

Steve

ceama

@stephenw10:

Yes. I would keep the msi disable sysctl. There hasn't been any specific fix gone into msk(4) that might fix it and it doesn't hurt to have it there.
Interestingly I just updated my test X750e and the original 8.1 modules work fine. You don't get any bug fixes that went between 8.1 and 8.3 though.

Steve

Hi Stephen10,
I tested the sk and msk drivers compiled in 8.3 and this is how they report on dmesg:

skc0: <marvell gigabit="" ethernet="" (led="" mod="" 0.9)="">port 0xc000-0xc0ff mem 0xd042c000
mskc0: <marvell yukon="" 88e8053="" gigabit="" ethernet="" (led="" mod="" 1.3)="">port 0x8000-0x80ff

So it appears the Watchdog patch is not in the new compile. I am wondering if the watchdog mod in the 8.1 driver has the same effect as the msi disable sysctl? I have been using the [Watchdog/LED mod 1.4] driver for a long time without the sysctl tunable. No issues except the annoying uncorrectable pci error.</marvell></marvell>

stephenw10

The watchdog mod appeared to do nothing, it didn't prevent the timeouts occurring. It seemed unwise to leave the non-functional code in there.
Has your experience been different? I still haven't found a good way to trigger the timeouts reliably so I was relying on the reports of others.

Steve

ceama

I have been using these 2 tunables:

hw.pci.enable_msix="0"
hw.pci.enable_msi="0"

And the sk driver (LED mod 0.9) downloaded from the forum.
And the msk driver (Watchdog/LED mod 1.4) downloaded from the forum.

I have sent that out to run on maybe 50 or so client boxes, some of them under very heavy use in large IT centers for the last year or so with no issues except the uncorrectable PCI error on the msk ports.

I haven't been able to get them to lock up.

stephenw10

Ah yes sorry I didn't recognise your username.
Those are some nice numbers. Should persuade anyone that the X-e is a good box for pfSense.

When I first moded the msk driver to correct the LED config the MSI disable fix wasn't in common use. I wasn't using it because I wasn't seeing any lockups. Some people were though so I added the watchdog timeout code suggested by the msk(4) maintainer in a mailing list. There was no confimation in the list as to it's effectiveness. After trying the updated driver with the code, 1.4, the reports were that it made no difference. The timeouts were still occouring and the driver was not recovering which is what the mod code should have done. Since it seemed to have effect but could have potentially introduced further problems I have been recommending the 1.3 mod module. That's what's referenced in the wiki and on my Google site.
I have seen no timeout problems personally or had any reported using either the standard driver or the 1.3 mod (LEDs only) and the loader tunable:

hw.msk.msi_disable=1

The only way to remove the PCIe error is to update the VPD.
http://forum.pfsense.org/index.php/topic,20095.msg207289.html#msg207289

If you know any different please tell me. :)

Steve

ceama

Stephen,
Like you, I notice there is more code in the FreeBSD 8.3 drivers. I didn't take time to see what was added to them, but I did the LED mods 0.9 on the sk driver, compiled it and it works in pfSense 2.1. I did the LED mods and one other mod in the msk driver to surpress the PCI express error. I noticed in the source code, there is a comment just before the printf that displays the error.

/* Ignore unsupported request error. */

The driver truly does just ignore anything that it can't handle. I commented out the printf statement to supress the printing of the error and compiled on 8.3. Then I set up a test with iperf. I ran 2 copies on the firebox unit. One server and one client and connected msk0 to another firewall with a Intel pro/1000 interface. I ran a test both directions simultaneously so the interface was transmitting and receiving at the same time for 12 straight hours. This activity pegged out the other units CPU at 100%. Transmit speed was 200Mbps, receive speed 120 Mbps. I got no errors or lockups. I wanted a way to get rid of the error without a VPD flash so clients wouldn't have to worry about that. The firebox is running a Pentium M 1.7 GHz. CPU showed 48% during test.

stephenw10

Nice real numbers and nice result. :)

It's tempting to flash the VPD because having done so the standard drivers wouldn't dipslay any errors in the future. However since I have no idea what the VPD code does it may just as likely introduce new errors. I'm sure it's possible to correct the LED behaviour in the VPD code also. Just need to find a friendly Marvell engineer who has access to documentation! :)

Steve

ghostshell

Hey Charlie!

I can confirm then work on

x1000 2.1
x1250e 2.1 (3 units)
x5500e 2.1

stephenw10

There aren't any Marvell NICs in the X1000, no need to load the modules. It won't hurt though.

Steve

ghostshell

Tested them for Charlie

charlie0440

@ghostshell ??? when did I ask for a test? you also pm'd me ??? (sorry if off topic)

ceama

Hi Stephen and others. I have found a bug/issue with the msk ports on the e-series boxes. A customer reported that ports 4-8 would not function in a bridged configuration. I had never tried it so I set up my box with ports 4-8 in a bridge assigned to the LAN. I found the ports constantly resetting in a loop at maybe 5 second intervals. When I started a ping originated from the LAN, I got replies when the interface was up for a few seconds then reset. The link light blinks on and off at a regular rate. The system log showed "rc.newwanip" script was resetting the port every time it linked up. The global variables for the wanip were NULL so some where in the process maze the values were lost and a call to reset whatever msk port linked up was issued. So I went to the script "rc.linkup" and tried to follow the logic there. I wound up duplicating the program flow for a sk port that did not have this problem. I wound up making changes to "rc.linkup" script. the script is really short, so I'll post it here. Make these changes, and the msk port will bridge just fine. It basically follows simple logic: Port linkup -> configure port -> exit. The lines I added/deleted are commented.


#!/usr/local/bin/php -f
/*
        rc.linkup - devd hotplug actions
        part of pfSense

        Copyright (C) 2003-2005 Scott Ullrich <sullrich@gmail.com>.
        All rights reserved.

        Redistribution and use in source and binary forms, with or without
        modification, are permitted provided that the following conditions are met:

        1\. Redistributions of source code must retain the above copyright notice,
           this list of conditions and the following disclaimer.

        2\. Redistributions in binary form must reproduce the above copyright
           notice, this list of conditions and the following disclaimer in the
           documentation and/or other materials provided with the distribution.

        THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
        INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
        AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
        AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY,
        OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
        SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
        INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
        CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
        ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
        POSSIBILITY OF SUCH DAMAGE.
*/

/* parse the configuration and include all functions used below */
require_once("globals.inc");
require_once("config.inc");
require_once("filter.inc");	
require_once("shaper.inc");	
require_once("interfaces.inc");

function handle_argument_group($iface, $argument2) {
	global $config;	

	if (!is_array($config['interfaces'][$iface]))
		return;

	$ipaddr = $config['interfaces'][$iface]['ipaddr'];
	$ip6addr = $config['interfaces'][$iface]['ipaddrv6'];
	$staticv4 = false;
	if (empty($ipaddr))
		$staticv4 = true;
	else
		$staticv4 = is_ipaddrv4($ipaddr);
	$staticv6 = false;
	if (empty($ip6addr))
		$staticv6 = true;
	else
		$staticv6 = is_ipaddrv6($ip6addr);
	if ($staticv4 === true && $staticv6 === true) {
		$friendly = convert_friendly_interface_to_friendly_descr($iface);
		log_error("Hotplug event detected for {$friendly}({$iface}) but ignoring since interface is configured with static IP ({$ipaddr} {$ip6addr})");
		interfaces_staticarp_configure($iface);
		$iface = get_real_interface($iface);
		interfaces_bring_up($iface);
		if ($argument2 == "start" || $argument2 == "up")
			log_error("HOTPLUG: Configuring interface {$iface}"); //New Charlie lines
			require_once("vpn.inc"); //New
			require_once("captiveportal.inc"); //New
			interface_configure($iface, true, true);  //New
			//send_event("interface newip {$iface}"); //Charlie removed. Just configure it
	} else {
		switch ($argument2) {
		case "stop":
		case "down":
			log_error("DEVD Ethernet detached event for {$iface}");
			interface_bring_down($iface);
			break;
		case "start":
		case "up":
			log_error("DEVD Ethernet attached event for {$iface}");
			log_error("HOTPLUG: Configuring interface {$iface}");
			require_once("vpn.inc");
			require_once("captiveportal.inc");
			// Do not try to readd to bridge otherwise em(4) has problems
			interface_configure($iface, true, true); 
			break;
		}
	}
}

global $g;
if (!file_exists("{$g['varrun_path']}/booting") && empty($g['booting'])) {
	if ($argc < 3) {
		log_error("HOTPLUG event: The number of required parameters not passed!");
		exit;
	}
	$action = $argv[1];
	switch($action) {
	case "start":
	case "stop":
		break;
	default:
		log_error("HOTPLUG event: The action parameter passed is wrong($action) only start/stop/up/down are allowed!");
		exit;
		/* NOTREACHED */
		break;
	}
	$interface = convert_real_interface_to_friendly_interface_name($argv[2]);
	if (!empty($interface))
		handle_argument_group($interface, $action);
}

?></sullrich@gmail.com>

stephenw10

Yes, that's a known bug and has been fixed for future snapshots. It's a problem with bridging on some NICs/drivers. A fix is available here: http://forum.pfsense.org/index.php/topic,66908.msg367991.html#msg367991

Steve