If_msk.ko and if_so.ko files to match 2.1 release for Watchguard X750e
-
Yes. I would keep the msi disable sysctl. There hasn't been any specific fix gone into msk(4) that might fix it and it doesn't hurt to have it there.
Interestingly I just updated my test X750e and the original 8.1 modules work fine. You don't get any bug fixes that went between 8.1 and 8.3 though.Steve
Hi Stephen10,
I tested the sk and msk drivers compiled in 8.3 and this is how they report on dmesg:skc0: <marvell gigabit="" ethernet="" (led="" mod="" 0.9)="">port 0xc000-0xc0ff mem 0xd042c000
mskc0: <marvell yukon="" 88e8053="" gigabit="" ethernet="" (led="" mod="" 1.3)="">port 0x8000-0x80ffSo it appears the Watchdog patch is not in the new compile. I am wondering if the watchdog mod in the 8.1 driver has the same effect as the msi disable sysctl? I have been using the [Watchdog/LED mod 1.4] driver for a long time without the sysctl tunable. No issues except the annoying uncorrectable pci error.</marvell></marvell>
-
The watchdog mod appeared to do nothing, it didn't prevent the timeouts occurring. It seemed unwise to leave the non-functional code in there.
Has your experience been different? I still haven't found a good way to trigger the timeouts reliably so I was relying on the reports of others.Steve
-
I have been using these 2 tunables:
hw.pci.enable_msix="0"
hw.pci.enable_msi="0"And the sk driver (LED mod 0.9) downloaded from the forum.
And the msk driver (Watchdog/LED mod 1.4) downloaded from the forum.I have sent that out to run on maybe 50 or so client boxes, some of them under very heavy use in large IT centers for the last year or so with no issues except the uncorrectable PCI error on the msk ports.
I haven't been able to get them to lock up.
-
Ah yes sorry I didn't recognise your username.
Those are some nice numbers. Should persuade anyone that the X-e is a good box for pfSense.When I first moded the msk driver to correct the LED config the MSI disable fix wasn't in common use. I wasn't using it because I wasn't seeing any lockups. Some people were though so I added the watchdog timeout code suggested by the msk(4) maintainer in a mailing list. There was no confimation in the list as to it's effectiveness. After trying the updated driver with the code, 1.4, the reports were that it made no difference. The timeouts were still occouring and the driver was not recovering which is what the mod code should have done. Since it seemed to have effect but could have potentially introduced further problems I have been recommending the 1.3 mod module. That's what's referenced in the wiki and on my Google site.
I have seen no timeout problems personally or had any reported using either the standard driver or the 1.3 mod (LEDs only) and the loader tunable:hw.msk.msi_disable=1The only way to remove the PCIe error is to update the VPD.
http://forum.pfsense.org/index.php/topic,20095.msg207289.html#msg207289If you know any different please tell me. :)
Steve
-
Stephen,
Like you, I notice there is more code in the FreeBSD 8.3 drivers. I didn't take time to see what was added to them, but I did the LED mods 0.9 on the sk driver, compiled it and it works in pfSense 2.1. I did the LED mods and one other mod in the msk driver to surpress the PCI express error. I noticed in the source code, there is a comment just before the printf that displays the error./* Ignore unsupported request error. */
The driver truly does just ignore anything that it can't handle. I commented out the printf statement to supress the printing of the error and compiled on 8.3. Then I set up a test with iperf. I ran 2 copies on the firebox unit. One server and one client and connected msk0 to another firewall with a Intel pro/1000 interface. I ran a test both directions simultaneously so the interface was transmitting and receiving at the same time for 12 straight hours. This activity pegged out the other units CPU at 100%. Transmit speed was 200Mbps, receive speed 120 Mbps. I got no errors or lockups. I wanted a way to get rid of the error without a VPD flash so clients wouldn't have to worry about that. The firebox is running a Pentium M 1.7 GHz. CPU showed 48% during test.
-
Nice real numbers and nice result. :)
It's tempting to flash the VPD because having done so the standard drivers wouldn't dipslay any errors in the future. However since I have no idea what the VPD code does it may just as likely introduce new errors. I'm sure it's possible to correct the LED behaviour in the VPD code also. Just need to find a friendly Marvell engineer who has access to documentation! :)
Steve
-
Hey Charlie!
I can confirm then work on
x1000 2.1
x1250e 2.1 (3 units)
x5500e 2.1 -
There aren't any Marvell NICs in the X1000, no need to load the modules. It won't hurt though.
Steve
-
Tested them for Charlie
-
@ghostshell ??? when did I ask for a test? you also pm'd me ??? (sorry if off topic)
-
Hi Stephen and others. I have found a bug/issue with the msk ports on the e-series boxes. A customer reported that ports 4-8 would not function in a bridged configuration. I had never tried it so I set up my box with ports 4-8 in a bridge assigned to the LAN. I found the ports constantly resetting in a loop at maybe 5 second intervals. When I started a ping originated from the LAN, I got replies when the interface was up for a few seconds then reset. The link light blinks on and off at a regular rate. The system log showed "rc.newwanip" script was resetting the port every time it linked up. The global variables for the wanip were NULL so some where in the process maze the values were lost and a call to reset whatever msk port linked up was issued. So I went to the script "rc.linkup" and tried to follow the logic there. I wound up duplicating the program flow for a sk port that did not have this problem. I wound up making changes to "rc.linkup" script. the script is really short, so I'll post it here. Make these changes, and the msk port will bridge just fine. It basically follows simple logic: Port linkup -> configure port -> exit. The lines I added/deleted are commented.
#!/usr/local/bin/php -f /* rc.linkup - devd hotplug actions part of pfSense Copyright (C) 2003-2005 Scott Ullrich <sullrich@gmail.com>. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1\. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2\. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* parse the configuration and include all functions used below */ require_once("globals.inc"); require_once("config.inc"); require_once("filter.inc"); require_once("shaper.inc"); require_once("interfaces.inc"); function handle_argument_group($iface, $argument2) { global $config; if (!is_array($config['interfaces'][$iface])) return; $ipaddr = $config['interfaces'][$iface]['ipaddr']; $ip6addr = $config['interfaces'][$iface]['ipaddrv6']; $staticv4 = false; if (empty($ipaddr)) $staticv4 = true; else $staticv4 = is_ipaddrv4($ipaddr); $staticv6 = false; if (empty($ip6addr)) $staticv6 = true; else $staticv6 = is_ipaddrv6($ip6addr); if ($staticv4 === true && $staticv6 === true) { $friendly = convert_friendly_interface_to_friendly_descr($iface); log_error("Hotplug event detected for {$friendly}({$iface}) but ignoring since interface is configured with static IP ({$ipaddr} {$ip6addr})"); interfaces_staticarp_configure($iface); $iface = get_real_interface($iface); interfaces_bring_up($iface); if ($argument2 == "start" || $argument2 == "up") log_error("HOTPLUG: Configuring interface {$iface}"); //New Charlie lines require_once("vpn.inc"); //New require_once("captiveportal.inc"); //New interface_configure($iface, true, true); //New //send_event("interface newip {$iface}"); //Charlie removed. Just configure it } else { switch ($argument2) { case "stop": case "down": log_error("DEVD Ethernet detached event for {$iface}"); interface_bring_down($iface); break; case "start": case "up": log_error("DEVD Ethernet attached event for {$iface}"); log_error("HOTPLUG: Configuring interface {$iface}"); require_once("vpn.inc"); require_once("captiveportal.inc"); // Do not try to readd to bridge otherwise em(4) has problems interface_configure($iface, true, true); break; } } } global $g; if (!file_exists("{$g['varrun_path']}/booting") && empty($g['booting'])) { if ($argc < 3) { log_error("HOTPLUG event: The number of required parameters not passed!"); exit; } $action = $argv[1]; switch($action) { case "start": case "stop": break; default: log_error("HOTPLUG event: The action parameter passed is wrong($action) only start/stop/up/down are allowed!"); exit; /* NOTREACHED */ break; } $interface = convert_real_interface_to_friendly_interface_name($argv[2]); if (!empty($interface)) handle_argument_group($interface, $action); } ?></sullrich@gmail.com> -
Yes, that's a known bug and has been fixed for future snapshots. It's a problem with bridging on some NICs/drivers. A fix is available here: http://forum.pfsense.org/index.php/topic,66908.msg367991.html#msg367991
Steve
