Igb poor performance

wallabybob

igb0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
options=53b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,tso4,lro></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,tso4,lro></up,broadcast,running,simplex,multicast>

any suggestions on how to switch this off and it survive a reboot.

The options here show the device capabilities, not whether those capabilities are enabled or disabled.

jeffreysmith

It seems to disapper from that list when i run ifconfig igb0 -lro but i just cant get it to stick after reboot without having to run the thing manually.

is there a file i can add it to to get ifconfig igb0 -lro to run on boot. If i make changes to interfaces and bring htem up and down i also need it to run.

any ideas on how to do this ?

wallabybob

Maybe you should try a snapshot build of pfSense 2.0 BETA. It will have possibly have a more up to date igb driver than the special driver for pfSense 1.2.3.

There have been a few problems in the recent 2.0BETA snapshot builds so be sure to take an appropriate backup before upgrading.

dreamslacker

Try adding the following into /boot/loader.conf instead
hw.igb.lro=0

wallabybob

@dreamslacker:

Try adding the following into /boot/loader.conf instead

/boot/loader.conf won't necessarily be preserved over firmware upgrades. /boot/loader.conf.local should be preserved over firmware upgrades.

dreamslacker

Yes, I understand that. It's just that there shouldn't be any 'upgrades' from 1.2.3, yes? Not to mention, the if_load for igb was added under loader.conf for him.
If he's headed for 2.0 then the option to do so would be under the Advance -> Network settings.

jimp

Disable TSO as well as LRO. They can both impact performance, especially on 1.2.x.

net.inet.tcp.tso=0

and/or ifconfig <if>-tso</if>

jeffreysmith

thanks for all the info

I also got the model of the cards wrong they are vt's not et's

Im sorry but this is going to be a production box so 2.0 does not seem to be an option.

I have disabled tso and lro in both /boot/loader.conf.local and /boot/loader.conf but they still appear to be on when i reboot the box.

If i manually run ifconfig -tso / ifconfig -lro the options on the interfaces goes from

options=53b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,tso4,lro>to
options=53b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu>So im assuming they are not being switched off correctly on reboot or am i wrong ?

Im also looking for tools in freebsd to test the network cards if anyone has a decent guide. Things like constantly sending data for a few hours accross a link to see how well it handles it and what strain the nics are under if this is possible.

I have been trying to setup multiple IpSec connections across the 2 broadcom nics on the motherboard of this machine with another identical machine through 1 switch 10 feet apart with very unreliable results and im starting to blame the network cards so it would be nice to eliminate them as a reliability concern.</rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,jumbo_mtu,tso4,lro>

wallabybob

What version of the igb driver did you build? There have been updates to the FreeBSD 7.x stream since pfSense 1.2.3. Perhaps a more current igb driver would give better results. Have you reviewed the igb driver revision history?

jeffreysmith

I built the drivers from this page http://downloadcenter.intel.com/Detail_Desc.aspx?agr=Y&DwnldID=15815&ProdId=2851&lang=eng

The info that comes with the drivers is pretty much the igb manpage with this added

Known Limitations

Detected Tx Unit Hang in Quad Port Adapters
–-----------------------------------------

In some cases ports 3 and 4 won’t pass traffic. Ports 1 and 2 don't show
any errors and will pass traffic.

This issue MAY be resolved by updating to the latest BIOS. You can
check your system's BIOS by downloading the Linux Firmware Developer Kit
that can be obtained at http://www.linuxfirmwarekit.org/

There are known performance issues with this driver when running UDP traffic
with Jumbo Frames.
----------------------------------------------------------------------------

dreamslacker

TBH, the current revisions of 2.0 (i386) are extremely stable once you configure it properly for what you need. I've found that most of the revisions add little things but the benefits are tremendous over 1.2.3.

Personally, I've been on the Aug beta 4 builds in production environments and haven't seen a need to actually upgrade. Everything works except that RRD queue graphs are shot once the shaper is reconfigured. A removal of the queue graphs and refreshing of the RRD script fixes it and since the traffic shaper is almost never reconfigured once it works, that's fine for me.