Poor performance with IGB Driver -> Known BUG

Juve

Hi all,

I've a special case here I've never seen before. It is a standard setup with a server that has 6 Nics:

2 integrated broadcom using BCE driver
a Quad port Nic using IGB driver
The box is running 1.2.3 RC3 11/14/09 snapshot

The setup is like this!

WAN
|
BOX–->DMZ
|
LAN

Wan is using igb0
LAN is using bce0
DMZ is using igb1

Trafic from WAN to LAN = OK
Trafic from LAN to WAN = OK
Trafic from LAN to DMZ = NOK
Trafic from WAN to DMZ = NOK

When I say NOK, this means that pakets get dropped.
I I do a tcpdump on the WAN interface I can see those type of packet each time I get drops:

192.168.1.2 > X.X.X.X: ICMP Y.Y.Y.Y unreachable - need to frag (mtu 1500), length 48

Where X.X.X.X is an external host and Y.Y.Y.Y a DMZ server (running Linux, windows, BSD whatever you want)

Changing MTU does nothing
Sometimes, it works well, the behaviour is totaly random.
Switches are brand new HP 2510-48G, câbles are CAT 6 brand new too.
Every error counters are at 0
Nothing in the logs

I'm thinking that the Quad port NIC has problem transmitting trafic inside itself.

What do you about that ?

danswartz

that's hard to believe - they are separate phys. i wonder if it's a driver bug?

Juve

10 hours on that problem….

I've just removed all TSO opts vi ifconfig -tso but it doesn't help.
Tomorrow I'll check firmware version.

Juve

I really need help,

Does someone has a NIC using the IGB driver ? It's strange a ifconfig says the NICs are VLAN capable but they do not show UP in the web GUI in the assign tab.
I'll try compiling the last IGB driver from intel…

Juve

Ok I finally got it:

http://lists.freebsd.org/pipermail/freebsd-doc/2009-June/015983.html

An issue was found in the igb(4) driver that causes desastrous
performance when FreeBSD acts as a gateway (ip forwarding enabled) and
LRO (large receive offloading) is enabled (which is the default
setting). As a workaround, disable LRO by adding the following lines to
/etc/sysctl.conf:
dev.igb.0.enable_lro=0
dev.igb.1.enable_lro=0
and reboot.

danswartz

good to hear!