Intel Pro/10Gbe Issue

[kenth]

I am installing pfSense on a system with 2 Quad port Intel Pro/1000 cards (em driver), 2 Intel onboard NICs (Disabled), and 1 Dual port Intel Pro/10GbE (ix driver).

If I enable jumbo frames (MTU 9000) then the system panics over and over.

If I disable jumbo frames (MTU 1500) then the system stays up but I'm unable to ping to or from the 10GbE interfaces.

Has anyone seen an issue like this and if so is there a fix and what might it be?

This is my first pfSense install so any help would be appreciated.

[wallabybob]

@kenth:

I am installing pfSense

Which build?

@kenth:

If I enable jumbo frames (MTU 9000) then the system panics over and over.

How did you enable jumbo frames? On which interface(s)? Can you capture the panic output?

@kenth:

If I disable jumbo frames (MTU 1500) then the system stays up but I'm unable to ping to or from the 10GbE interfaces.

You haven't mentioned the pfSense interface names for these interfaces. If they are NOT the LAN interface then you will have to add firewall rules to allow incoming pings (and, other traffic I presume) on those interfaces. Please post the ping command(s?) you are using and the ping response to the command. Depending on the exact form of the ping command, there can be a fair bit of network infrastructure needed to even send a ping. The ping report is nearly always much more informative than the user summary "unable to ping".

[kenth]

@wallabybob:

@kenth:

I am installing pfSense

Which build?

2.0.1 from the ISO image.
@wallabybob:

@kenth:

If I enable jumbo frames (MTU 9000) then the system panics over and over.

How did you enable jumbo frames? On which interface(s)? Can you capture the panic output?

I've submitted the panic output using the "crash_report.php" a couple times.  Jumbo frames are enabled on the only 10gbE interface I'm using on the system currently ix0 (pfSense name is SrvLAN).  The system currently has these active interfaces.

NIC    pfSense Name
–----------------------
em0  FWLAN  (WAN)
em1  LabLAN
em4  DevLAN
em5  MgmtLAN
ix0    SrvLAN

I expect to bring up ix1 connected to our backups network and one more em to our admin net.

I've both used the web GUI to enable/disable jumbo frames (MTU 8164 and 9000 have both been tried) as well as direct from the OS using ifconfig ix0 mtu xxxx.  The interface options already show jumbo frame support.

@wallabybob:

@kenth:

If I disable jumbo frames (MTU 1500) then the system stays up but I'm unable to ping to or from the 10GbE interfaces.

You haven't mentioned the pfSense interface names for these interfaces. If they are NOT the LAN interface then you will have to add firewall rules to allow incoming pings (and, other traffic I presume) on those interfaces. Please post the ping command(s?) you are using and the ping response to the command. Depending on the exact form of the ping command, there can be a fair bit of network infrastructure needed to even send a ping. The ping report is nearly always much more informative than the user summary "unable to ping".

pfSense names are listed above, and there are rules in place to allow pings in and out on each interface where it is required, including the ix0 interface.  The weird part of this is that the switch sees our MAC and I dropped a copy of LLDPD on the FreeBSD system and it sees the switch as a neighbor, but if I packet capture from for example our DNS server I never see DNS queries from the firewall.  Ping (just ping xx.xx.xx.xx) results in "ping: sendto: Host is down".  Pinging to the gives the same.

[cyclone3d]

Try booting pfsense into safe mode.

If that fixes it, then you will probably need to disable APIC.

What hardware is pfsense installed on?

[wallabybob]

@kenth:

@wallabybob:

@kenth:

I am installing pfSense

Which build?

2.0.1 from the ISO image.

You might get better results if you try a pfSense 2.1 snapshot build. These are based on FreeBSD 8.3 rather than FreeBSD 8.1 and so have more up to date device drivers.

[kenth]

This is working now.  Not sure what magic caused it to start working, I tried turning off acpi which didn't help, changed mbufs, re-enabled the onboard NIC's, did a few other things and it's working.  Sorry I don't have a better report of what fixed it. Even the jumbo frames is going and happy so I'm happy.

Thanks for your suggestions.