Pfsense C2758 crashes when connecting SFP+ cable. (Extreme x670v 48t)

[gmanual]

I'm having issue with the C2758 product and connecting to a switch using SFP+ cable (10Gbe), as soon as or shortly after connecting the SFP+ cable (direct attach copper) the C2758 will crash and reboot, it will stay in a reboot loop until the cable is removed.

I've had a look at the following wiki and tried apply some of the fixes, but the Chelsio dual port SFP+ adapter that comes with the C2758 doesn't seem to have any mention:

https://doc.pfsense.org/index.php/Tuning_and_Troubleshooting_Network_Cards

The switch I'm trying to connect the C2758 to is a Extreme x670v 48t (48 port 10Gbe BaseT switch with 4 10Gbe SFP+ ports).

I've performed a factory reset via the console on the C2758 and perform the following steps to replicate the issue:

• Add 10Gbe interfaces after first launch wizard, and enable them, no ip config. (Screenshot 01)

• Insert SFP+ cable into C2758, it shows link down as not connected to switch but recognises cable. (Screenshot 02)

• Connect SFP+ cable to switch, link appears to come up. (Screenshot 03)

• Shortly after ~20 seconds, the C2758 will crash, losing connectivity and automatically reboot (Screenshot 04)

• I've tried 3 different types of SFP+ cables to isolate if it's a cable issue, all brand new. (Screenshot 05)

I've attached the crashlog, I'm hoping someone with more experience with pfsense can help me narrow down the issue, or turn on right debugging.

We actually have 4x C2758 and 2x Extreme x670v 48t and the issue occurs on all devices so I suspect there is some form of compatibility issue between these devices.

Any assistance greatly appreciated.

Edit: I've now tried connecting the C2758 to a Dell PowerConnect 6224P switch using 10Gbe SFP+ ports, same issue, the C2758 will crash and reboot.
crashlog-public.txt

[charliem]

@gmanual:

I'm having issue with the C2758 product and connecting to a switch using SFP+ cable (10Gbe), as soon as or shortly after connecting the SFP+ cable (direct attach copper) the C2758 will crash and reboot, it will stay in a reboot loop until the cable is removed.

I've had a look at the following wiki and tried apply some of the fixes, but the Chelsio dual port SFP+ adapter that comes with the C2758 doesn't seem to have any mention

C2758 is both an Intel CPU designation and (unfortunately) a pfSense product (http://store.pfsense.org/c2758 ) and netgate product for that matter (http://store.netgate.com/Firewall/C2758.aspx).

Assuming you bought the 1U product, you should follow up warranty issues with the vendor technical support (http://store.pfsense.org/contact-us/ or http://store.netgate.com/Technical-Support-Web-Page-W3.aspx)  Though you may get some help here from the pfsense community, you paid for warranty / support so you should get some benefit from it.

A good first step would be to grap a console log file of a boot, attach cable, crash and reboot cycle.

[Guest]

he bought these from us.  contacted support and posted here.

We're looking into it.

[gmanual]

Thanks for your reply, I've attached console logging.

• Full console log.

• Console boot no cable.

• Console cable inserted

• Console connected to switch and crash.

• Reboot and crash.

Also attached diagram of my lab, I've recently found if I don't try to login/ssh to the firewall, and just ping, it will stay up and give ICMP responses, if I try and connect in any way it will crash.

/boot/loader.conf.local

conf.local
kern.cam.boot_delay="10000"
ahci_load="YES"
kern.cam.boot_delay=10000
kern.ipc.nmbclusters="1000000"
hw.igb.rxd=4096
hw.igb.txd=4096
hw.igb.max_interrupt_rate=32000
hw.igb.num_queues=8
if_cxgbe_load="YES"

[01 - consolelog-full.txt](/public/imported_attachments/1/01 - consolelog-full.txt)
[02 - console-boot.txt](/public/imported_attachments/1/02 - console-boot.txt)
[03 - console-insert.txt](/public/imported_attachments/1/03 - console-insert.txt)
[04 - console-crash.txt](/public/imported_attachments/1/04 - console-crash.txt)
[05 - console-reboot-crash.txt](/public/imported_attachments/1/05 - console-reboot-crash.txt)

[stephenw10]

Your switch diagram shows the pfSense LAN as clx1 but in the earlier screenshots it's an igb nic. Which is it?

Edit: logs show it's clx1. Works for icmp but not tcp, some hardware off-loading issue?

Steve

[gmanual]

Thanks for your reply, it was both, as I continued testing I limited the number of interfaces I used on the firewall to replicate the issue.

Thanks for your suggestion, it looks like from googling "hardward off-loading" I have some ifconfig options to try.

My time for testing is very limited over the next 24-48 hours, but will be giving this a try over the weekend.

[gmanual]

An update, I did do some testing with ifconfig and hardware off-loading options, it seemed to make no difference.

After following up with support, I have tried the 2.2-Beta which I was advised had better native driver support for the SFP+ card.

2.2-BETA-amd64-20141106-0359

I've been able to test this version and it does appear to resolve the issues I've been having.

Thanks for the assistance.

[stephenw10]

Thanks for following up.  :)