Intel Pro/1000 VT hard lock

shutch

Hi all,

I seem to be getting this strange kernel panic when traffic is going from WAN<->LAN. I am using an HP T730 thin client with a Dell YT674 (Intel Pro/1000 VT) quad port NIC.

Using the pfsense shell I can pipe a download to /dev/null with no apparent errors. However, as soon as I run a speedtest on the LAN computer, pfsense hard locks (no traffic, shell freezes, hard reboot required).

The crash reporter on restart shows the following:

<118>pfSense 2.4.4-RELEASE amd64 Thu Sep 20 09:03:12 EDT 2018
<118>Bootup complete
spin lock 0xffffffff82ba9a40 (smp rendezvous) held by 0xfffff801b0b8b000 (tid 100546) too long
timeout stopping cpus
panic: spin lock held too long
cpuid = 3
KDB: enter: panic
panic.txt0600002713360263234  7141 ustarrootwheelspin lock held too longversion.txt06000030313360263234  7611 ustarrootwheelFreeBSD 11.2-RELEASE-p3 #17 e6b497fa0a3(RELENG_2_4_4): Thu Sep 20 09:04:45 EDT 2018
    root@buildbot3:/crossbuild/ce-244/obj/amd64/WvDslnYb/crossbuild/ce-244/pfSense/tmp/FreeBSD-src/sys/pfSense

Any thoughts would be appreciated.

stephenw10

Disable all hardware off-loading if you haven't done so already.

Try increasing mbufs or reducing the queues to 1:
https://www.netgate.com/docs/pfsense/hardware/tuning-and-troubleshooting-network-cards.html?highlight=tuning#intel-igb-4-and-em-4-cards

Unforrunately the VT is probably the only Intel card I've heard bad things about:

@jasonlitka said in UPG 2.1 -> 2.1.1.: extremely high latency & package loss Intel IGB:

If that doesn't fix it then light the card on fire and buy a i350. The fewer VT cards that remain in this world the better.

This seems to be exactly the same setup you have and they ended up replacing the card:
https://www.reddit.com/r/PFSENSE/comments/864mos/help_troubleshooting_kernel_panic/

Steve

shutch

Thanks @stephenw10. As an update to this, I replaced the Intel Pro/1000 VT to an HP NC365T and all issues have been resolved.

Thanks!