HP DL360G6 - VLAN trunk causes bce NIC flapping and watchdog timeout
so I got a cheap HP DL360 G6, swapped a bit of hardware from my spares and installed pfSense 2.3.2_1 on it to migrate my current class C network to class A with multiple VLANs.
It's going to route all inter VLAN traffic, run pfBlockerNG, snort, squid and a lot of VPNs (client, server and s2s) so it hits the CPU quite a bit hence I dont mind the power consumption compared to an embedded system.
I planned on a router-on-a-stick setup with a LACP trunk of 2 ports from a C3750G to pfSense. However this only seems to work for a few seconds before it crashes.
The syslog indicated that the interfaces go down, come back up and all services are restarted. For troubleshooting purposes I turned off LACP and moved VLANs to a single trunk port both on pfSense and on the switch.
Upon changing the LAG conf pfSense crashed. After the reboot the issue still persists and the following lines showed up in the crash dump (which I submitted to devs):
I am running the following tunables:
kern.ipc.nmbclusters="131072" hw.bce.tso_enable=0 hw.pci.enable_msix=0 net.isr.direct_force=1 net.isr.direct=1 kern.ipc.soacceptqueue="1024"
The server is updated to the latest firmware versions.
Any ideas? My next move would be to see if I can get some Intel NICs and see if those work or if its an issue with pfSense..
I have also tested disabling hyperthreading and now even cores 1-3, only leaving core0. Issue remains.