C3K NIC with QinQ interfaces really slow upload
-
We have C3K NIC - ix(4) running on pfsense 2.4.4-p2
Everything works fine, BUT, we have a few QinQ interfaces set up for downstream DSL/Fibre clients and seems that they are having issues with super slow upload speeds. and by that I mean on a 100/100 connection they get 100mbps down and 0.2mbps up and constant "drop outs"
I saw there is on the https://docs.netgate.com/pfsense/en/latest/hardware/tuning-and-troubleshooting-network-cards.html page a mention of thew ix(4) NICS
I see that
kern.ipc.nmbclusters="1000000" kern.ipc.nmbjumbop="524288"
is already in
/boot/loader.conf
This QinQ set up worked fine on our other pfsense box (with some crappy Realtek card) but with this it seems really bad and weird and inconstant.
Does anyone have the ability to point me the right direction of what is the issue and what I can try to potentially resolve this?
-
If that's the only difference there I'd look at hardware offloading. Specifically vlan tagging:
ix2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=e400bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
re2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
VLAN_HWSO maybe looking at what I have here.
Steve
-
Thanks for the reply @stephenw10 are you talking about the "Hardware Checksum Offloading" and "Hardware TCP Segmentation Offloading" in the UI? or is there something else.
My current options used
re0: flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC> metric 0 mtu 1500 options=82098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
and
ix3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=e407bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
Edit: my NICs:
re0@pci0:2:0:0: class=0x020000 card=0x34687470 chip=0x816810ec rev=0x06 hdr=0x00 vendor = 'Realtek Semiconductor Co., Ltd.' device = 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller' class = network subclass = ethernet
and
ix3@pci0:7:0:1: class=0x020000 card=0x00008086 chip=0x15e48086 rev=0x11 hdr=0x00 vendor = 'Intel Corporation' device = 'Ethernet Connection X553 1GbE' class = network subclass = ethernet
And Drivers loaded
grep ix3 /var/run/dmesg.boot ix3: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.2.12-k> mem 0xdd200000-0xdd3fffff,0xdd600000-0xdd603fff at device 0.1 on pci7 ix3: Using MSI-X interrupts with 9 vectors ix3: Ethernet address: ac:1f:6b:b1:d8:af ix3: netmap queues/slots: TX 8/2048, RX 8/2048```
-
You can disable/enable any of those options manually only those that have proved an issue in the past are in the GUI.
However it looks like you have LRO enabled on the ix NIC and that's something we disable by defaullt. I have it disabled there on the 5100, which is C3K. That is a GUI option so disable that first.
Steve