Intel X710 Issues

mloiterman

I cannot say what might be causing your issues, but I can tell you that I enabled Flow Control on the LAN switch ports and enabled full Flow Control on the LAN port on the X710 card. I had Flow Control disabled on the WAN port of the X710 card.

My understanding is the Flow Control is negotiated and must be active on both sides of the connection.

This whole thing is still a bit of a mystery to me and it’s not really clear why it’s needed. I suspect it may be partially rooted in the fact that my switches, HP 2920 48 port with PoE+ are 1Gbs switches with a 10Gbs add in cards. NBASE-T is a newer standard and there maybe bugs or issues like this that are triggered when you have a mixed environment. That’s only speculation from me, however.

Please let me know what you find because I would like to eventually disable Flow Control.

ChuckLasher

@mloiterman

Messing around last night yielded results.

In the GUI, added System Tunable dev.ixl.1.fc=3
Reverted back to the shipped driver (iflib)
Enabled Flow Control on the Netgear switch (XS724EM)
Restarted

Screenshot from 2021-05-25 11-16-25.png

Not entirely sure why this works but there is significance when CE 2.5.1 adds the tunable vs sysctl

In testing, ATV 4K will obtain (has to work up to it) 940 Mbps.

Speeds do not "snap" up to their values. Speedtest seems to 'slew' its way up to the final value.

I suspect there is a better way to handle this but is well above my understanding level. Perhaps priority flow control ?

lnguyen

@mloiterman

IPv6 addresses did eventually populate. I have a Cisco switch that is not negotiating FC with the Intel X710-T4L ports. The default kernel drivers report the correct speed negotiated from ifconfig

lnguyen

@stephenw10

I think this scenario is due to WAN speed >1Gbps overflowing 1Gbps connected LAN hosts.

mloiterman

@chucklasher

Two other things:

I noticed performance was subjectively better when flow control for the X710 lan port to “1” which is rxpause. 3 worked too, but “seemed” less performing. Although, I have no objective data to see if that’s really true or not.

Performance seemed better when I Disabled hardware checksum offload.

ChuckLasher

@mloiterman

Thanks,

Made the change.
I'll play around with the offload. I will see what impact that has on the 10 GbE machines.

lnguyen

2 year old thread. Same issue but with WAN 2x1Gbps LAGG instead of 2.5Gbps
Low throughput over lagg with 1gb clients

lnguyen

I think using ethernet flow control to solve this issue is not an ideal workaround. I think looking at TCP flow control or packet buffer sizes in pfSense is the long term resolution here. Just not sure what to tune in pfSense to accommodate WAN connections greater than 1Gbps LAN connections.

chiapuddingmaster

Is there any update to this issue? I'm using the x710-t2l with FC=1 on lan and a netgear MS510TX multigig switch with global flow control since the switch doesnt offer individual port controls. I would like to turn off FC if possible.

rle

@mloiterman Followed your procedure, but I also had to replace the file completely in boot/kernel otherwise it would not load the newly compiled driver.

Notice the up-to-date nvm and driver combination (latest as of this writing).

22.05-RELEASE (amd64)
built on Wed Jun 22 18:56:13 UTC 2022
FreeBSD 12.3-STABLE

sysctl -a | grep dev.ixl.0

dev.ixl.0.fw_version: fw 9.101.71489 api 1.15 nvm 9.00 etid 8000cec3 oem 0.0.0
dev.ixl.0.current_speed: 10 Gbps
dev.ixl.0.supported_speeds: 6
dev.ixl.0.advertise_speed: 6
dev.ixl.0.fc: 3
dev.ixl.0.%parent: pci5
dev.ixl.0.%pnpinfo: vendor=0x8086 device=0x1572 subvendor=0x8086 subdevice=0x0000 class=0x020000
dev.ixl.0.%location: slot=0 function=0 dbsf=pci0:5:0:0 handle=\_SB_.PCI0.BR3A.H000
dev.ixl.0.%driver: ixl
dev.ixl.0.%desc: Intel(R) Ethernet Connection 700 Series PF Driver, Version - 1.12.35

dmesg | grep ixl0

ixl0: <Intel(R) Ethernet Connection 700 Series PF Driver, Version - 1.12.35> mem 0xf8000000-0xf87fffff,0xf9008000-0xf900ffff irq 16 at device 0.0 on pci5
ixl0: using 1024 tx descriptors and 1024 rx descriptors
ixl0: fw 9.101.71489 api 1.15 nvm 9.00 etid 8000cec3 oem 0.0.0

stephenw10

@rle said in Intel X710 Issues:

I also had to replace the file completely in boot/kernel otherwise it would not load the newly compiled driver.

You could have also added:

if_ixl_name="/boot/modules/if_ixl.ko"

To make it load the new module.

Steve